Compare commits

...

65 Commits

Author SHA1 Message Date
0dc8fc78b4 [Version] 更新版本(2.4.0 -> 2.5.0-20200504.01-SNAPSHOT); 2020-05-04 17:07:47 +08:00
40c5284be2 [Change] Dockerfile.sample 调整构建指令顺序以充分利用构建缓存;
[Fix] Main 修复SystemProperties设置null抛出NPE的问题;
2020-05-04 17:07:39 +08:00
46cb47c7d1 [Add] LICENSE 添加开源许可证(LGPLv3); 2020-05-04 16:55:16 +08:00
fe213deecb [Change] Main 调整参数的接收形式;
[Fix] Dockerfile.sample 修复Java应用无法接收stop命令发送信号的问题;
[Change] RankingUpdateTimer 调整更新投递形式, 将同步更新调整为异步更新;
[Change] BotEventHandler 实装TimeLimitThreadPoolExecutor;
2020-05-04 02:07:46 +08:00
f279d99fda [Add] TimeLimitThreadPoolExecutor 增加一个带有执行时间限制的线程池及对应单元测试类; 2020-05-04 02:04:59 +08:00
a09aef5be2 [Delete] 删除已完成或已经没有存在意义的TODO; 2020-05-02 10:40:11 +08:00
766411fa09 [Fix] Dockerfile.sample 修复应用无法接收容器退出信号的问题(现在应用将可以接受信号主动退出, 即使ExitCode不为0); 2020-05-02 00:56:05 +08:00
84b544cea9 [Change] Main 调整日志输出级别, 补充日志输出内容;
[Change] Dockerfile.sample 调整镜像内应用运行路径;
2020-05-01 22:28:15 +08:00
2f492f5b03 [Add] log4j2.xml, log4j2-test.xml 添加允许指定日志存储目录的功能(sys:cgj.logsPath); 2020-05-01 13:19:08 +08:00
15af939c3f [Fix] PixivDownload 修复因排行榜总量的不确定导致排行榜获取异常的问题;
[Change] BotEventHandler 调整事件处理完成后的日志输出形式;
2020-04-30 19:08:34 +08:00
a28cb142b4 [Change] 调整数据存储的路径设置及存储目录参数名; 2020-04-30 17:51:57 +08:00
e8fda3214b [Version] 更新版本(2.3.0 -> 2.4.0); 2020-04-30 02:24:21 +08:00
fba6d3532c [Delete] 删除不再使用的类和方法;
[Update] 优化Javadoc并设置编译注解, 优化代码;
[Change] 调整部分日志的输出级别;
[Change] RedisPoolCacheStore clear方法将根据返回信息确定是否清空成功;
[Change] Dockerfile.sample 调整镜像构建步骤;
2020-04-30 02:23:31 +08:00
0075446412 [Fix] MiraiMessageSender 修复因BotCode-Image中flashImage参数不存在导致NPE的问题; 2020-04-30 01:54:27 +08:00
2388cd419e [Add] Dockerfile.sample 增加一个用于构建CGJ-BotMode镜像的Dockerfile模板; 2020-04-30 00:45:41 +08:00
7f83b16118 [Add] Main 添加日志输出内容; 2020-04-29 23:44:38 +08:00
10cad32efa [Version] 更新版本(2.2.1 -> 2.3.0); 2020-04-28 23:16:10 +08:00
22a113ef38 [Change] 调整CGJ包结构, 将机器人平台相关类单独存放在framework包内; 2020-04-28 23:15:55 +08:00
6824b12e8f [Add] MiraiMessageSender 增加对闪照的支持; 2020-04-28 10:57:17 +08:00
1c556c5b94 [Update] MiraiMessageSender 对已上传的图片所属ImageId设置10天有效期; 2020-04-28 00:32:40 +08:00
dbc9f4c90b [Fix] BotCommandProcess 修复了通过缓存获取排行榜时排行榜从第2名开始读取的bug;
[Update] MiraiMain, MiraiMessageSender, MiraiMessageEvent 更新Mirai框架(0.32.0 -> 0.39.4)并适配框架新版本的变动;
2020-04-28 00:19:24 +08:00
d3c1975722 [Fix] PixivURL 修复由于智障修复导致的bug; 2020-04-27 23:26:22 +08:00
c09b750fe6 [Version] 更新版本(2.2.0 -> 2.2.1); 2020-04-27 14:13:51 +08:00
ca479ef1af [Change] log4j2.xml 调整SYSTEM_OUT控制台输出配置; 2020-04-27 14:06:33 +08:00
c2c49d2355 [Add] PixivDownload 增加对排行榜获取失败抛出异常的信息;
[Fix] PixivURL.RankingContentType 修复isSupportedMode判断错误的问题;
[Fix] RankingUpdateTimer 调整每日更新时间(12:10 -> 12:30)以尝试修复排行榜更新时机过早的问题;
2020-04-27 12:32:25 +08:00
22e74e8cd5 [Add] PixivIllustType 增加了对IllustInfo的解析; 2020-04-26 15:53:39 +08:00
d549c5674d [Update] BotCommandProcess 补充Javadoc;
[Change] PixivUgoiraBuilder 调整日志输出级别;
2020-04-26 15:45:23 +08:00
c6952de84c [Add] log4j2-test.xml 增加测试运行时的日志配置文件;
[Add] pom.xml 设置package goal的文件排除项, 将测试运行所使用的日志配置文件(log4j2-test.xml)排除;
2020-04-25 17:07:13 +08:00
410d6c0828 [Add] BotAdminCommandProcess, PixivDownload 增加日志输出内容;
[Change] RandomIntervalSendTimer 对AutoSender增加异常捕获以减少对其他Timer执行的影响;
2020-04-25 17:02:01 +08:00
aaa1bc932b [Add] RandomRankingArtworksSender 增加参数;
[Add] BotAdminCommandProcess 增加更多可自定义的参数项, 增加json配置项常量;
2020-04-25 02:29:38 +08:00
a1e54e70d9 [Change] PixivURL 调整RankingContentType.ALL名称(ALL -> TYPE_ALL);
[Add] PixivDownload 增加调试日志信息;
[Fix] RandomIntervalSendTimer 修复cancel方法无法停止定时器的问题;
2020-04-24 23:54:26 +08:00
16522155e1 [Add] 通过增加参数信息来为命令方法提供来源群和来源QQ; 2020-04-24 23:52:05 +08:00
5f796f7da0 [Update] BotCommandProcess 更新帮助信息, 补充Javadoc, 调整命令名;
[Change] RandomIntervalSendTimer 将下一执行时间从分钟调整为具体时间;
[Change] MessageEvent toString方法调整为输出具体实现类名而不是"MessageEvent";
2020-04-24 10:45:03 +08:00
5a52dd9208 [Add] 初步添加成人内容检测器, 尚未使用; 2020-04-24 01:42:47 +08:00
b53aafa81b [Fix] RandomRankingArtworksSender 修复排行榜选取错误的问题;
[Add] RandomRankingArtworksSender 增加对rankingStart, rankingStop的范围选取检查;
[Update] RandomRankingArtworksSender 更新Javadoc;
2020-04-24 01:41:53 +08:00
28aa086f15 [Version] 更新版本(2.1.0 -> 2.2.0);
[Change] groupId纳入net.lamgc;
2020-04-24 01:28:06 +08:00
0eadefa74f [Add] BotAdminCommandProcess 完善作品报告管理功能;
[Add] BotCommandProcess 对Report增加报告时间;
2020-04-24 01:24:03 +08:00
4afa414725 [Add] CacheStore 增加keys, remove方法;
[Update] HotDataCacheStore, LocalHashCacheStore, RedisPoolCacheStore 适配CacheStore的更改;
2020-04-24 01:23:11 +08:00
0f202cb076 [Update] RedisPoolCacheStore 优化代码; 2020-04-24 00:42:41 +08:00
1f3d99ac10 [Add] BotCommandProcess 增加作品报告功能, 以允许用户主动报告不当作品;
[Update] 适配报告功能;
2020-04-24 00:34:50 +08:00
d33f4028d1 [Fix] BotEventHandler 修复无命令帮助信息无法发送的问题(前缀识别失效); 2020-04-24 00:33:51 +08:00
4020bbfea8 [Change] BotAdminCommandProcess 调整方法名, 增加提示信息; 2020-04-23 23:18:41 +08:00
19605a9401 [Add] BotEventHandler 增加启动预处理方法;
[Add] BotAdminCommandProcess 增加推送功能相关管理命令;
[Change] RandomIntervalSendTimer 调整Timer管理过程;
[Change] BotCommandProcess ranking方法对参数错误的处理调整为返回错误信息;
[Fix] RankingUpdateTimer 修复参数错误的问题;
2020-04-23 16:23:06 +08:00
21466a49f9 [Update] pom.xml 补充插件版本;
[Delete] pom.xml 删除kotlin编译插件;
2020-04-23 09:23:26 +08:00
73ae9a268b [Add] MessageSenderBuilder, MessageSenderFactory 增加用于创建MessageSender而又与平台无关的Builder, 初步增加对应Factory; 2020-04-21 22:43:24 +08:00
620c3785ad [Change] MessageSender 调整包路径; 2020-04-21 22:33:20 +08:00
2df5513727 [Change] BotEventHandler 修改命令前缀以防止错误的触发; 2020-04-20 01:35:02 +08:00
8798633c2c [Fix] BotCommandProcess 修复排行榜数据选取错误的问题; 2020-04-20 01:33:30 +08:00
9dfc20a525 [Add] AutoSender 增加自动发送器接口;
[Add] RandomIntervalSendTimer 增加随机延迟自动发送器;
[Add] RecommendArtworksSender 增加随机排行榜作品发送器;
[Change] PixivDownload 重新开发 getRanking 方法;
2020-04-20 01:32:11 +08:00
dd88f2acab [Update] MiraiMessageSender 增加新构造器, 补充Javadoc; 2020-04-19 23:58:33 +08:00
36460e4c34 [Add] MiraiMain 增加正常关闭过程; 2020-04-19 13:03:10 +08:00
084be3970a [Change] 将各MessageEvent的消息发送部分分离成单独的MessageSender, 将MessageEvent依赖于对应的MessageSender; 2020-04-19 12:23:02 +08:00
11005b0f6c [Add] BotCode 增加 getCodePattern 方法以允许其他类获取BotCode所使用的正则匹配表达式; 2020-04-19 12:20:58 +08:00
970be847a0 [Change] RankingUpdateTimer 调整更新时间(12:30 -> 12:10);
[Change] RankingUpdateTimer 调整日志级别, 补充日志输出信息;
2020-04-19 12:16:03 +08:00
09fa1bd2e8 [Add] MessageSource 将MiraiMessageEvent的MessageSource分离成独立的类;
[Change] SpringCQMessageEvent, MiraiMessageEvent 适配新的更改;
2020-04-19 01:34:45 +08:00
ae27141fea [Add] MessageSender 增加消息发送器接口;
[Change] MessageEvent 实现 MessageSender 接口;
2020-04-19 01:00:52 +08:00
b328def8f9 [Change] BotCommandProcess 将isNoSafe方法公开;
[Change] MiraiMessageEvent 将UploadImage方法公开;
[Add] MiraiMessageEvent 增加MessageSource枚举类;
2020-04-19 00:31:24 +08:00
dbfed874c0 [Add] VirtualLoadMessageEvent 增加假负载消息事件, 该事件在消息发送上将没有操作, 纯属执行命令;
[Change] RankingUpdateTimer 调整排行榜更新方式, 利用假负载消息触发一次完全执行;
2020-04-18 01:15:04 +08:00
f02b0e9e98 [Update] 补充Javadoc, 设置@SuppressWarning注解; 2020-04-18 01:13:01 +08:00
260cfe3dd0 [Delete] MessageEvent 删除可能会导致兼容性降低的未使用方法;
[Delete] MiraiMessageEvent, SpringCQMessageEvent 适配 MessageEvent 的更改;
2020-04-17 18:29:59 +08:00
c1427379c6 [Add] StringListRedisCacheStore 增加类型为String的RedisListCacheStore实现;
[Change] BotCommandProcess 更换pagesCache的缓存组件(RedisPoolCacheStore<List<String>> -> StringListRedisCacheStore);
2020-04-17 18:25:48 +08:00
f844d150e8 [Change] 更改artifactId(CGJ_2 -> ContentGrabbingJi); 2020-04-17 17:46:42 +08:00
2ec696b3cd [Change] PreLoadDataComparator Attribute类属性attrName访问权修改(default -> public); 2020-04-17 17:46:01 +08:00
c8c6dbe4fd [Update] 优化Logger名称;
[Update] 为 processMessage(MessageEvent) 方法增加 @SuppressWarnings("unused") 注解并补充Javadoc;
2020-04-17 17:20:37 +08:00
4dff477ded [Change] log4j2.xml 调整Root Logger最低日志级别(Debug -> Info); 2020-04-17 15:54:13 +08:00
52 changed files with 1885 additions and 1059 deletions

10
Dockerfile.sample Normal file
View File

@ -0,0 +1,10 @@
FROM openjdk:8u252-jre
ENV jarFileName=ContentGrabbingJi-exec.jar
ENV CGJ_REDIS_URI="127.0.0.1:6379"
ENV CGJ_PROXY=""
RUN mkdir /data/
COPY ${jarFileName} /CGJ.jar
CMD ["java", "-Dcgj.logsPath=/data/logs", "-jar", "/CGJ.jar", "botMode", "-botDataDir=/data"]

165
LICENSE Normal file
View File

@ -0,0 +1,165 @@
GNU LESSER GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
This version of the GNU Lesser General Public License incorporates
the terms and conditions of version 3 of the GNU General Public
License, supplemented by the additional permissions listed below.
0. Additional Definitions.
As used herein, "this License" refers to version 3 of the GNU Lesser
General Public License, and the "GNU GPL" refers to version 3 of the GNU
General Public License.
"The Library" refers to a covered work governed by this License,
other than an Application or a Combined Work as defined below.
An "Application" is any work that makes use of an interface provided
by the Library, but which is not otherwise based on the Library.
Defining a subclass of a class defined by the Library is deemed a mode
of using an interface provided by the Library.
A "Combined Work" is a work produced by combining or linking an
Application with the Library. The particular version of the Library
with which the Combined Work was made is also called the "Linked
Version".
The "Minimal Corresponding Source" for a Combined Work means the
Corresponding Source for the Combined Work, excluding any source code
for portions of the Combined Work that, considered in isolation, are
based on the Application, and not on the Linked Version.
The "Corresponding Application Code" for a Combined Work means the
object code and/or source code for the Application, including any data
and utility programs needed for reproducing the Combined Work from the
Application, but excluding the System Libraries of the Combined Work.
1. Exception to Section 3 of the GNU GPL.
You may convey a covered work under sections 3 and 4 of this License
without being bound by section 3 of the GNU GPL.
2. Conveying Modified Versions.
If you modify a copy of the Library, and, in your modifications, a
facility refers to a function or data to be supplied by an Application
that uses the facility (other than as an argument passed when the
facility is invoked), then you may convey a copy of the modified
version:
a) under this License, provided that you make a good faith effort to
ensure that, in the event an Application does not supply the
function or data, the facility still operates, and performs
whatever part of its purpose remains meaningful, or
b) under the GNU GPL, with none of the additional permissions of
this License applicable to that copy.
3. Object Code Incorporating Material from Library Header Files.
The object code form of an Application may incorporate material from
a header file that is part of the Library. You may convey such object
code under terms of your choice, provided that, if the incorporated
material is not limited to numerical parameters, data structure
layouts and accessors, or small macros, inline functions and templates
(ten or fewer lines in length), you do both of the following:
a) Give prominent notice with each copy of the object code that the
Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the object code with a copy of the GNU GPL and this license
document.
4. Combined Works.
You may convey a Combined Work under terms of your choice that,
taken together, effectively do not restrict modification of the
portions of the Library contained in the Combined Work and reverse
engineering for debugging such modifications, if you also do each of
the following:
a) Give prominent notice with each copy of the Combined Work that
the Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the Combined Work with a copy of the GNU GPL and this license
document.
c) For a Combined Work that displays copyright notices during
execution, include the copyright notice for the Library among
these notices, as well as a reference directing the user to the
copies of the GNU GPL and this license document.
d) Do one of the following:
0) Convey the Minimal Corresponding Source under the terms of this
License, and the Corresponding Application Code in a form
suitable for, and under terms that permit, the user to
recombine or relink the Application with a modified version of
the Linked Version to produce a modified Combined Work, in the
manner specified by section 6 of the GNU GPL for conveying
Corresponding Source.
1) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (a) uses at run time
a copy of the Library already present on the user's computer
system, and (b) will operate properly with a modified version
of the Library that is interface-compatible with the Linked
Version.
e) Provide Installation Information, but only if you would otherwise
be required to provide such information under section 6 of the
GNU GPL, and only to the extent that such information is
necessary to install and execute a modified version of the
Combined Work produced by recombining or relinking the
Application with a modified version of the Linked Version. (If
you use option 4d0, the Installation Information must accompany
the Minimal Corresponding Source and Corresponding Application
Code. If you use option 4d1, you must provide the Installation
Information in the manner specified by section 6 of the GNU GPL
for conveying Corresponding Source.)
5. Combined Libraries.
You may place library facilities that are a work based on the
Library side by side in a single library together with other library
facilities that are not Applications and are not covered by this
License, and convey such a combined library under terms of your
choice, if you do both of the following:
a) Accompany the combined library with a copy of the same work based
on the Library, uncombined with any other library facilities,
conveyed under the terms of this License.
b) Give prominent notice with the combined library that part of it
is a work based on the Library, and explaining where to find the
accompanying uncombined form of the same work.
6. Revised Versions of the GNU Lesser General Public License.
The Free Software Foundation may publish revised and/or new versions
of the GNU Lesser General Public License from time to time. Such new
versions will be similar in spirit to the present version, but may
differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the
Library as you received it specifies that a certain numbered version
of the GNU Lesser General Public License "or any later version"
applies to it, you have the option of following the terms and
conditions either of that published version or of any later version
published by the Free Software Foundation. If the Library as you
received it does not specify a version number of the GNU Lesser
General Public License, you may choose any version of the GNU Lesser
General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide
whether future versions of the GNU Lesser General Public License shall
apply, that proxy's public statement of acceptance of any version is
permanent authorization for you to choose that version for the
Library.

38
pom.xml
View File

@ -4,9 +4,9 @@
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>CGJ_2</artifactId>
<version>2.1.0</version>
<groupId>net.lamgc</groupId>
<artifactId>ContentGrabbingJi</artifactId>
<version>2.5.0-20200504.01-SNAPSHOT</version>
<repositories>
<repository>
@ -19,7 +19,7 @@
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<maven.compiler.encoding>UTF-8</maven.compiler.encoding>
<mirai.CoreVersion>0.32.0</mirai.CoreVersion>
<mirai.CoreVersion>0.39.4</mirai.CoreVersion>
<mirai.JaptVersion>1.1.1</mirai.JaptVersion>
<kotlin.version>1.3.71</kotlin.version>
<ktor.version>1.3.2</ktor.version>
@ -30,14 +30,26 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<source>8</source>
<target>8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.0.2</version>
<configuration>
<excludes>
<exclude>log4j2-test.xml</exclude>
</excludes>
</configuration>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>2.2.6.RELEASE</version>
<configuration>
<mainClass>net.lamgc.cgj.Main</mainClass>
<classifier>exec</classifier>
@ -50,20 +62,6 @@
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-maven-plugin</artifactId>
<version>${kotlin.version}</version>
<executions>
<execution>
<id>compile</id>
<phase>process-sources</phase>
<goals>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
@ -153,12 +151,12 @@
</dependency>
<dependency>
<groupId>net.mamoe</groupId>
<artifactId>mirai-core-jvm</artifactId>
<artifactId>mirai-core</artifactId>
<version>${mirai.CoreVersion}</version>
</dependency>
<dependency>
<groupId>net.mamoe</groupId>
<artifactId>mirai-core-qqandroid-jvm</artifactId>
<artifactId>mirai-core-qqandroid</artifactId>
<version>${mirai.CoreVersion}</version>
</dependency>
<dependency>

View File

@ -7,10 +7,11 @@ import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.CQConfig;
import net.lamgc.cgj.bot.MiraiMain;
import net.lamgc.cgj.pixiv.*;
import net.lamgc.cgj.proxy.PixivAccessProxyServer;
import net.lamgc.cgj.bot.framework.coolq.CQConfig;
import net.lamgc.cgj.bot.framework.mirai.MiraiMain;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivSearchBuilder;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.plps.PixivLoginProxyServer;
import net.lamgc.utils.base.ArgumentsProperties;
import net.lamgc.utils.base.runner.Argument;
@ -20,9 +21,7 @@ import org.apache.commons.io.IOUtils;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.client.CookieStore;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.message.BasicHeader;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@ -52,9 +51,19 @@ public class Main {
public static HttpHost proxy;
public static void main(String[] args) throws IOException, ClassNotFoundException {
log.trace("ContentGrabbingJi 正在启动...");
log.debug("Args: {}, LogsPath: {}", Arrays.toString(args), System.getProperty("cgj.logsPath"));
log.debug("运行目录: {}", System.getProperty("user.dir"));
ArgumentsProperties argsProp = new ArgumentsProperties(args);
if(argsProp.containsKey("proxy")) {
URL proxyUrl = new URL(argsProp.getValue("proxy"));
if(!getSettingToSysProp(argsProp, "proxy", null)) {
getEnvSettingToSysProp("CGJ_PROXY", "proxy", null);
}
String proxyAddress = System.getProperty("cgj.proxy");
if(!Strings.isNullOrEmpty(proxyAddress)) {
URL proxyUrl = new URL(proxyAddress);
proxy = new HttpHost(proxyUrl.getHost(), proxyUrl.getPort());
log.info("已启用Http协议代理{}", proxy.toHostString());
} else {
@ -65,28 +74,20 @@ public class Main {
log.error("创建文件夹失败!");
}
// TODO: 需要修改参数名了, 大概改成类似于 workerDir这样的吧
if(argsProp.containsKey("cqRootDir")) {
log.info("cqRootDir: {}", argsProp.getValue("cqRootDir"));
System.setProperty("cgj.cqRootDir", argsProp.getValue("cqRootDir"));
} else {
log.warn("未设置cqRootDir, 当前运行目录将作为酷Q机器人所在目录.");
System.setProperty("cgj.cqRootDir", "./");
if(!getSettingToSysProp(argsProp, "botDataDir", "./") &&
!getEnvSettingToSysProp("CGJ_BOT_DATA_DIR", "botDataDir", "./")) {
log.warn("未设置botDataDir, 当前运行目录将作为酷Q机器人所在目录.");
}
if(!getSettingToSysProp(argsProp, "redisAddress", "127.0.0.1") &&
!getEnvSettingToSysProp("CGJ_REDIS_URI", "redisAddress", "127.0.0.1")) {
log.warn("未设置RedisAddress, 将使用默认值连接Redis服务器(127.0.0.1:6379)");
}
if(argsProp.containsKey("redisAddr")) {
log.info("redisAddress: {}", argsProp.getValue("redisAddr"));
System.setProperty("cgj.redisAddress", argsProp.getValue("redisAddr"));
} else {
log.info("未设置RedisAddress, 将使用默认值连接Redis服务器(127.0.0.1:6379)");
System.setProperty("cgj.redisAddress", "127.0.0.1");
}
File cookieStoreFile = new File("cookies.store");
File cookieStoreFile = new File(System.getProperty("cgj.botDataDir"), "cookies.store");
if(!cookieStoreFile.exists()) {
log.error("未找到cookies.store文件, 是否启动PixivLoginProxyServer? (yes/no)");
log.warn("未找到cookies.store文件, 是否启动PixivLoginProxyServer? (yes/no)");
Scanner scanner = new Scanner(System.in);
if(scanner.nextLine().equalsIgnoreCase("yes")) {
if(scanner.nextLine().trim().equalsIgnoreCase("yes")) {
startPixivLoginProxyServer();
} else {
System.exit(1);
@ -103,6 +104,37 @@ public class Main {
ArgumentsRunner.run(Main.class, args);
}
/**
* 从ArgumentsProperties获取设置项到System Properties
* @param prop ArgumentsProperties对象
* @param key 设置项key
* @param defaultValue 默认值
* @return 如果成功从ArgumentsProperties获得设置项, 返回true, 如未找到(使用了defaultValue或null), 返回false;
*/
private static boolean getSettingToSysProp(ArgumentsProperties prop, String key, String defaultValue) {
if(prop.containsKey(key)) {
log.info("{}: {}", key, prop.getValue(key));
System.setProperty("cgj." + key, prop.getValue(key));
return true;
} else {
if(defaultValue != null) {
System.setProperty("cgj." + key, defaultValue);
}
return false;
}
}
private static boolean getEnvSettingToSysProp(String envKey, String sysPropKey, String defaultValue) {
String env = System.getenv(envKey);
if(env != null) {
System.setProperty("cgj." + sysPropKey, env);
return true;
} else if(defaultValue != null) {
System.setProperty("cgj." + sysPropKey, defaultValue);
}
return false;
}
@Command
public static void botMode(@Argument(name = "args", force = false) String argsStr) {
new MiraiMain().init();
@ -110,10 +142,10 @@ public class Main {
@Command
public static void pluginMode(@Argument(name = "args", force = false) String argsStr) {
if(!System.getProperty("cgj.cqRootDir").endsWith("\\") && !System.getProperty("cgj.cqRootDir").endsWith("/")) {
System.setProperty("cgj.cqRootDir", System.getProperty("cgj.cqRootDir") + "/");
if(!System.getProperty("cgj.botDataDir").endsWith("\\") && !System.getProperty("cgj.botDataDir").endsWith("/")) {
System.setProperty("cgj.botDataDir", System.getProperty("cgj.botDataDir") + "/");
}
log.info("酷Q机器人根目录: {}", System.getProperty("cgj.cqRootDir"));
log.info("酷Q机器人根目录: {}", System.getProperty("cgj.botDataDir"));
CQConfig.init();
Pattern pattern = Pattern.compile("/\\s*(\".+?\"|[^:\\s])+((\\s*:\\s*(\".+?\"|[^\\s])+)|)|(\".+?\"|[^\"\\s])+");
Matcher matcher = pattern.matcher(Strings.nullToEmpty(argsStr));
@ -353,89 +385,19 @@ public class Main {
builder,
PixivURL.getPixivRefererLink(illustId)
);
/*log.info("正在下载...");
List<String> list = PixivDownload.getIllustAllPageDownload(
HttpClientBuilder.create()
.setProxy(proxy)
.build(),
illustId, PixivDownload.PageQuality.ORIGINAL);*/
}
}
/*
if(searchBuilder.getSearchArea().equals(PixivSearchBuilder.SearchArea.TOP)) {
} else {
JsonArray illustsArray = resultBody
.getAsJsonObject(searchBuilder.getSearchArea().jsonKey).getAsJsonArray("data");
log.info("已找到与 {} 相关插图信息:", content);
int count = 1;
for (JsonElement jsonElement : illustsArray) {
JsonObject illustObj = jsonElement.getAsJsonObject();
//TODO: 防止数据内混入无效内容, 需要检查对象是否有illustId
if(!illustObj.has("illustId")) {
continue;
}
int illustId = illustObj.get("illustId").getAsInt();
StringBuilder builder = new StringBuilder("[");
illustObj.get("tags").getAsJsonArray().forEach(el -> builder.append(el.getAsString()).append(", "));
builder.replace(builder.length() - 2, builder.length(), "]");
log.info("({} / {})\n\t作品id: {}, \n\t作者名(作者id): {} ({}), \n\t作品标题: {}, \n\t作品Tags: {}, \n\t作品链接: {}",
count++,
illustsArray.size(),
illustId,
illustObj.get("userName").getAsString(),
illustObj.get("userId").getAsInt(),
illustObj.get("illustTitle").getAsString(),
builder,
PixivURL.getPixivRefererLink(illustId)
);
}
}
*/
}
@Command(defaultCommand = true)
public static void testRun() {
/*loadCookieStoreFromFile();
if(cookieStore == null){
startPixivLoginProxyServer();
}*/
//accessPixivToFile();
//startPixivAccessProxyServer();
//saveCookieStoreToFile();
log.info("这里啥都没有哟w");
}
private static void loadCookieStoreFromFile() throws IOException {
log.info("正在加载CookieStore...");
File storeFile = new File("./cookies.store");
if(!storeFile.exists()){
log.info("未找到CookieStore, 跳过加载.");
return;
}
ObjectInputStream stream = new ObjectInputStream(new FileInputStream(storeFile));
Object result;
try {
result = stream.readObject();
} catch (ClassNotFoundException e) {
log.error("加载出错", e);
return;
}
cookieStore = (CookieStore) result;
cookieStore.getCookies().forEach(cookie -> log.debug(cookie.getName() + ": " + cookie.getValue() + ", isExpired: " + cookie.isExpired(new Date())));
log.info("CookieStore加载完成.");
}
private static void saveCookieStoreToFile() throws IOException {
log.info("正在保存CookieStore...");
File outputFile = new File("./cookies.store");
File outputFile = new File(System.getProperty("cgj.botDataDir"), "cookies.store");
if(!outputFile.exists() && !outputFile.delete() && !outputFile.createNewFile()){
log.error("保存CookieStore失败.");
return;
@ -482,75 +444,4 @@ public class Main {
}
}
private static void accessPixivToFile() throws IOException {
File cookieStoreFile = new File("./cookie.txt");
if (!cookieStoreFile.exists() && !cookieStoreFile.createNewFile()) {
log.info("Cookie文件存储失败");
}
/*log.info("正在写出Cookie, Cookie count: " + cookieStore.getCookies().size());
FileWriter cookieWriter = new FileWriter(cookieStoreFile);
cookieStore.getCookies().forEach(cookie -> {
try {
StringBuilder sb = new StringBuilder().append(cookie.getName()).append(" = ").append(cookie.getValue());
log.info("正在导出Cookie: " + sb.toString());
cookieWriter.append(sb.toString()).append("\n").flush();
} catch (IOException e) {
e.printStackTrace();
}
});
log.info("Cookie写出完成");*/
log.info("尝试通过捕获的Cookie访问Pixiv...");
HttpClient httpClient = new PixivSession(new HttpHost("127.0.0.1", 1080), cookieStore).getHttpClient();
HttpGet request = new HttpGet("https://www.pixiv.net");
request.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:73.0) Gecko/20100101 Firefox/73.0");
request.addHeader(new BasicHeader("accept-encoding", "gzip, deflate, br"));
request.addHeader(new BasicHeader("accept-language", "zh-CN,zh;q=0.9"));
StringBuilder cookieBuilder = new StringBuilder();
cookieStore.getCookies().forEach(cookie -> {
if(cookie.isExpired(new Date())){
return;
}
cookieBuilder.append(cookie.getName()).append("=").append(cookie.getValue()).append("; ");
});
request.addHeader("cookie", cookieBuilder.toString());
HttpResponse response = httpClient.execute(request);
log.info("正在写入文件...");
File outFile = new File("./pixiv.html");
if (outFile.createNewFile() && !outFile.exists()) {
log.info("文件创建失败!");
}else {
new FileWriter(outFile).append(EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8)).flush();
}
Pixiv pixiv = new Pixiv(httpClient);
pixiv.getRecommend().forEach(illustMap -> {
StringBuilder builder = new StringBuilder();
illustMap.forEach((key, value) -> builder.append(key).append(": ").append(value).append("\n"));
try {
builder.append("download Link: ").append(Arrays.toString(pixiv.getAllDownloadLink(Integer.parseInt(illustMap.get(Pixiv.ATTR_ILLUST_ID)))));
} catch (IOException e) {
log.error("获取下载链接时出错!", e);
}
log.info(builder.append("\n").toString());
});
}
private static void startPixivAccessProxyServer(){
log.info("正在启动访问代理服务器, 将浏览器相关缓存清空后, 使用浏览器进行访问以尝试Cookie正确性.");
final PixivAccessProxyServer accessProxyServer = new PixivAccessProxyServer(cookieStore, new ProxyConfig(ProxyType.SOCKS5, "127.0.0.1", 1080));
Thread accessProxyServerThread = new Thread(() -> {
log.info("正在启动PAPS...");
accessProxyServer.start(1007);
log.info("PAPS已关闭.");
});
accessProxyServerThread.setName("AccessProxyServerThread");
accessProxyServerThread.start();
new Scanner(System.in).nextLine();
log.info("关闭PAPS服务器...");
accessProxyServer.close();
}
}

View File

@ -0,0 +1,32 @@
package net.lamgc.cgj.bot;
import net.lamgc.cgj.bot.message.MessageSender;
import java.util.Objects;
/**
* 自动发送器
*/
public abstract class AutoSender {
private MessageSender messageSender;
/**
* 构造一个自动发送器
* @param messageSender 自动发送器所使用的消息发送器
*/
public AutoSender(MessageSender messageSender) {
this.messageSender = Objects.requireNonNull(messageSender);
}
/**
* 获取设置等等消息发送器
* @return 消息发送器
*/
MessageSender getMessageSender() {
return this.messageSender;
}
public abstract void send();
}

View File

@ -1,29 +1,45 @@
package net.lamgc.cgj.bot;
import com.google.common.base.Strings;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonObject;
import com.google.gson.reflect.TypeToken;
import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.utils.base.runner.Argument;
import net.lamgc.utils.base.runner.Command;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.*;
import java.util.Date;
import java.util.Properties;
import java.text.SimpleDateFormat;
import java.util.*;
public class BotAdminCommandProcess {
private final static Logger log = LoggerFactory.getLogger(BotAdminCommandProcess.class.getSimpleName());
private final static File globalPropFile = new File("./global.properties");
private final static File globalPropFile = new File(System.getProperty("cgj.botDataDir"), "global.properties");
private final static File pushListFile = new File(System.getProperty("cgj.botDataDir"), "pushList.json");
private final static Hashtable<Long, JsonObject> pushInfoMap = new Hashtable<>();
private final static Gson gson = new GsonBuilder()
.setPrettyPrinting()
.create();
@Command
public String clearCache() {
public static String cleanCache() {
BotCommandProcess.clearCache();
return "操作已完成.";
}
@Command
public String setGlobalProperty(@Argument(name = "key") String key, @Argument(name = "value") String value, @Argument(name = "save", force = false) boolean saveNow) {
public static String setGlobalProperty(@Argument(name = "key") String key, @Argument(name = "value") String value, @Argument(name = "save", force = false) boolean saveNow) {
String lastValue = BotCommandProcess.globalProp.getProperty(key);
BotCommandProcess.globalProp.setProperty(key, Strings.nullToEmpty(value));
if(saveNow) {
@ -33,12 +49,12 @@ public class BotAdminCommandProcess {
}
@Command
public String getGlobalProperty(@Argument(name = "key") String key) {
public static String getGlobalProperty(@Argument(name = "key") String key) {
return "全局配置项 " + key + " 当前值: " + BotCommandProcess.globalProp.getProperty(key, "(Empty)");
}
@Command
public String saveGlobalProperties() {
public static String saveGlobalProperties() {
log.info("正在保存全局配置文件...");
try {
@ -58,7 +74,7 @@ public class BotAdminCommandProcess {
}
@Command
public String loadGlobalProperties(@Argument(name = "reload", force = false) boolean reload) {
public static String loadGlobalProperties(@Argument(name = "reload", force = false) boolean reload) {
Properties cache = new Properties();
if(!globalPropFile.exists()) {
return "未找到全局配置文件, 无法重载";
@ -79,7 +95,7 @@ public class BotAdminCommandProcess {
}
@Command
public String runUpdateTask(@Argument(force = false, name = "date") Date queryTime) {
public static String runUpdateTask(@Argument(force = false, name = "date") Date queryTime) {
try {
BotCommandProcess.runUpdateTimer(queryTime);
} catch (Exception e) {
@ -89,4 +105,227 @@ public class BotAdminCommandProcess {
return "操作已完成.";
}
private final static String RANKING_SETTING_TIME_MIN = "time.min";
private final static String RANKING_SETTING_TIME_FLOAT = "time.float";
private final static String RANKING_SETTING_RANKING_START = "ranking.start";
private final static String RANKING_SETTING_RANKING_END = "ranking.end";
private final static String RANKING_SETTING_RANKING_MODE = "ranking.mode";
private final static String RANKING_SETTING_RANKING_CONTENT_TYPE = "ranking.contentType";
private final static String RANKING_SETTING_PAGE_QUALITY = "page.quality";
@Command
public static String addPushGroup(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "group", force = false, defaultValue = "0") long groupId,
@Argument(name = "minTime", force = false, defaultValue = "21600000") long minTime,
@Argument(name = "floatTime", force = false, defaultValue = "10800000") int floatTime,
@Argument(name = "rankingStart", force = false, defaultValue = "1") int rankingStart,
@Argument(name = "rankingStop", force = false, defaultValue = "150") int rankingStop,
@Argument(name = "mode", force = false, defaultValue = "DAILY") String rankingMode,
@Argument(name = "type", force = false, defaultValue = "ILLUST") String rankingContentType,
@Argument(name = "original", force = false, defaultValue = "false") boolean original
) {
long group = groupId <= 0 ? fromGroup : groupId;
JsonObject setting = new JsonObject();
setting.addProperty(RANKING_SETTING_TIME_MIN, minTime);
setting.addProperty(RANKING_SETTING_TIME_FLOAT, floatTime);
setting.addProperty(RANKING_SETTING_RANKING_START, rankingStart);
setting.addProperty(RANKING_SETTING_RANKING_END, rankingStop);
setting.addProperty(RANKING_SETTING_RANKING_MODE, rankingMode);
setting.addProperty(RANKING_SETTING_RANKING_CONTENT_TYPE, rankingContentType);
setting.addProperty(RANKING_SETTING_PAGE_QUALITY, original ?
PixivDownload.PageQuality.ORIGINAL.name() :
PixivDownload.PageQuality.REGULAR.name());
if(pushInfoMap.containsKey(group)) {
log.info("群 {} 已存在Timer, 删除Timer...", group);
removePushGroup(fromGroup, groupId);
}
log.info("正在增加Timer...(Setting: {})", setting);
pushInfoMap.put(group, setting);
addPushTimer(group, setting);
return "已在 " + group + " 开启定时推送功能。";
}
/**
* 重载推送列表
*/
@Command
public static String loadPushList() {
pushInfoMap.clear();
if(!pushListFile.exists()) {
log.warn("推送列表文件不存在, 跳过加载.");
return "文件不存在, 跳过加载.";
}
try (Reader reader = new BufferedReader(new FileReader(pushListFile))) {
pushInfoMap.putAll(gson.fromJson(reader, new TypeToken<Map<Long, JsonObject>>(){}.getType()));
loadAllPushTimer(false);
return "列表重载完成";
} catch (IOException e) {
log.error("重载推送列表时发生错误", e);
return "加载时发生异常";
}
}
@Command
public static String savePushList() {
try {
if(!pushListFile.exists()) {
pushListFile.createNewFile();
}
} catch (IOException e) {
log.error("PushList.json文件创建失败", e);
return "保存失败!请检查控制台信息.";
}
try (Writer writer = new FileWriter(pushListFile)) {
writer.write(gson.toJson(pushInfoMap));
return "保存成功.";
} catch (IOException e) {
log.error("写入PushList.json文件失败!", e);
return "保存失败!请检查控制台信息.";
}
}
/**
* 加载所有推送Timer
* @param flush 是否完全重载, 如为true则加载前会删除所有已加载的Timer
*/
public static void loadAllPushTimer(boolean flush) {
if(flush) {
RandomIntervalSendTimer.timerIdSet().forEach(id -> RandomIntervalSendTimer.getTimerById(id).destroy());
} else {
cleanPushTimer();
}
pushInfoMap.forEach(BotAdminCommandProcess::addPushTimer);
}
/**
* 根据设置增加Timer
* @param id 群组id
* @param setting jsonObject设置集
*/
private static void addPushTimer(long id, JsonObject setting) {
try {
RandomIntervalSendTimer.getTimerById(id);
return;
} catch(NoSuchElementException ignored) {
}
int rankingStart = setting.has(RANKING_SETTING_RANKING_START) ? setting.get(RANKING_SETTING_RANKING_START).getAsInt() : 1;
int rankingEnd = setting.has(RANKING_SETTING_RANKING_END) ? setting.get(RANKING_SETTING_RANKING_END).getAsInt() : 150;
PixivURL.RankingMode rankingMode = PixivURL.RankingMode.MODE_DAILY;
PixivURL.RankingContentType rankingContentType = PixivURL.RankingContentType.TYPE_ILLUST;
PixivDownload.PageQuality pageQuality = PixivDownload.PageQuality.REGULAR;
if(rankingStart <= 0 || rankingStart > 500) {
log.warn("群组 [{}] - 无效的RankingStart设定值, 将重置为默认设定值(1): {}", id, rankingStart);
rankingStart = 1;
} else if(rankingEnd > 500 || rankingEnd <= 0) {
log.warn("群组 [{}] - 无效的RankingEnd设定值, 将重置为默认设定值(150): {}", id, rankingEnd);
rankingEnd = 150;
} else if(rankingStart > rankingEnd) {
log.warn("群组 [{}] - 无效的排行榜选取范围, 将重置为默认设定值(1 ~ 150): start={}, end={}", id, rankingStart, rankingEnd);
rankingStart = 1;
rankingEnd = 150;
}
if(setting.has(RANKING_SETTING_RANKING_MODE)) {
String value = setting.get(RANKING_SETTING_RANKING_MODE).getAsString().trim().toUpperCase();
try {
rankingMode = PixivURL.RankingMode.valueOf(value.startsWith("MODE_") ? value : "MODE_" + value);
} catch(IllegalArgumentException e) {
log.warn("群组ID [{}] - 无效的RankingMode设定值, 将重置为默认值: {}", id, value);
}
}
if(setting.has(RANKING_SETTING_RANKING_CONTENT_TYPE)) {
String value = setting.get(RANKING_SETTING_RANKING_CONTENT_TYPE).getAsString().trim().toUpperCase();
try {
rankingContentType = PixivURL.RankingContentType.valueOf(value.startsWith("TYPE_") ? value : "TYPE_" + value);
} catch(IllegalArgumentException e) {
log.warn("群组ID [{}] - 无效的RankingContentType设定值: {}", id, value);
}
}
if(setting.has(RANKING_SETTING_PAGE_QUALITY)) {
String value = setting.get(RANKING_SETTING_PAGE_QUALITY).getAsString().trim().toUpperCase();
try {
pageQuality = PixivDownload.PageQuality.valueOf(value);
} catch(IllegalArgumentException e) {
log.warn("群组ID [{}] - 无效的PageQuality设定值: {}", id, value);
}
}
AutoSender sender = new RandomRankingArtworksSender(
MessageSenderBuilder.getMessageSender(MessageSource.Group, id),
rankingStart,
rankingEnd,
rankingMode, rankingContentType,
pageQuality
);
RandomIntervalSendTimer timer = RandomIntervalSendTimer.createTimer(
id,
sender,
setting.get("time.min").getAsLong(),
setting.get("time.float").getAsInt(),
true, true);
log.info("群组 {} 已创建对应Timer: {}", id, Integer.toHexString(timer.hashCode()));
}
/**
* 删除一个推送定时器
* @param id 群号
* @throws NoSuchElementException 当这个群号没有定时器的时候抛出异常
*/
@Command
public static String removePushGroup(@Argument(name = "$fromGroup") long fromGroup, @Argument(name = "group", force = false) long id) {
long group = id <= 0 ? fromGroup : id;
RandomIntervalSendTimer.getTimerById(group).destroy();
pushInfoMap.remove(group);
return "已关闭群 " + group + " 的美图推送功能。";
}
/**
* 根据已修改的pushInfoMap将已经被删除的Timer取消
*/
private static void cleanPushTimer() {
RandomIntervalSendTimer.timerIdSet().forEach(id -> {
if(!pushInfoMap.containsKey(id)) {
RandomIntervalSendTimer.getTimerById(id).destroy();
}
});
}
@Command
public static String getReportList() {
SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd_HH:mm:ss");
Set<String> keys = BotCommandProcess.reportStore.keys();
StringBuilder msgBuilder = new StringBuilder();
msgBuilder.append("当前被报告的作品列表:\n");
int count = 1;
for(String key : keys) {
String illustIdStr = key.substring(key.indexOf(".") + 1);
JsonObject report = BotCommandProcess.reportStore.getCache(illustIdStr).getAsJsonObject();
log.debug("{} - Report: {}", illustIdStr, report);
String reason = report.get("reason").isJsonNull() ? "" : report.get("reason").getAsString();
msgBuilder.append(count).append(". 作品Id: ").append(illustIdStr)
.append("(").append(dateFormat.format(new Date(report.get("reportTime").getAsLong()))).append(")\n")
.append("报告者QQ").append(report.get("fromQQ").getAsLong()).append("\n")
.append("报告所在群:").append(report.get("fromGroup").getAsLong()).append("\n")
.append("报告原因:\n").append(reason).append("\n");
}
return msgBuilder.toString();
}
@Command
public static String unBanArtwork(@Argument(name = "id") int illustId) {
if(illustId <= 0) {
return "无效的作品id!";
}
boolean removeResult = BotCommandProcess.reportStore.remove(String.valueOf(illustId));
return removeResult ? "作品已解封!" : "解封失败!可能该作品并未被封禁。";
}
}

View File

@ -54,6 +54,14 @@ public class BotCode {
return new BotCode(keys[0], keys[1], param);
}
/**
* 获取BotCode所使用的匹配正则表达式
* @return 用于匹配BotCode的正则表达式对象
*/
public static Pattern getCodePattern() {
return Pattern.compile(codePattern.pattern());
}
private String platformName;
private String functionName;
private Hashtable<String, String> parameter = new Hashtable<>();

View File

@ -2,7 +2,6 @@ package net.lamgc.cgj.bot;
import com.google.common.base.Strings;
import com.google.common.base.Throwables;
import com.google.common.reflect.TypeToken;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import com.google.gson.*;
import io.netty.handler.codec.http.HttpHeaderNames;
@ -36,13 +35,14 @@ import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
@SuppressWarnings({"SynchronizationOnLocalVariableOrMethodParameter", "SameParameterValue"})
public class BotCommandProcess {
private final static PixivDownload pixivDownload = new PixivDownload(Main.cookieStore, Main.proxy);
private final static Logger log = LoggerFactory.getLogger(BotCommandProcess.class.getSimpleName());
private final static File imageStoreDir = new File(System.getProperty("cgj.cqRootDir"), "data/image/cgj/");
private final static File imageStoreDir = new File(System.getProperty("cgj.botDataDir"), "data/image/cgj/");
public final static Properties globalProp = new Properties();
private final static Gson gson = new GsonBuilder()
.serializeNulls()
@ -55,17 +55,8 @@ public class BotCommandProcess {
new LocalHashCacheStore<>(), 3600000, 900000);
private final static CacheStore<JsonElement> searchBodyCache = new JsonRedisCacheStore(BotEventHandler.redisServer, "searchBody", gson);
private final static CacheStore<List<JsonObject>> rankingCache = new JsonObjectRedisListCacheStore(BotEventHandler.redisServer, "ranking", gson);
private final static CacheStore<List<String>> pagesCache = new RedisPoolCacheStore<List<String>>(BotEventHandler.redisServer, "imagePages") {
@Override
protected String parse(List<String> dataObj) {
return gson.toJson(dataObj);
}
@Override
protected List<String> analysis(String dataStr) {
return gson.fromJson(dataStr, new TypeToken<List<String>>(){}.getType());
}
};
private final static CacheStore<List<String>> pagesCache = new StringListRedisCacheStore(BotEventHandler.redisServer, "imagePages");
public final static CacheStore<JsonElement> reportStore = new JsonRedisCacheStore(BotEventHandler.redisServer, "report", gson);
/**
* 图片异步缓存执行器
@ -86,7 +77,7 @@ public class BotCommandProcess {
public static void initialize() {
log.info("正在初始化...");
File globalPropFile = new File("./global.properties");
File globalPropFile = new File(System.getProperty("cgj.botDataDir"), "global.properties");
if(globalPropFile.exists() && globalPropFile.isFile()) {
log.info("正在加载全局配置文件...");
try {
@ -123,16 +114,69 @@ public class BotCommandProcess {
public static String help() {
StringBuilder helpStrBuilder = new StringBuilder();
helpStrBuilder.append("CGJ Bot使用指南").append("\n");
helpStrBuilder.append("使用方法:.cgj <命令> [参数...]").append("\n");
helpStrBuilder.append("例如查询作品信息功能:").append("\n");
helpStrBuilder.append(".cgj artwork -id 80846159").append("\n");
helpStrBuilder.append("目前可用的命令:").append("\n");
helpStrBuilder.append("\t").append("ranking - 获取今天或指定日期排行榜的前10名作品").append("\n");
helpStrBuilder.append("\t\t").append("-date - 指定查询日期(年-月-日)").append("\n");
helpStrBuilder.append("\t\t").append("-type - 排行榜类型(illust/插画, ugoira/动图, manga/漫画)").append("\n");
helpStrBuilder.append("\t").append("search - 搜索指定关键词并显示前10个作品").append("\n");
helpStrBuilder.append("\t\t").append("-content - 搜索内容").append("\n");
helpStrBuilder.append("\t").append("artworks - 获取作品的Pixiv页面").append("\n");
helpStrBuilder.append("\t").append("link - 获取作品的Pixiv页面").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品id").append("\n");
helpStrBuilder.append("\t").append("info - 获取Pixiv作品信息").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品id").append("\n");
helpStrBuilder.append("\t").append("image - 获取指定作品的图片").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品id").append("\n");
helpStrBuilder.append("\t\t").append("-quality - 图片质量(original/原图 regular/预览图)").append("\n");
helpStrBuilder.append("\t\t").append("-page - 页数").append("\n");
helpStrBuilder.append("\t").append("report - 报告不当作品").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品Id").append("\n");
helpStrBuilder.append("\t\t").append("-msg - 报告原因").append("\n");
return helpStrBuilder.toString();
}
@Command(commandName = "info")
public static String artworkInfo(@Argument(name = "id") int illustId) {
if(illustId <= 0) {
return "错误的作品id";
}
try {
if(isNoSafe(illustId, globalProp, false) || isReported(illustId)) {
return "阅览禁止:该作品已被封印!!";
}
JsonObject illustPreLoadData = getIllustPreLoadData(illustId, false);
StringBuilder builder = new StringBuilder("---------------- 作品信息 ----------------\n");
builder.append("作品Id: ").append(illustId).append("\n");
builder.append("作品标题:").append(illustPreLoadData.get("illustTitle").getAsString()).append("\n");
builder.append("作者(作者Id)").append(illustPreLoadData.get("userName").getAsString())
.append("(").append(illustPreLoadData.get("userId").getAsInt()).append(")\n");
builder.append("点赞数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.LIKE.attrName).getAsInt()).append("\n");
builder.append("收藏数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.BOOKMARK.attrName).getAsInt()).append("\n");
builder.append("围观数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.VIEW.attrName).getAsInt()).append("\n");
builder.append("评论数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.COMMENT.attrName).getAsInt()).append("\n");
builder.append("页数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.PAGE.attrName).getAsInt()).append("\n");
builder.append("---------------- 作品图片 ----------------\n");
builder.append(getImageById(illustId, PixivDownload.PageQuality.REGULAR, 1)).append("\n");
builder.append("使用 \".cgj image -id ")
.append(illustId)
.append("\" 获取原图。\n如有不当作品可使用\".cgj report -id ")
.append(illustId).append("\"向色图姬反馈。");
return builder.toString();
} catch (IOException e) {
e.printStackTrace();
}
return "尚未支持";
}
@Command
public static String ranking(
@Argument(force = false, name = "date") Date queryTime,
@ -158,18 +202,22 @@ public class BotCommandProcess {
}
}
PixivURL.RankingMode mode = PixivURL.RankingMode.MODE_DAILY;
PixivURL.RankingMode mode;
try {
mode = PixivURL.RankingMode.valueOf("MODE_" + contentMode.toUpperCase());
String rankingModeValue = contentMode.toUpperCase();
mode = PixivURL.RankingMode.valueOf(rankingModeValue.startsWith("MODE_") ? rankingModeValue : "MODE_" + rankingModeValue);
} catch (IllegalArgumentException e) {
log.warn("无效的RankingMode值: {}", contentMode);
return "参数无效, 请查看帮助信息";
}
PixivURL.RankingContentType type = PixivURL.RankingContentType.TYPE_ILLUST;
PixivURL.RankingContentType type;
try {
type = PixivURL.RankingContentType.valueOf("TYPE_" + contentType.toUpperCase());
String contentTypeValue = contentType.toUpperCase();
type = PixivURL.RankingContentType.valueOf(contentTypeValue.startsWith("TYPE_") ? contentTypeValue : "TYPE_" + contentTypeValue);
} catch (IllegalArgumentException e) {
log.warn("无效的RankingContentType值: {}", contentType);
return "参数无效, 请查看帮助信息";
}
if(!type.isSupportedMode(mode)) {
@ -197,7 +245,6 @@ public class BotCommandProcess {
log.warn("配置项 {} 的参数值格式有误!", imageLimitPropertyKey);
}
//TODO(LamGC, 2020.4.11): 将JsonRedisCacheStore更改为使用Redis的List集合, 以提高性能
List<JsonObject> rankingInfoList = getRankingInfoByCache(type, mode, queryDate, 1, Math.max(0, itemLimit), false);
if(rankingInfoList.isEmpty()) {
return "无法查询排行榜,可能排行榜尚未更新。";
@ -221,7 +268,7 @@ public class BotCommandProcess {
log.error("消息处理异常", e);
return "排名榜获取失败!详情请查看机器人控制台。";
}
return resultBuilder.append("如查询当前时间获取到昨天时间,则今日排名榜未更新。").toString();
return resultBuilder.append("如查询当前时间获取到昨天时间,则今日排名榜未更新。\n如有不当作品可使用\".cgj report -id 作品id\"向色图姬反馈。").toString();
}
@Command(commandName = "userArt")
@ -230,7 +277,18 @@ public class BotCommandProcess {
return "功能未完成";
}
private final static Object searchCacheLock = new Object();
/**
* 搜索命令
* @param content 搜索内容
* @param type 搜索类型
* @param area 搜索区域
* @param includeKeywords 包括关键字
* @param excludeKeywords 排除关键字
* @param contentOption 搜索选项
* @param pagesIndex 搜索页索引
* @return 返回搜索内容消息
* @throws IOException 当搜索发生异常时抛出
*/
@Command
public static String search(@Argument(name = "content") String content,
@Argument(name = "type", force = false) String type,
@ -266,26 +324,26 @@ public class BotCommandProcess {
if (!Strings.isNullOrEmpty(includeKeywords)) {
for (String keyword : includeKeywords.split(";")) {
searchBuilder.removeExcludeKeyword(keyword);
searchBuilder.addIncludeKeyword(keyword);
searchBuilder.removeExcludeKeyword(keyword.trim());
searchBuilder.addIncludeKeyword(keyword.trim());
log.debug("已添加关键字: {}", keyword);
}
}
if (!Strings.isNullOrEmpty(excludeKeywords)) {
for (String keyword : excludeKeywords.split(";")) {
searchBuilder.removeIncludeKeyword(keyword);
searchBuilder.addExcludeKeyword(keyword);
searchBuilder.removeIncludeKeyword(keyword.trim());
searchBuilder.addExcludeKeyword(keyword.trim());
log.debug("已添加排除关键字: {}", keyword);
}
}
log.info("正在搜索作品, 条件: {}", searchBuilder.getSearchCondition());
String requestUrl = searchBuilder.buildURL();
String requestUrl = searchBuilder.buildURL().intern();
log.debug("RequestUrl: {}", requestUrl);
JsonObject resultBody = null;
if(!searchBodyCache.exists(requestUrl)) {
synchronized (searchCacheLock) {
synchronized (requestUrl) {
if (!searchBodyCache.exists(requestUrl)) {
log.debug("searchBody缓存失效, 正在更新...");
JsonObject jsonObject;
@ -375,6 +433,9 @@ public class BotCommandProcess {
if (isNoSafe(illustId, globalProp, true)) {
log.warn("作品Id {} 为R-18作品, 跳过.", illustId);
continue;
} else if(isReported(illustId)) {
log.warn("作品Id {} 被报告, 正在等待审核, 跳过该作品.", illustId);
continue;
}
result.append(searchArea.name()).append(" (").append(count).append(" / ").append(limit).append(")\n\t作品id: ").append(illustId)
@ -388,7 +449,7 @@ public class BotCommandProcess {
break;
}
}
return Strings.nullToEmpty(result.toString()) + "预览图片并非原图,使用“.cgj image -id 作品id”获取原图";
return Strings.nullToEmpty(result.toString()) + "预览图片并非原图,使用“.cgj image -id 作品id”获取原图\n如有不当作品可使用\".cgj report -id 作品id\"向色图姬反馈。";
}
@Command(commandName = "pages")
@ -407,12 +468,15 @@ public class BotCommandProcess {
}
}
@Command(commandName = "artworks")
@Command(commandName = "link")
public static String artworksLink(@Argument(name = "id") int illustId) {
try {
if (isNoSafe(illustId, globalProp, false)) {
log.warn("作品Id {} 已被屏蔽.", illustId);
return "由于相关设置,该作品已被屏蔽!";
} else if(isReported(illustId)) {
log.warn("作品Id {} 被报告, 正在等待审核, 跳过该作品.", illustId);
return "该作品暂时被封印,请等待色图姬进一步审核!";
}
} catch (IOException e) {
log.error("获取作品信息失败!", e);
@ -457,6 +521,9 @@ public class BotCommandProcess {
if (isNoSafe(illustId, globalProp, false)) {
log.warn("作品 {} 存在R-18内容且设置\"image.allowR18\"为false将屏蔽该作品不发送.", illustId);
return "(根据设置,该作品已被屏蔽!)";
} else if(isReported(illustId)) {
log.warn("作品Id {} 被报告, 正在等待审核, 跳过该作品.", illustId);
return "(该作品已被封印)";
}
} catch (IOException e) {
log.warn("作品信息无法获取!", e);
@ -501,6 +568,12 @@ public class BotCommandProcess {
return getImageToBotCode(imageCache.get(fileName), false).toString();
}
/**
* 通过文件获取图片的BotCode代码
* @param targetFile 图片文件
* @param updateCache 是否刷新缓存(只是让机器人重新上传, 如果上传接口有重复检测的话是无法处理的)
* @return 返回设定好参数的BotCode
*/
private static BotCode getImageToBotCode(File targetFile, boolean updateCache) {
String fileName = targetFile.getName();
BotCode code = BotCode.parse(CQCode.image(getImageStoreDir().getName() + "/" + fileName));
@ -517,7 +590,7 @@ public class BotCommandProcess {
illustPreLoadDataCache.clear();
pagesCache.clear();
searchBodyCache.clear();
File imageStoreDir = new File(System.getProperty("cgj.cqRootDir") + "data/image/cgj/");
File imageStoreDir = new File(System.getProperty("cgj.botDataDir") + "data/image/cgj/");
File[] listFiles = imageStoreDir.listFiles();
if (listFiles == null) {
log.debug("图片缓存目录为空或内部文件获取失败!");
@ -530,6 +603,34 @@ public class BotCommandProcess {
log.warn("缓存删除完成.");
}
/**
* 举报某一作品
* @param illustId 需要举报的作品id
* @param reason 举报原因
* @return 返回提示信息
*/
@Command
public static String report(@Argument(name = "$fromGroup") long fromGroup, @Argument(name = "$fromQQ") long fromQQ, @Argument(name = "id") int illustId, @Argument(name = "msg", force = false) String reason) {
log.warn("收到作品反馈(IllustId: {}, 原因: {})", illustId, reason);
JsonObject reportJson = new JsonObject();
reportJson.addProperty("illustId", illustId);
reportJson.addProperty("reportTime", new Date().getTime());
reportJson.addProperty("fromGroup", fromGroup);
reportJson.addProperty("fromQQ", fromQQ);
reportJson.addProperty("reason", reason);
reportStore.update(String.valueOf(illustId), reportJson, 0);
return "色图姬收到了你的报告,将屏蔽该作品并对作品违规情况进行核实,感谢你的反馈!";
}
/**
* 检查某一作品是否被报告
* @param illustId 作品Id
* @return 如果被报告了, 返回true
*/
public static boolean isReported(int illustId) {
return reportStore.exists(String.valueOf(illustId));
}
/*
下一目标:
添加定时发图
@ -537,7 +638,15 @@ public class BotCommandProcess {
标签....标签支持搜索吧
*/
private static boolean isNoSafe(int illustId, Properties settingProp, boolean returnRaw) throws IOException {
/**
* 检查指定作品是否为r18
* @param illustId 作品Id
* @param settingProp 配置项
* @param returnRaw 是否返回原始值
* @return 如果为true, 则不为全年龄
* @throws IOException 获取数据时发生异常时抛出
*/
public static boolean isNoSafe(int illustId, Properties settingProp, boolean returnRaw) throws IOException {
boolean rawValue = getIllustInfo(illustId, false).getAsJsonArray("tags").contains(new JsonPrimitive("R-18"));
return returnRaw || settingProp == null ? rawValue : rawValue && !settingProp.getProperty("image.allowR18", "false").equalsIgnoreCase("true");
}
@ -546,7 +655,7 @@ public class BotCommandProcess {
String illustIdStr = buildSyncKey(Integer.toString(illustId));
JsonObject illustInfoObj = null;
if (!illustInfoCache.exists(illustIdStr) || flushCache) {
synchronized (illustIdStr) { // TODO: 这里要不做成HashMap存储key而避免使用常量池?
synchronized (illustIdStr) {
if (!illustInfoCache.exists(illustIdStr) || flushCache) {
illustInfoObj = pixivDownload.getIllustInfoByIllustId(illustId);
illustInfoCache.update(illustIdStr, illustInfoObj, null);
@ -575,7 +684,7 @@ public class BotCommandProcess {
if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) {
synchronized (illustIdStr) {
if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) {
log.info("IllustId {} 缓存失效, 正在更新...", illustId);
log.debug("IllustId {} 缓存失效, 正在更新...", illustId);
JsonObject preLoadDataObj = pixivDownload.getIllustPreLoadDataById(illustId)
.getAsJsonObject("illust")
.getAsJsonObject(Integer.toString(illustId));
@ -591,7 +700,7 @@ public class BotCommandProcess {
result = preLoadDataObj;
illustPreLoadDataCache.update(illustIdStr, preLoadDataObj, expire);
log.info("作品Id {} preLoadData缓存已更新(有效时间: {})", illustId, expire);
log.debug("作品Id {} preLoadData缓存已更新(有效时间: {})", illustId, expire);
}
}
}
@ -663,15 +772,16 @@ public class BotCommandProcess {
if(!rankingCache.exists(requestSign) || flushCache) {
synchronized(requestSign) {
if(!rankingCache.exists(requestSign) || flushCache) {
log.info("Ranking缓存失效, 正在更新...(RequestSign: {})", requestSign);
log.debug("Ranking缓存失效, 正在更新...(RequestSign: {})", requestSign);
List<JsonObject> rankingResult = pixivDownload.getRanking(contentType, mode, queryDate, 1, 500);
long expireTime = 0;
if(rankingResult.size() == 0) {
log.info("数据获取失败, 将设置浮动有效时间以准备下次更新.");
expireTime = 5400000 + expireTimeFloatRandom.nextInt(1800000);
log.warn("数据获取失败, 将设置浮动有效时间以准备下次更新. (ExpireTime: {}ms)", expireTime);
}
result = new ArrayList<>(rankingResult).subList(start - 1, range);
rankingCache.update(requestSign, rankingResult,
rankingResult.size() == 0 ? 5400000 + expireTimeFloatRandom.nextInt(1800000) : 0);
log.info("Ranking缓存更新完成.(RequestSign: {})", requestSign);
result = new ArrayList<>(rankingResult).subList(start - 1, start + range - 1);
rankingCache.update(requestSign, rankingResult, expireTime);
log.debug("Ranking缓存更新完成.(RequestSign: {})", requestSign);
}
}
}

View File

@ -0,0 +1,153 @@
package net.lamgc.cgj.bot;
import com.google.common.base.Throwables;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.lang.reflect.Field;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.concurrent.atomic.AtomicBoolean;
/**
* 随机间隔发送器
*/
public class RandomIntervalSendTimer extends TimerTask {
private final static Timer timer = new Timer("Thread-RIST");
private final static Logger log = LoggerFactory.getLogger("RandomIntervalSendTimer");
private final static Map<Long, RandomIntervalSendTimer> timerMap = new HashMap<>();
private final long timerId;
private final Random timeRandom = new Random();
private final AutoSender sender;
private final long time;
private final int floatTime;
private AtomicBoolean loop = new AtomicBoolean();
private final AtomicBoolean start = new AtomicBoolean();
/**
* 创建一个随机延迟发送器
* @param timerId 该Timer的标识,
* 标识必须是唯一的, 当使用相同id创建时, 将会返回该id所属的Timer对象
* @param sender 自动发送器
* @param time 最低时间(ms)
* @param floatTime 浮动时间(ms)
* @param startNow 现在开始
* @param loop 是否循环
*/
public static RandomIntervalSendTimer createTimer(long timerId, AutoSender sender, long time, int floatTime, boolean startNow, boolean loop) {
if(timerMap.containsKey(timerId)) {
return timerMap.get(timerId);
}
RandomIntervalSendTimer timer = new RandomIntervalSendTimer(timerId, sender, time, floatTime, startNow, loop);
timerMap.put(timerId, timer);
return timer;
}
/**
* 通过Id获取Timer
* @param id 待获取Timer对应的Id
* @return 返回RandomIntervalSendTimer对象
* @throws NoSuchElementException 当不存在Timer时抛出
*/
public static RandomIntervalSendTimer getTimerById(long id) {
if(!timerMap.containsKey(id)) {
throw new NoSuchElementException("id=" + id);
}
return timerMap.get(id);
}
/**
* 获取所有id
* @return 所有TimerId的集合
*/
public static Set<Long> timerIdSet() {
return new HashSet<>(timerMap.keySet());
}
/**
* 创建一个随机延迟发送器
* @param timerId 该Timer的标识
* @param sender 自动发送器
* @param time 最低时间(ms)
* @param floatTime 浮动时间(ms)
* @param startNow 现在开始
* @param loop 是否循环
*/
private RandomIntervalSendTimer(long timerId, AutoSender sender, long time, int floatTime, boolean startNow, boolean loop) {
this.timerId = timerId;
this.sender = sender;
this.time = time;
this.floatTime = floatTime;
timerMap.put(timerId, this);
if(startNow) {
start(loop);
}
}
public void start() {
start(this.loop.get());
}
private final static DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-ss HH:mm:ss");
/**
* 启动定时器
* @param loop 是否循环, 如果为true, 则任务完成后会自动调用start方法继续循环, 直到被调用{@code #}或总定时器被销毁;
*/
public void start(boolean loop) {
this.loop.set(loop);
long nextDelay = time + timeRandom.nextInt(floatTime);
Date nextDate = new Date();
nextDate.setTime(nextDate.getTime() + nextDelay);
log.info("定时器 {} 下一延迟: {}ms ({})", Integer.toHexString(this.hashCode()), nextDelay, dateFormat.format(nextDate));
if(start.get()) {
try {
Field state = this.getClass().getSuperclass().getDeclaredField("state");
state.setAccessible(true);
state.setInt(this, 0);
state.setAccessible(false);
} catch (NoSuchFieldException | IllegalAccessException e) {
e.printStackTrace();
return;
}
}
start.set(true);
timer.schedule(this, nextDelay);
}
@Override
public void run() {
try {
sender.send();
} catch (Exception e) {
log.error("定时器 {} 执行时发生异常:\n{}", Integer.toHexString(this.hashCode()), Throwables.getStackTraceAsString(e));
}
if (this.loop.get()) {
start();
}
}
/**
* 取消该定时器
* @return 取消成功返回true
*/
@Override
public boolean cancel() {
start.set(false);
loop.set(false);
return super.cancel();
}
/**
* 销毁这个定时器
*/
public void destroy() {
cancel();
timerMap.remove(this.timerId);
}
}

View File

@ -0,0 +1,99 @@
package net.lamgc.cgj.bot;
import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivURL;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.util.Calendar;
import java.util.Date;
import java.util.List;
import java.util.Random;
/**
* 推荐作品发送器
*/
public class RandomRankingArtworksSender extends AutoSender {
private final Logger log;
private final int rankingStart;
private final int rankingStop;
private final PixivURL.RankingMode mode;
private final PixivURL.RankingContentType contentType;
private final PixivDownload.PageQuality quality;
/**
* 构造一个推荐作品发送器
* @param messageSender 消息发送器
* @param rankingStart 排行榜开始范围(从1开始, 名次)如传入0或负数则为默认值默认为1
* @param rankingStop 排名榜结束范围(包括该名次)如传入0或负数则为默认值默认为150
* @param mode 排行榜模式
* @param contentType 排行榜内容类型
* @param quality 图片质量, 详见{@link PixivDownload.PageQuality}
* @throws IndexOutOfBoundsException 当 rankingStart > rankingStop时抛出
*/
public RandomRankingArtworksSender(MessageSender messageSender, int rankingStart, int rankingStop, PixivURL.RankingMode mode, PixivURL.RankingContentType contentType, PixivDownload.PageQuality quality) {
super(messageSender);
this.mode = mode;
this.contentType = contentType;
log = LoggerFactory.getLogger("RecommendArtworksSender@" + Integer.toHexString(this.hashCode()));
this.rankingStart = rankingStart > 0 ? rankingStart : 1;
this.rankingStop = rankingStop > 0 ? rankingStop : 150;
if(this.rankingStart > this.rankingStop) {
throw new IndexOutOfBoundsException("rankingStart=" + this.rankingStart + ", rankingStop=" + this.rankingStop);
}
this.quality = quality == null ? PixivDownload.PageQuality.REGULAR : quality;
}
@Override
public void send() {
Date queryDate = new Date();
Calendar calendar = Calendar.getInstance();
calendar.setTime(queryDate);
if(calendar.get(Calendar.HOUR_OF_DAY) < 12) {
calendar.add(Calendar.DAY_OF_YEAR, -2);
} else {
calendar.add(Calendar.DAY_OF_YEAR, -1);
}
queryDate = calendar.getTime();
int selectRanking = rankingStart + new Random().nextInt(rankingStop - rankingStart + 1);
try {
List<JsonObject> rankingList = BotCommandProcess.getRankingInfoByCache(
contentType,
mode,
queryDate,
selectRanking,
1, false);
log.debug("RankingResult.size: {}", rankingList.size());
if(rankingList.size() != 1) {
log.error("排行榜选取失败!(获取到了多个结果)");
return;
}
JsonObject rankingInfo = rankingList.get(0);
int illustId = rankingInfo.get("illust_id").getAsInt();
if(BotCommandProcess.isNoSafe(illustId, BotCommandProcess.globalProp, false)) {
log.warn("作品为r18作品, 取消本次发送.");
return;
} else if(BotCommandProcess.isReported(illustId)) {
log.warn("作品Id {} 被报告, 正在等待审核, 跳过该作品.", illustId);
return;
}
StringBuilder message = new StringBuilder();
message.append("#美图推送 - 今日排行榜 第 ").append(rankingInfo.get("rank").getAsInt()).append("\n");
message.append("标题:").append(rankingInfo.get("title").getAsString()).append("(").append(illustId).append(")\n");
message.append("作者:").append(rankingInfo.get("user_name").getAsString()).append("\n");
message.append(BotCommandProcess.getImageById(illustId, quality, 1));
message.append("\n如有不当作品可使用\".cgj report -id ").append(illustId).append("\"向色图姬反馈。");
getMessageSender().sendMessage(message.toString());
} catch (IOException e) {
e.printStackTrace();
}
}
}

View File

@ -1,10 +1,11 @@
package net.lamgc.cgj.bot;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.event.VirtualLoadMessageEvent;
import net.lamgc.cgj.pixiv.PixivURL;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.time.LocalDate;
import java.util.Calendar;
import java.util.Date;
@ -26,10 +27,10 @@ public class RankingUpdateTimer {
cal.setTime(firstRunDate == null ? new Date() : firstRunDate);
LocalDate currentLocalDate = LocalDate.now();
if(cal.get(Calendar.DAY_OF_YEAR) <= currentLocalDate.getDayOfYear() && cal.get(Calendar.HOUR_OF_DAY) >= 12) {
cal.set(Calendar.DAY_OF_YEAR, cal.get(Calendar.DAY_OF_YEAR) + 1);
cal.set(Calendar.DAY_OF_YEAR, currentLocalDate.getDayOfYear() + 1);
}
cal.set(Calendar.HOUR_OF_DAY, 12);
cal.set(Calendar.MINUTE, 30);
cal.set(Calendar.MINUTE, 0);
cal.set(Calendar.SECOND, 0);
cal.set(Calendar.MILLISECOND, 0);
@ -43,7 +44,7 @@ public class RankingUpdateTimer {
}
public void now(Date queryDate) {
log.info("当前时间 {}, 定时任务开始执行...", new Date());
log.warn("当前时间 {}, 定时任务开始执行...", new Date());
Calendar calendar = Calendar.getInstance();
calendar.setTime(queryDate == null ? new Date() : queryDate);
@ -64,15 +65,13 @@ public class RankingUpdateTimer {
log.debug("不支持的类型, 填空值跳过...(类型: {}.{})", rankingMode.name(), contentType.name());
}
log.info("当前排行榜类型: {}.{}, 正在更新...", rankingMode.name(), contentType.name());
try {
BotCommandProcess.getRankingInfoByCache(contentType, rankingMode, calendar.getTime(), 1, 0, true);
log.info("排行榜 {}.{} 更新完成.", rankingMode.name(), contentType.name());
} catch (IOException e) {
log.error("排行榜 {}.{} 更新时发生异常", rankingMode.name(), contentType.name());
log.error("异常信息如下", e);
}
BotEventHandler.executor.executor(
new VirtualLoadMessageEvent(0,0,
".cgj ranking -type=" + contentType.name() + " -mode=" + rankingMode.name()));
log.info("排行榜 {}.{} 负载指令已投递.", rankingMode.name(), contentType.name());
}
}
log.warn("定时任务更新完成.");
}
/**

View File

@ -1,7 +1,12 @@
package net.lamgc.cgj.bot.cache;
import java.util.Date;
import java.util.Set;
/**
* 缓存库接口
* @param <T> 缓存数据类型
*/
public interface CacheStore<T> {
/**
@ -64,6 +69,19 @@ public interface CacheStore<T> {
*/
boolean clear();
/**
* 获取key集合
* @return 返回存储缓存库中所有缓存项key的集合
*/
Set<String> keys();
/**
* 删除指定缓存项
* @param key 缓存项key
* @return 删除成功返回true
*/
boolean remove(String key);
/**
* 是否支持持久化
* @return 如果支持返回true

View File

@ -3,9 +3,7 @@ package net.lamgc.cgj.bot.cache;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Date;
import java.util.Objects;
import java.util.Random;
import java.util.*;
/**
* 具有继承性的热点数据缓存库
@ -99,6 +97,21 @@ public class HotDataCacheStore<T> implements CacheStore<T> {
return current.clear();
}
@Override
public Set<String> keys() {
Set<String> keys = new HashSet<>();
keys.addAll(current.keys());
keys.addAll(parent.keys());
return keys;
}
@Override
public boolean remove(String key) {
parent.remove(key);
current.remove(key);
return true;
}
@Override
public boolean supportedPersistence() {
return current.supportedPersistence() || parent.supportedPersistence();

View File

@ -27,9 +27,10 @@ public class ImageCacheHandler implements EventHandler {
private final static Set<ImageCacheObject> cacheQueue = Collections.synchronizedSet(new HashSet<>());
@SuppressWarnings("unused")
public void getImageToCache(ImageCacheObject event) {
if(cacheQueue.contains(event)) {
log.info("图片 {} 已存在相同缓存任务, 跳过.", event.getStoreFile().getName());
log.debug("图片 {} 已存在相同缓存任务, 跳过.", event.getStoreFile().getName());
return;
} else {
cacheQueue.add(event);
@ -38,7 +39,7 @@ public class ImageCacheHandler implements EventHandler {
try {
log.info("图片 {} Event正在进行...({})", event.getStoreFile().getName(), Integer.toHexString(event.hashCode()));
File storeFile = event.getStoreFile();
log.info("正在缓存图片 {} (Path: {})", storeFile.getName(), storeFile.getAbsolutePath());
log.debug("正在缓存图片 {} (Path: {})", storeFile.getName(), storeFile.getAbsolutePath());
try {
if(!storeFile.exists() && !storeFile.createNewFile()) {
log.error("无法创建文件(Path: {})", storeFile.getAbsolutePath());
@ -63,7 +64,7 @@ public class ImageCacheHandler implements EventHandler {
return;
}
log.info("正在下载...(Content-Length: {}KB)", response.getEntity().getContentLength() / 1024);
log.debug("正在下载...(Content-Length: {}KB)", response.getEntity().getContentLength() / 1024);
try(FileOutputStream fos = new FileOutputStream(storeFile)) {
IOUtils.copy(response.getEntity().getContent(), fos);
} catch (IOException e) {

View File

@ -5,8 +5,13 @@ import org.jetbrains.annotations.NotNull;
import java.util.Date;
import java.util.Hashtable;
import java.util.Objects;
import java.util.Set;
import java.util.concurrent.atomic.AtomicReference;
/**
* 基于Hashtable的本地缓存库
* @param <T> 缓存类型
*/
public class LocalHashCacheStore<T> implements CacheStore<T> {
private final Hashtable<String, CacheObject<T>> cache;
@ -93,6 +98,16 @@ public class LocalHashCacheStore<T> implements CacheStore<T> {
return true;
}
@Override
public Set<String> keys() {
return cache.keySet();
}
@Override
public boolean remove(String key) {
return cache.remove(key) != null;
}
@Override
public boolean supportedPersistence() {
return false;

View File

@ -9,6 +9,7 @@ import redis.clients.jedis.*;
import java.net.URI;
import java.util.Date;
import java.util.Objects;
import java.util.Set;
import java.util.function.Consumer;
import java.util.function.Function;
@ -50,29 +51,27 @@ public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
@Override
public void update(String key, T value, Date expire) {
Jedis jedis = jedisPool.getResource();
jedis.set(keyPrefix + key, parse(value));
if(expire != null) {
jedis.pexpireAt(keyPrefix + key, expire.getTime());
log.debug("已设置Key {} 的过期时间(Expire: {})", key, expire.getTime());
try (Jedis jedis = jedisPool.getResource()) {
jedis.set(keyPrefix + key, parse(value));
if(expire != null) {
jedis.pexpireAt(keyPrefix + key, expire.getTime());
log.debug("已设置Key {} 的过期时间(Expire: {})", key, expire.getTime());
}
}
jedis.close();
}
@Override
public T getCache(String key) {
Jedis jedis = jedisPool.getResource();
T result = analysis(jedis.get(keyPrefix + key));
jedis.close();
return result;
try (Jedis jedis = jedisPool.getResource()) {
return analysis(jedis.get(keyPrefix + key));
}
}
@Override
public boolean exists(String key) {
Jedis jedis = jedisPool.getResource();
boolean result = jedis.exists(keyPrefix + key);
jedis.close();
return result;
try (Jedis jedis = jedisPool.getResource()) {
return jedis.exists(keyPrefix + key);
}
}
@Override
@ -82,11 +81,23 @@ public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
@Override
public boolean clear() {
Jedis jedis = jedisPool.getResource();
String result = jedis.flushDB();
jedis.close();
log.info("flushDB返回结果: {}", result);
return true;
try (Jedis jedis = jedisPool.getResource()) {
return jedis.flushDB().equalsIgnoreCase("ok");
}
}
@Override
public Set<String> keys() {
try (Jedis jedis = jedisPool.getResource()) {
return jedis.keys(keyPrefix + "*");
}
}
@Override
public boolean remove(String key) {
try (Jedis jedis = jedisPool.getResource()) {
return jedis.del(keyPrefix + key) == 1;
}
}
/**
@ -145,13 +156,4 @@ public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
return false;
}
/**
* 替换原本的分隔符(.)为(:).<br/>
* 即将启用
* @param key 要处理的键名
* @return 处理后的键名
*/
public static String replaceKey(String key) {
return key.replaceAll("\\.", ":");
}
}

View File

@ -0,0 +1,30 @@
package net.lamgc.cgj.bot.cache;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.JedisPoolConfig;
import java.net.URI;
public class StringListRedisCacheStore extends RedisListCacheStore<String> {
public StringListRedisCacheStore(URI redisServerUri, String prefix) {
super(redisServerUri, prefix);
}
public StringListRedisCacheStore(URI redisServerUri, JedisPoolConfig config, int timeout, String password, String prefix) {
super(redisServerUri, config, timeout, password, prefix);
}
public StringListRedisCacheStore(JedisPool pool, String prefix) {
super(pool, prefix);
}
@Override
public String parseData(String dataObj) {
return dataObj;
}
@Override
public String analysisData(String str) {
return str;
}
}

View File

@ -7,6 +7,7 @@ import net.lamgc.cgj.bot.BotAdminCommandProcess;
import net.lamgc.cgj.bot.BotCommandProcess;
import net.lamgc.cgj.util.DateParser;
import net.lamgc.cgj.util.PagesQualityParser;
import net.lamgc.cgj.util.TimeLimitThreadPoolExecutor;
import net.lamgc.utils.base.runner.ArgumentsRunner;
import net.lamgc.utils.base.runner.ArgumentsRunnerConfig;
import net.lamgc.utils.base.runner.exception.DeveloperRunnerException;
@ -23,24 +24,22 @@ import redis.clients.jedis.JedisPool;
import java.lang.reflect.Method;
import java.net.URI;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Date;
import java.util.Objects;
import java.util.*;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class BotEventHandler implements EventHandler {
public final static String COMMAND_PREFIX = ".cgj";
public final static String ADMIN_COMMAND_PREFIX = ".cgjadmin ";
private final ArgumentsRunner processRunner;
private final ArgumentsRunner adminRunner;
private final Logger log = LoggerFactory.getLogger("BotEventHandler@" + Integer.toHexString(this.hashCode()));
private final static Logger log = LoggerFactory.getLogger("BotEventHandler");
/**
* 所有缓存共用的JedisPool
@ -51,7 +50,8 @@ public class BotEventHandler implements EventHandler {
/**
* 消息事件执行器
*/
public final static EventExecutor executor = new EventExecutor(new ThreadPoolExecutor(
public final static EventExecutor executor = new EventExecutor(new TimeLimitThreadPoolExecutor(
60 * 1000,
(int) Math.ceil(Runtime.getRuntime().availableProcessors() / 2F),
Runtime.getRuntime().availableProcessors(),
30L,
@ -95,6 +95,21 @@ public class BotEventHandler implements EventHandler {
initialled = true;
}
private final static AtomicBoolean preLoaded = new AtomicBoolean();
/**
* 预加载
*/
public synchronized static void preLoad() {
if(preLoaded.get()) {
return;
}
try {
BotAdminCommandProcess.loadPushList();
} finally {
preLoaded.set(true);
}
}
private BotEventHandler() {
ArgumentsRunnerConfig runnerConfig = new ArgumentsRunnerConfig();
runnerConfig.setUseDefaultValueInsteadOfException(true);
@ -110,6 +125,11 @@ public class BotEventHandler implements EventHandler {
BotCommandProcess.initialize();
}
/**
* 以事件形式处理消息事件
* @param event 消息事件对象
*/
@SuppressWarnings("unused")
public void processMessage(MessageEvent event) {
String msg = event.getMessage();
log.debug(event.toString());
@ -119,7 +139,7 @@ public class BotEventHandler implements EventHandler {
Pattern pattern = Pattern.compile("/\\s*(\".+?\"|[^:\\s])+((\\s*:\\s*(\".+?\"|[^\\s])+)|)|(\".+?\"|[^\"\\s])+");
Matcher matcher = pattern.matcher(Strings.nullToEmpty(msg));
ArrayList<String> argsList = new ArrayList<>();
List<String> argsList = new ArrayList<>();
while (matcher.find()) {
String arg = matcher.group();
int startIndex = 0;
@ -141,17 +161,22 @@ public class BotEventHandler implements EventHandler {
String[] args = new String[argsList.size()];
argsList.toArray(args);
log.debug("传入参数: {}", Arrays.toString(args));
argsList.add("-$fromGroup");
argsList.add(String.valueOf(event.getFromGroup()));
argsList.add("-$fromQQ");
argsList.add(String.valueOf(event.getFromQQ()));
args = Arrays.copyOf(args, args.length + 4);
argsList.toArray(args);
log.info("正在处理命令...");
long time = System.currentTimeMillis();
Object result;
try {
if(msg.toLowerCase().startsWith(COMMAND_PREFIX + "admin")) {
if(msg.toLowerCase().startsWith(ADMIN_COMMAND_PREFIX)) {
if(!String.valueOf(event.getFromQQ()).equals(BotCommandProcess.globalProp.getProperty("admin.adminId"))) {
event.sendMessage("你没有执行该命令的权限!");
return;
result = "你没有执行该命令的权限!";
} else {
result = adminRunner.run(new BotAdminCommandProcess(), args.length <= 1 ? new String[0] : Arrays.copyOfRange(args, 1, args.length));
result = adminRunner.run(args.length <= 1 ? new String[0] : Arrays.copyOfRange(args, 1, args.length));
}
} else {
result = processRunner.run(args.length <= 1 ? new String[0] : Arrays.copyOfRange(args, 1, args.length));
@ -161,10 +186,15 @@ public class BotEventHandler implements EventHandler {
} catch(ParameterNoFoundException e) {
result = "命令缺少参数: " + e.getParameterName();
} catch(DeveloperRunnerException e) {
log.error("执行命令时发生异常", e);
result = "命令执行时发生错误,无法完成!";
if (!(e.getCause() instanceof InterruptedException)) {
log.error("执行命令时发生异常", e);
result = "色图姬在执行命令时遇到了一个错误!";
} else {
log.error("命令执行超时, 终止执行.");
result = "色图姬发现这个命令的处理时间太久了!所以打断了这个命令。";
}
}
log.info("命令处理完成.(耗时: {}ms)", System.currentTimeMillis() - time);
long processTime = System.currentTimeMillis() - time;
if(Objects.requireNonNull(result) instanceof String) {
try {
event.sendMessage((String) result);
@ -172,7 +202,10 @@ public class BotEventHandler implements EventHandler {
log.error("发送消息时发生异常", e);
}
}
log.info("命令反馈完成.(耗时: {}ms)", System.currentTimeMillis() - time);
long totalTime = System.currentTimeMillis() - time;
log.info("命令反馈完成.(事件耗时: {}ms, P: {}%({}ms), R: {}%({}ms))", totalTime,
String.format("%.3f", ((double) processTime / (double)totalTime) * 100F), processTime,
String.format("%.3f", ((double) (totalTime - processTime) / (double)totalTime) * 100F), totalTime - processTime);
}
/**
@ -181,7 +214,7 @@ public class BotEventHandler implements EventHandler {
* @return 如果为true则提交
*/
public static boolean match(String message) {
return message.startsWith(COMMAND_PREFIX);
return message.startsWith(COMMAND_PREFIX) || message.startsWith(ADMIN_COMMAND_PREFIX);
}
}

View File

@ -1,8 +1,9 @@
package net.lamgc.cgj.bot.event;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.utils.event.EventObject;
public abstract class MessageEvent implements EventObject {
public abstract class MessageEvent implements EventObject, MessageSender {
private final long fromGroup;
private final long fromQQ;
@ -19,6 +20,7 @@ public abstract class MessageEvent implements EventObject {
* @param message 消息内容
* @return 成功返回MessageId, 如没有MessageId则返回0, 失败返回负数错误码
*/
@Override
public abstract int sendMessage(final String message);
/**
@ -28,12 +30,6 @@ public abstract class MessageEvent implements EventObject {
*/
public abstract String getImageUrl(String image);
/**
* 获取原始消息对象.(不推荐使用)
* @return 消息对象
*/
public abstract Object getRawMessage();
/**
* 获取来源群组号
* @return 如非群组消息, 返回0
@ -60,7 +56,7 @@ public abstract class MessageEvent implements EventObject {
@Override
public String toString() {
return "MessageEvent{" +
return this.getClass().getSimpleName() + "{" +
"fromGroup=" + getFromGroup() +
", fromQQ=" + getFromQQ() +
", message='" + getMessage() + '\'' +

View File

@ -0,0 +1,22 @@
package net.lamgc.cgj.bot.event;
/**
* 假负载消息事件, 一般用于预处理某个命令使用,可以增强在高峰期来临时的处理速度.
*/
public class VirtualLoadMessageEvent extends MessageEvent {
public VirtualLoadMessageEvent(long fromGroup, long fromQQ, String message) {
super(fromGroup, fromQQ, message);
}
@Override
public int sendMessage(String message) {
return 0;
}
@Override
public String getImageUrl(String image) {
return null;
}
}

View File

@ -1,4 +1,4 @@
package net.lamgc.cgj.bot;
package net.lamgc.cgj.bot.framework.coolq;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import net.lz1998.cq.CQGlobal;

View File

@ -1,7 +1,7 @@
package net.lamgc.cgj.bot;
package net.lamgc.cgj.bot.framework.coolq;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.event.SpringCQMessageEvent;
import net.lamgc.cgj.bot.framework.coolq.message.SpringCQMessageEvent;
import net.lamgc.utils.event.EventHandler;
import net.lz1998.cq.event.message.CQDiscussMessageEvent;
import net.lz1998.cq.event.message.CQGroupMessageEvent;
@ -16,25 +16,24 @@ import org.springframework.stereotype.Component;
public class CQPluginMain extends CQPlugin implements EventHandler {
public CQPluginMain() {
// TODO(LamGC, 2020.04.21): SpringCQ无法适配MessageSenderBuilder
BotEventHandler.preLoad();
LoggerFactory.getLogger(this.toString())
.info("BotEventHandler.COMMAND_PREFIX = {}", BotEventHandler.COMMAND_PREFIX);
}
@Override
public int onPrivateMessage(CoolQ cq, CQPrivateMessageEvent event) {
//log.info("私聊消息到达: 发送者[{}], 消息内容: {}", event.getSender().getUserId(), event.getMessage());
return processMessage(cq, event);
}
@Override
public int onGroupMessage(CoolQ cq, CQGroupMessageEvent event) {
//log.info("群消息到达: 群[{}], 发送者[{}], 消息内容: {}", event.getGroupId(), event.getSender().getUserId(), event.getMessage());
return processMessage(cq, event);
}
@Override
public int onDiscussMessage(CoolQ cq, CQDiscussMessageEvent event) {
//log.info("讨论组消息到达: 群[{}], 发送者[{}], 消息内容: {}", event.getDiscussId(), event.getSender().getUserId(), event.getMessage());
return processMessage(cq, event);
}

View File

@ -1,6 +1,9 @@
package net.lamgc.cgj.bot.event;
package net.lamgc.cgj.bot.framework.coolq.message;
import net.lamgc.cgj.bot.BotCode;
import net.lamgc.cgj.bot.event.MessageEvent;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lz1998.cq.event.message.CQDiscussMessageEvent;
import net.lz1998.cq.event.message.CQGroupMessageEvent;
import net.lz1998.cq.event.message.CQMessageEvent;
@ -11,13 +14,8 @@ import java.util.Objects;
public class SpringCQMessageEvent extends MessageEvent {
private final static int TYPE_PRIVATE = 0;
private final static int TYPE_GROUP = 1;
private final static int TYPE_DISCUSS = 2;
private final CoolQ cq;
private final int type;
private final CQMessageEvent messageEvent;
private final MessageSender messageSender;
public SpringCQMessageEvent(CoolQ cq, CQMessageEvent messageEvent) {
super(messageEvent instanceof CQGroupMessageEvent ? (
@ -26,27 +24,20 @@ public class SpringCQMessageEvent extends MessageEvent {
((CQDiscussMessageEvent) messageEvent).getDiscussId() : 0,
messageEvent.getUserId(), messageEvent.getMessage());
this.cq = Objects.requireNonNull(cq);
MessageSource source;
if(messageEvent instanceof CQGroupMessageEvent) {
type = TYPE_GROUP;
source = MessageSource.Group;
} else if (messageEvent instanceof CQDiscussMessageEvent) {
type = TYPE_DISCUSS;
source = MessageSource.Discuss;
} else {
type = TYPE_PRIVATE;
source = MessageSource.Private;
}
this.messageEvent = messageEvent;
messageSender = new SpringCQMessageSender(cq, source, source == MessageSource.Private ? getFromQQ() : getFromGroup());
}
@Override
public int sendMessage(final String message) {
switch(type) {
case TYPE_PRIVATE:
return cq.sendPrivateMsg(getFromQQ(), message, false).getData().getMessageId();
case TYPE_GROUP:
case TYPE_DISCUSS:
return cq.sendGroupMsg(getFromGroup(), message, false).getData().getMessageId();
default:
return -1;
}
return messageSender.sendMessage(message);
}
/**
@ -70,8 +61,4 @@ public class SpringCQMessageEvent extends MessageEvent {
}
}
@Override
public Object getRawMessage() {
return messageEvent;
}
}

View File

@ -0,0 +1,32 @@
package net.lamgc.cgj.bot.framework.coolq.message;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lz1998.cq.robot.CoolQ;
public class SpringCQMessageSender implements MessageSender {
private CoolQ coolQ;
private MessageSource source;
private long target;
public SpringCQMessageSender(CoolQ coolQ, MessageSource source, long target) {
this.coolQ = coolQ;
this.source = source;
this.target = target;
}
@Override
public int sendMessage(String message) {
switch (source) {
case Private:
return coolQ.sendPrivateMsg(target, message, false).getData().getMessageId();
case Group:
return coolQ.sendGroupMsg(target, message, false).getData().getMessageId();
case Discuss:
return coolQ.sendDiscussMsg(target, message, false).getData().getMessageId();
default:
return -1;
}
}
}

View File

@ -0,0 +1,20 @@
package net.lamgc.cgj.bot.framework.coolq.message;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lz1998.cq.robot.CoolQ;
public class SpringCQMessageSenderFactory implements MessageSenderFactory {
private final CoolQ coolQ;
public SpringCQMessageSenderFactory(CoolQ coolQ) {
this.coolQ = coolQ;
}
@Override
public MessageSender createMessageSender(MessageSource source, long id) {
return new SpringCQMessageSender(coolQ, source, id);
}
}

View File

@ -1,12 +1,14 @@
package net.lamgc.cgj.bot;
package net.lamgc.cgj.bot.framework.mirai;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.event.MiraiMessageEvent;
import net.lamgc.cgj.bot.framework.mirai.message.MiraiMessageEvent;
import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import net.lamgc.cgj.bot.framework.mirai.message.MiraiMessageSenderFactory;
import net.mamoe.mirai.Bot;
import net.mamoe.mirai.BotFactoryJvm;
import net.mamoe.mirai.japt.Events;
import net.mamoe.mirai.message.FriendMessage;
import net.mamoe.mirai.message.GroupMessage;
import net.mamoe.mirai.qqandroid.QQAndroid;
import net.mamoe.mirai.utils.BotConfiguration;
import org.apache.commons.net.util.Base64;
import org.slf4j.Logger;
@ -24,6 +26,7 @@ public class MiraiMain implements Closeable {
private final static Properties botProperties = new Properties();
public void init() {
Runtime.getRuntime().addShutdownHook(new Thread(this::close));
try {
Class.forName(BotEventHandler.class.getName());
} catch (ClassNotFoundException e) {
@ -31,7 +34,7 @@ public class MiraiMain implements Closeable {
return;
}
File botPropFile = new File("./bot.properties");
File botPropFile = new File(System.getProperty("cgj.botDataDir"), "./bot.properties");
try (Reader reader = new BufferedReader(new FileReader(botPropFile))) {
botProperties.load(reader);
} catch (IOException e) {
@ -39,15 +42,19 @@ public class MiraiMain implements Closeable {
return;
}
bot = QQAndroid.INSTANCE.newBot(Long.parseLong(botProperties.getProperty("bot.qq", "0")), Base64.decodeBase64(botProperties.getProperty("bot.password", "")), new BotConfiguration());
bot = BotFactoryJvm.newBot(Long.parseLong(botProperties.getProperty("bot.qq", "0")), Base64.decodeBase64(botProperties.getProperty("bot.password", "")), new BotConfiguration());
Events.subscribeAlways(GroupMessage.class, (msg) -> BotEventHandler.executor.executor(new MiraiMessageEvent(msg)));
Events.subscribeAlways(FriendMessage.class, (msg) -> BotEventHandler.executor.executor(new MiraiMessageEvent(msg)));
bot.login();
MessageSenderBuilder.setCurrentMessageSenderFactory(new MiraiMessageSenderFactory(bot));
BotEventHandler.preLoad();
bot.join();
}
public void close() {
log.warn("正在关闭机器人...");
bot.close(null);
log.warn("机器人已关闭.");
}
}

View File

@ -0,0 +1,39 @@
package net.lamgc.cgj.bot.framework.mirai.message;
import net.lamgc.cgj.bot.event.MessageEvent;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.message.ContactMessage;
import net.mamoe.mirai.message.GroupMessage;
import net.mamoe.mirai.message.data.MessageUtils;
import java.util.Objects;
public class MiraiMessageEvent extends MessageEvent {
private final ContactMessage messageObject;
private final MessageSender messageSender;
public MiraiMessageEvent(ContactMessage message) {
super(message instanceof GroupMessage ? ((GroupMessage) message).getGroup().getId() : 0,
message.getSender().getId(), message.getMessage().contentToString());
this.messageObject = Objects.requireNonNull(message);
if(message instanceof GroupMessage) {
messageSender = new MiraiMessageSender(((GroupMessage) message).getGroup(), MessageSource.Group);
} else {
messageSender = new MiraiMessageSender(message.getSender(), MessageSource.Private);
}
}
@Override
public int sendMessage(final String message) {
return messageSender.sendMessage(message);
}
@Override
public String getImageUrl(String imageId) {
return messageObject.getBot().queryImageUrl(MessageUtils.newImage(imageId));
}
}

View File

@ -1,4 +1,4 @@
package net.lamgc.cgj.bot.event;
package net.lamgc.cgj.bot.framework.mirai.message;
import com.google.common.base.Strings;
import net.lamgc.cgj.bot.BotCode;
@ -6,36 +6,54 @@ import net.lamgc.cgj.bot.cache.CacheStore;
import net.lamgc.cgj.bot.cache.HotDataCacheStore;
import net.lamgc.cgj.bot.cache.LocalHashCacheStore;
import net.lamgc.cgj.bot.cache.StringRedisCacheStore;
import net.mamoe.mirai.message.ContactMessage;
import net.mamoe.mirai.message.FriendMessage;
import net.mamoe.mirai.message.GroupMessage;
import net.mamoe.mirai.message.data.CombinedMessage;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.Bot;
import net.mamoe.mirai.contact.Contact;
import net.mamoe.mirai.message.data.Image;
import net.mamoe.mirai.message.data.Message;
import net.mamoe.mirai.message.data.MessageChain;
import net.mamoe.mirai.message.data.MessageUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
import java.util.ArrayList;
import java.util.NoSuchElementException;
import java.util.Objects;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class MiraiMessageEvent extends MessageEvent {
public class MiraiMessageSender implements MessageSender {
private final ContactMessage messageObject;
private final Logger log = LoggerFactory.getLogger(this.getClass().getSimpleName() + "@" + Integer.toHexString(this.hashCode()));
private final Contact member;
private final MessageSource source;
private final static Logger log = LoggerFactory.getLogger("MiraiMessageSender");
private final static CacheStore<String> imageIdCache = new HotDataCacheStore<>(
new StringRedisCacheStore(BotEventHandler.redisServer, "mirai.imageId"),
new LocalHashCacheStore<>(),
5400000, 1800000);
/**
* 使用id构造发送器
* @param bot 机器人对象
* @param source 消息源类型
* @param id id, 将会根据消息源类型判断为什么号(QQ号或群号)
* @throws NoSuchElementException 当在机器人好友列表或群列表里没有这个好友或群的时候抛出
*/
public MiraiMessageSender(Bot bot, MessageSource source, long id) {
this(source == MessageSource.Private ? bot.getFriend(id) : bot.getGroup(id), source);
}
public MiraiMessageEvent(ContactMessage message) {
super(message instanceof GroupMessage ? ((GroupMessage) message).getGroup().getId() : 0,
message.getSender().getId(), message.getMessage().toString());
this.messageObject = Objects.requireNonNull(message);
/**
* 通过联系人对象构造发送器
* @param contact 联系人
* @param source 消息源类型
*/
public MiraiMessageSender(Contact contact, MessageSource source) {
this.member = contact;
this.source = source;
}
@Override
@ -43,28 +61,11 @@ public class MiraiMessageEvent extends MessageEvent {
log.debug("处理前的消息内容:\n{}", message);
Message msgBody = processMessage(Objects.requireNonNull(message));
log.debug("处理后的消息内容(可能出现乱序的情况, 但实际上顺序是没问题的):\n{}", msgBody);
if(getFromGroup() == 0) {
FriendMessage msgObject = (FriendMessage) messageObject;
//FIXME(LamGC, 2020.04.10): 当前 Mirai 不支持私聊长文本, 所以发生异常是正常情况...
msgObject.getSender().sendMessage(msgBody);
} else {
GroupMessage msgObject = (GroupMessage) messageObject;
msgObject.getGroup().sendMessage(msgBody);
}
member.sendMessage(msgBody);
return 0;
}
@Override
public String getImageUrl(String imageId) {
return messageObject.getBot().queryImageUrl(MessageUtils.newImage(imageId));
}
@Override
public Object getRawMessage() {
return messageObject;
}
private final static Pattern cqCodePattern = Pattern.compile("\\[.*?:.*?]");
private final static Pattern cqCodePattern = BotCode.getCodePattern();
private Message processMessage(final String message) {
Matcher matcher = cqCodePattern.matcher(message);
ArrayList<String> cqCode = new ArrayList<>();
@ -79,7 +80,7 @@ public class MiraiMessageEvent extends MessageEvent {
.replaceAll("&38", "&")
.split("\\|");
CombinedMessage chain = MessageUtils.newChain().plus("");
MessageChain messages = MessageUtils.newChain().plus("");
int codeIndex = 0;
for(String text : texts) {
if(text.equals("{BotCode}")) {
@ -90,32 +91,44 @@ public class MiraiMessageEvent extends MessageEvent {
log.warn("解析待发送消息内的BotCode时发生异常, 请检查错误格式BotCode的来源并尽快排错!", e);
continue;
}
chain = chain.plus(processBotCode(code));
messages = messages.plus(processBotCode(code));
} else {
chain = chain.plus(text);
messages = messages.plus(text);
}
}
return chain;
return messages;
}
private Message processBotCode(BotCode code) {
switch(code.getFunctionName().toLowerCase()) {
case "image":
Image img;
if(code.containsParameter("id")) {
return MessageUtils.newImage(code.getParameter("id"));
img = MessageUtils.newImage(code.getParameter("id"));
} else if(code.containsParameter("absolutePath")) {
return uploadImage(code);
img = uploadImage(code);
} else {
return MessageUtils.newChain("(参数不存在)");
}
if(Strings.nullToEmpty(code.getParameter("flashImage"))
.equalsIgnoreCase("true")) {
return MessageUtils.flash(img);
} else {
return img;
}
default:
log.warn("解析到不支持的BotCode: {}", code);
return MessageUtils.newChain("(不支持的BotCode)");
}
}
private Image uploadImage(BotCode code) {
/**
* 存在缓存的上传图片.
* @param code 图片BotCode
* @return Image对象
*/
public Image uploadImage(BotCode code) {
log.debug("传入BotCode信息:\n{}", code);
String absolutePath = code.getParameter("absolutePath");
if(Strings.isNullOrEmpty(absolutePath)) {
@ -125,20 +138,16 @@ public class MiraiMessageEvent extends MessageEvent {
String imageName = code.getParameter("imageName");
if(!Strings.isNullOrEmpty(imageName)) {
Image image = null;
imageName = (getMessageSource() + "." + imageName).intern();
imageName = (source + "." + imageName).intern();
if(!imageIdCache.exists(imageName) ||
Strings.nullToEmpty(code.getParameter("updateCache")).equalsIgnoreCase("true")) {
Strings.nullToEmpty(code.getParameter("updateCache")).equalsIgnoreCase("true")) {
synchronized (imageName) {
if(!imageIdCache.exists(imageName) ||
Strings.nullToEmpty(code.getParameter("updateCache")) .equalsIgnoreCase("true")) {
Strings.nullToEmpty(code.getParameter("updateCache")) .equalsIgnoreCase("true")) {
log.debug("imageName [{}] 缓存失效或强制更新, 正在更新缓存...", imageName);
image = uploadImage0(new File(absolutePath));
if(Objects.isNull(image)) {
return null;
}
String cacheExpireAt;
long expireTime = 0;
long expireTime = 864000000; // 10d
if(!Strings.isNullOrEmpty(cacheExpireAt = code.getParameter("cacheExpireAt"))) {
try {
expireTime = Integer.parseInt(cacheExpireAt);
@ -147,7 +156,7 @@ public class MiraiMessageEvent extends MessageEvent {
}
}
imageIdCache.update(imageName, image.getImageId(), expireTime);
log.info("imageName [{}] 缓存更新完成.(有效时间: {})", imageName, expireTime);
log.debug("imageName [{}] 缓存更新完成.(有效时间: {})", imageName, expireTime);
} else {
log.debug("ImageName: [{}] 缓存命中.", imageName);
}
@ -169,25 +178,7 @@ public class MiraiMessageEvent extends MessageEvent {
}
private Image uploadImage0(File imageFile) {
if(messageObject instanceof FriendMessage) {
return messageObject.getSender().uploadImage(imageFile);
} else if(messageObject instanceof GroupMessage) {
return ((GroupMessage) messageObject).getGroup().uploadImage(imageFile);
} else {
log.warn("未知的ContactMessage类型: " + messageObject.toString());
return null;
}
}
private String getMessageSource() {
if(messageObject instanceof FriendMessage) {
return "Private";
} else if(messageObject instanceof GroupMessage) {
return "Group";
} else {
log.warn("未知的ContactMessage类型: " + messageObject.toString());
return "Unknown";
}
return member.uploadImage(imageFile);
}
}

View File

@ -0,0 +1,28 @@
package net.lamgc.cgj.bot.framework.mirai.message;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.Bot;
public class MiraiMessageSenderFactory implements MessageSenderFactory {
private final Bot bot;
public MiraiMessageSenderFactory(Bot bot) {
this.bot = bot;
}
@Override
public MessageSender createMessageSender(MessageSource source, long id) throws Exception {
switch(source) {
case Group:
case Discuss:
return new MiraiMessageSender(bot.getGroup(id), source);
case Private:
return new MiraiMessageSender(bot.getFriend(id), source);
default:
throw new NoSuchFieldException(source.toString());
}
}
}

View File

@ -0,0 +1,15 @@
package net.lamgc.cgj.bot.message;
public interface MessageSender {
/**
* 发送消息并返回消息id
* @param message 消息内容
* @return 返回非负数则发送成功,
* 返回0则发送器不支持消息Id,
* 返回非0正整数则为消息Id,
* 返回负数则为错误.
*/
int sendMessage(final String message);
}

View File

@ -0,0 +1,43 @@
package net.lamgc.cgj.bot.message;
import java.util.concurrent.atomic.AtomicReference;
/**
* 消息发送器构造
*/
public final class MessageSenderBuilder {
private final static AtomicReference<MessageSenderFactory> currentFactory = new AtomicReference<>();
private MessageSenderBuilder() {}
/**
* 获取消息发送器
* @param source 消息源类型
* @param id 消息源Id
* @return 返回新建的发送器
*/
public static MessageSender getMessageSender(MessageSource source, long id) {
MessageSenderFactory messageSenderFactory = currentFactory.get();
if(messageSenderFactory == null) {
throw new IllegalStateException("The factory is not ready");
}
try {
return messageSenderFactory.createMessageSender(source, id);
} catch (Exception e) {
throw new IllegalStateException(e);
}
}
/**
* 设置消息发送器工厂
* @param factory 工厂对象
*/
public static void setCurrentMessageSenderFactory(MessageSenderFactory factory) {
if(currentFactory.get() != null) {
throw new IllegalStateException("Factory already exists");
}
currentFactory.set(factory);
}
}

View File

@ -0,0 +1,13 @@
package net.lamgc.cgj.bot.message;
public interface MessageSenderFactory {
/**
* 通过Id创建发送器
* @param source 消息源
* @param id 消息源id
* @return 如果成功返回MessageSender
*/
MessageSender createMessageSender(MessageSource source, long id) throws Exception;
}

View File

@ -0,0 +1,23 @@
package net.lamgc.cgj.bot.message;
/**
* 消息来源
*/
public enum MessageSource {
/**
* 私聊消息
*/
Private,
/**
* 群组消息
*/
Group,
/**
* 讨论组消息
*/
Discuss,
/**
* 未知来源
*/
Unknown
}

View File

@ -71,7 +71,7 @@ public class PreLoadDataComparator implements Comparator<JsonElement> {
VIEW("viewCount"),
;
final String attrName;
public final String attrName;
Attribute(String attrName) {
this.attrName = attrName;

View File

@ -0,0 +1,33 @@
package net.lamgc.cgj.pixiv;
public interface AdultContentDetector {
/**
* 检查某一作品的成人内容判断指数
* @param illustId 作品Id
* @param isUgoira 是否为动图
* @param pageIndex 指定页数, 设为0或负数则视为单页面作品
* @return 返回成人作品判断指数(0 ~ 1), 需按照情况设置阀值.
*/
double detect(int illustId, boolean isUgoira, int pageIndex) throws Exception;
/**
* 检查某一作品是否为成人内容
* @param illustId 作品Id
* @param isUgoira 是否为动图
* @param pageIndex 指定页数, 设为0或负数则视为单页面作品
* @return 如果为true则为成人作品, 该方法将由检测器决定如何为成人作品.
*/
boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex) throws Exception;
/**
* 检查某一作品是否为成人内容
* @param illustId 作品Id
* @param isUgoira 是否为动图
* @param pageIndex 指定页数, 设为0或负数则视为单页面作品
* @param threshold 指数阀值, 当等于或大于该阀值时返回true.
* @return 如果为true则为成人作品, 该方法将由 threshold 参数决定是否为成人作品.
*/
boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex, double threshold) throws Exception;
}

View File

@ -1,69 +0,0 @@
package net.lamgc.cgj.pixiv;
/**
* 插图(集)的信息
*/
public class IllustInfo{
/**
* 插图(集)ID
*/
public final int illustID;
/**
* 第几页
*/
public final int page;
/**
* 插图标题
*/
public final String title;
/**
* 插图说明
*/
public final String description;
/**
* 插图标签
*/
public final String[] tags;
/**
* 插图图片长度
*/
public final int width;
/**
* 插图图片高度
*/
public final int height;
/**
* 作者名
*/
public final String authorName;
/**
* 作者用户ID
*/
public final int authorUserID;
public IllustInfo(int illustID, String title, String description, String[] tags, int width, int height, String authorName, int authorUserID){
this(illustID, 0, title, description, tags, width, height, authorName, authorUserID);
}
public IllustInfo(int illustID, int p, String title, String description, String[] tags, int width, int height, String authorName, int authorUserID){
this.illustID = illustID;
this.page = p;
this.title = title;
this.description = description;
this.tags = tags;
this.width = width;
this.height = height;
this.authorName = authorName;
this.authorUserID = authorUserID;
}
}

View File

@ -0,0 +1,65 @@
package net.lamgc.cgj.pixiv;
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
import java.io.IOException;
/**
* 使用ModerateContent服务开发的检测器.<br/>
* ModerateContent: www.moderatecontent.com
*/
public class ModerateContentDetector implements AdultContentDetector {
private final static HttpClient httpClient = HttpClientBuilder.create().build();
private final static Gson gson = new Gson();
private final String requestUrl;
private final static String API_URL = "https://www.moderatecontent.com/api/v2?key={key}&url=https://pixiv.cat/";
/**
* 创建一个使用ModerateContent鉴黄服务的检测器
* @param apiKey API密钥
*/
public ModerateContentDetector(String apiKey) {
requestUrl = API_URL.replace("{key}", apiKey);
}
private JsonObject accessInterface(int illustId, boolean isUgoira, int pageIndex) throws IOException {
HttpResponse response;
if(pageIndex <= 0) {
response = httpClient.execute(new HttpGet(requestUrl + illustId + (isUgoira ? ".gif" : ".jpg")));
} else {
response = httpClient.execute(new HttpGet(requestUrl + illustId + "-" + pageIndex + (isUgoira ? ".gif" : ".jpg")));
}
if(response.getStatusLine().getStatusCode() != 200) {
throw new IOException("Http response error: " + response.getStatusLine());
}
JsonObject result = gson.fromJson(EntityUtils.toString(response.getEntity()), JsonObject.class);
if (result.get("error_code").getAsInt() != 0) {
throw new IOException("Interface result error: " + (result.has("error") ? result.get("error").getAsString() : "(error message is empty)"));
}
return result;
}
@Override
public double detect(int illustId, boolean isUgoira, int pageIndex) throws IOException {
return accessInterface(illustId, isUgoira, pageIndex).getAsJsonObject("predictions").get("adult").getAsDouble();
}
@Override
public boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex) throws IOException {
return accessInterface(illustId, isUgoira, pageIndex).get("rating_index").getAsInt() == 3;
}
@Override
public boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex, double threshold) throws IOException {
return detect(illustId, isUgoira, pageIndex) >= threshold;
}
}

View File

@ -1,267 +0,0 @@
package net.lamgc.cgj.pixiv;
import com.google.gson.JsonArray;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.conn.ConnectionPoolTimeoutException;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.*;
public class Pixiv {
/**
* illust Link
*/
public final static String ATTR_LINK = "link";
/**
* illust Id
*/
public final static String ATTR_ILLUST_ID = "illustId";
/**
* illust Title
*/
public final static String ATTR_TITLE = "title";
/**
* illust Author Name
*/
public final static String ATTR_AUTHOR_NAME = "authorName";
/**
* illust Author UserID
*/
public final static String ATTR_AUTHOR_ID = "authorId";
private final HttpClient httpClient;
public Pixiv(HttpClient client){
this.httpClient = client;
}
/**
* 使用帐号密码登录Pixiv
* @param PixivID Pixiv账户登录名
* @param Password Pixiv帐号密码
* @throws IOException 当登录连接出现异常时抛出
* @deprecated {@link PixivSession#Login(String, String)} 已经废除, 故本方法不可用
*/
public Pixiv(String PixivID, String Password) throws IOException {
this(PixivID, Password, null);
}
/**
* 使用帐号密码登录Pixiv
* @param PixivID Pixiv账户登录名
* @param Password Pixiv帐号密码
* @param proxy 代理设置
* @throws IOException 当登录连接出现异常时抛出
* @deprecated {@link PixivSession#Login(String, String)} 已经废除, 故本方法不可用
*/
public Pixiv(String PixivID, String Password, HttpHost proxy) throws IOException {
PixivSession pixivSession = new PixivSession(proxy, null);
if(pixivSession.Login(PixivID, Password)){
System.out.println("P站登录成功!");
}else{
System.out.println("P站登录失败!错误信息: " + pixivSession.getErrMsg());
throw new RuntimeException(pixivSession.getErrMsg());
}
//httpClient = pixivSession.getHttpClient();
httpClient = HttpClientBuilder.create()
.setDefaultCookieStore(pixivSession.getCookieStore())
.build();
}
/**
* 获取首页推荐列表
* @return 首页推荐列表, 一个Map对应一个推荐项, 使用<code>ATTR_</code>开头常量访问即可
* @throws IOException
*/
public List<Map<String, String>> getRecommend() throws IOException {
HttpGet getRequest = new HttpGet(PixivURL.PIXIV_INDEX_URL);
HttpResponse response = httpClient.execute(getRequest);
String pageAsXML = EntityUtils.toString(response.getEntity(),"utf-8");
//获取推荐图列表(li)
//System.out.println(pageAsXML);
Document document = Jsoup.parse(pageAsXML);
List<String> links = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-thumbnail-link").eachAttr("href");
List<String> illustId = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-thumbnail-link").eachAttr("data-gtm-recommend-illust-id");
List<String> title = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-title>h1").eachAttr("title");
List<String> authorName = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-user-name").eachText();
List<String> authorId = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-user-name").eachAttr("data-user_id");
List<Map<String, String>> recommendList = new ArrayList<>();
for(int i = 0; i < links.size(); i++){
//System.out.println(links.get(i));
Map<String, String> info = new HashMap<>();
info.put(ATTR_LINK, PixivURL.PIXIV_INDEX_URL + links.get(i));
info.put(ATTR_ILLUST_ID, illustId.get(i));
info.put(ATTR_TITLE, title.get(i));
info.put(ATTR_AUTHOR_NAME, authorName.get(i));
info.put(ATTR_AUTHOR_ID, authorId.get(i));
recommendList.add(info);
}
return recommendList;
}
public String[] getAllDownloadLink(int illustID) throws IOException {
HttpGet illustPage = new HttpGet(PixivURL.PIXIV_ILLUST_API_URL.replaceAll("\\{illustId}", String.valueOf(illustID)));
HttpResponse response = httpClient.execute(illustPage);
String pageAsXML = EntityUtils.toString(response.getEntity(),"utf-8");
//System.out.println(pageAsXML);
JsonObject resultObj = (JsonObject) new JsonParser().parse(pageAsXML);
if(!resultObj.get("error").getAsBoolean()){
JsonArray bodyArray = resultObj.get("body").getAsJsonArray();
int length = bodyArray.size();
String[] result = new String[length];
for(int i = 0; i < length; i++){
JsonObject childObj = bodyArray.get(i).getAsJsonObject();
result[i] = childObj.get("urls").getAsJsonObject().get("original").getAsString();
}
return result;
}else{
return null;
}
}
/**
* 下载P站图片
* @param illustID 插图ID
* @return 成功返回图片输入流,失败或为多图则返回null
*/
public InputStream[] downloadIllustImage(int illustID) throws IOException {
String[] links = getAllDownloadLink(illustID);
List<InputStream> inputStreamList = new ArrayList<>();
int count = 1;
boolean retry = false;
for(int i = 0; i < links.length; i++){
try {
long sleepTime = (new Random().nextInt(4) + 2) * 1000;
System.out.println("nextTime: " + (float)(sleepTime / 1000));
Thread.sleep(sleepTime);
} catch (InterruptedException ignored) {}
String link = links[i];
System.out.print("page:" + count++ + "/" + links.length + " ...");
HttpGet imgGet = new HttpGet(link);
//关键!如果不加上Referer的话,会返回403
imgGet.setHeader("Referer", PixivURL.PIXIV_ILLUST_MEDIUM_URL.replaceAll("\\{illustId}", String.valueOf(illustID)));
RequestConfig config = RequestConfig.custom()
.setConnectTimeout(20 * 1000)
.setConnectionRequestTimeout(20 * 1000)
.setSocketTimeout(30 * 1000)
.build();
imgGet.setConfig(config);
HttpResponse response;
try {
response = httpClient.execute(imgGet);
}catch(ConnectionPoolTimeoutException e){
if(retry){
retry = false;
System.out.println("获取失败,跳过...");
continue;
}
System.out.println("连接超时,重新获取...");
retry = true;
i--;
continue;
}
retry = false;
ByteArrayOutputStream cacheOutputStream = new ByteArrayOutputStream((int)response.getEntity().getContentLength());
InputStream content = response.getEntity().getContent();
int readLength;
byte[] cache = new byte[4096];
while((readLength = content.read(cache)) != -1){
cacheOutputStream.write(cache, 0, readLength);
}
byte[] data = cacheOutputStream.toByteArray();
//System.out.println("读到数据: " + data.length);
inputStreamList.add(new ByteArrayInputStream(data));
System.out.println("done!length: " + response.getEntity().getContentLength() + ")");
}
return inputStreamList.toArray(new InputStream[0]);
}
/**
* 下载P站图片
* @param illustID 插图ID
* @return 成功返回图片输入流,失败或为多图则返回null
*/
public InputStream downloadIllustImages(int illustID){
throw new UnsupportedOperationException();
}
/**
* 通过解析插图详情页获取
* - 插图标题
* - 插图作者(及其UserId)
* - 插图上传时间
* - 插图标签(原始标签)
* ...
* @return 成功返回IllustInfo对象,失败返回null
*/
public IllustInfo[] getIllustInfo(int[] illustIDs) throws IOException {
//获取Api
HttpGet apiRequest = new HttpGet(PixivURL.getPixivIllustInfoAPI(illustIDs));
HttpResponse response = httpClient.execute(apiRequest);
String resultText = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
System.out.println(resultText);
JsonObject resultObj = ((JsonObject) new JsonParser().parse(resultText));
if(resultObj.get("error").getAsBoolean()){
System.err.println("获取失败!");
return null;
}
List<IllustInfo> illustInfoList = new ArrayList<>();
JsonArray illustArray = resultObj.get("body").getAsJsonObject().get("illusts").getAsJsonArray();
illustArray.forEach(jsonElement -> {
JsonObject illustInfoObj = jsonElement.getAsJsonObject();
JsonArray tagsArray = illustInfoObj.get("tags").getAsJsonArray();
String[] tags = new String[tagsArray.size()];
for(int i = 0; i < tags.length; i++){
tags[i] = tagsArray.get(i).getAsString();
}
//TODO: 通过不需要作者id就能获取图片信息的api无法获取图片尺寸
IllustInfo illustInfo = new IllustInfo(
illustInfoObj.get("workId").getAsInt(),
illustInfoObj.get("title").getAsString(),
null,
tags,
-1,
-1,
illustInfoObj.get("userName").getAsString(),
illustInfoObj.get("userId").getAsInt()
);
});
return null;
}
/**
* 获取指定用户的所有插画
*/
public void getUserAllIllustTest() {
}
}

View File

@ -4,6 +4,7 @@ import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import com.google.gson.reflect.TypeToken;
import io.netty.handler.codec.http.HttpHeaderNames;
import org.apache.http.Header;
import org.apache.http.HttpHost;
@ -256,8 +257,8 @@ public class PixivDownload {
public static List<JsonObject> getRanking(List<JsonObject> rankingList, int rankStart, int range) {
log.debug("正在读取JsonArray...(rankStart: {}, range: {})", rankStart, range);
ArrayList<JsonObject> results = new ArrayList<>(rankingList.size());
for (int rankIndex = rankStart; rankIndex < rankingList.size() && rankIndex < range; rankIndex++) {
JsonElement jsonElement = rankingList.get(rankIndex);
for (int rankIndex = rankStart; rankIndex < rankStart + range; rankIndex++) {
JsonElement jsonElement = rankingList.get(rankIndex - rankStart);
JsonObject rankInfo = jsonElement.getAsJsonObject();
int rank = rankInfo.get("rank").getAsInt();
int illustId = rankInfo.get("illust_id").getAsInt();
@ -279,90 +280,64 @@ public class PixivDownload {
* @return 返回List对象
*/
public static List<JsonObject> getRanking(JsonArray rankingArray, int rankStart, int range) {
log.debug("正在读取JsonArray...(rankStart: {}, range: {})", rankStart, range);
ArrayList<JsonObject> results = new ArrayList<>(rankingArray.size());
for (int rankIndex = rankStart; rankIndex < rankingArray.size() && rankIndex < range; rankIndex++) {
JsonElement jsonElement = rankingArray.get(rankIndex);
JsonObject rankInfo = jsonElement.getAsJsonObject();
int rank = rankInfo.get("rank").getAsInt();
int illustId = rankInfo.get("illust_id").getAsInt();
int authorId = rankInfo.get("user_id").getAsInt();
String authorName = rankInfo.get("user_name").getAsString();
String title = rankInfo.get("title").getAsString();
log.debug("Array-当前到第 {}/{} 名(总共 {} 名), IllustID: {}, Author: ({}) {}, Title: {}", rank, rankStart + range, range, illustId, authorId, authorName, title);
results.add(rankInfo);
}
log.debug("JsonArray读取完成.");
return results;
List<JsonObject> list = new Gson().fromJson(rankingArray, new TypeToken<List<JsonObject>>(){}.getType());
return getRanking(list, rankStart, range);
}
/**
* 获取排行榜
* 获取排行榜.
* <p>注意: 如果范围实际上没超出, 但返回排行榜不足, 会导致与实际请求的数量不符, 需要检查</p>
* @param contentType 排行榜类型
* @param mode 排行榜模式
* @param time 查询时间
* @param rankStart 开始排名, 从1开始
* @param range 取范围
* @return 成功返回有值List, 失败且无异常返回空
* @throws IOException 获取异常时抛出
* @throws IllegalArgumentException 当{@linkplain net.lamgc.cgj.pixiv.PixivURL.RankingContentType RankingContentType}
* 与{@linkplain net.lamgc.cgj.pixiv.PixivURL.RankingMode RankingMode}互不兼容时抛出
* @throws IndexOutOfBoundsException 当排行榜选取范围超出排行榜范围时抛出(排行榜范围为 1 ~ 500 名)
* @throws IOException 当Http请求发生异常时抛出, 或Http请求响应码非200时抛出
*/
public List<JsonObject> getRanking(PixivURL.RankingContentType contentType, PixivURL.RankingMode mode,
Date time, int rankStart, int range) throws IOException {
if(rankStart <= 0) {
throw new IllegalArgumentException("rankStart cannot be less than or equal to zero");
}
if(range <= 0) {
throw new IllegalArgumentException("range cannot be less than or equal to zero");
}
if(!contentType.isSupportedMode(mode)) {
Date time, int rankStart, int range) throws IOException {
Objects.requireNonNull(time);
if(!Objects.requireNonNull(contentType).isSupportedMode(Objects.requireNonNull(mode))) {
throw new IllegalArgumentException("ContentType不支持指定的RankingMode: ContentType: " + contentType.name() + ", Mode: " + mode.name());
} else if(rankStart <= 0) {
throw new IndexOutOfBoundsException("rankStart cannot be less than or equal to zero: " + rankStart);
} else if(range <= 0) {
throw new IndexOutOfBoundsException("range cannot be less than or equal to zero:" + range);
} else if(rankStart + range - 1 > 500) {
throw new IndexOutOfBoundsException("排名选取范围超出了排行榜范围: rankStart=" + rankStart + ", range=" + range + ", length:" + (rankStart + range - 1));
}
int startPage = (int) Math.ceil(rankStart / 50F);
int requestFrequency = (int) Math.ceil((rankStart + (range - 1)) / 50F);
int surplusQuantity = range;
boolean firstRequest = true;
int startPages = (int) Math.max(1, Math.floor(rankStart / 50F));
int endPages = (int) Math.min(10, Math.ceil((rankStart + range) / 50F));
int startIndex = rankStart - 1;
int count = 0;
Gson gson = new Gson();
ArrayList<JsonObject> results = new ArrayList<>();
for (int requestCount = startPage; requestCount <= requestFrequency && requestCount <= 10; requestCount++) {
int rangeStart = (requestCount - 1) * 50 + 1;
log.debug("正在请求第 {} 到 {} 位排名榜 (第{}次请求, 共 {} 次)", rangeStart, rangeStart + 49, requestCount - startPage + 1, requestFrequency - startPage + 1);
HttpGet request = createHttpGetRequest(PixivURL.getRankingLink(contentType, mode, time, requestCount, true));
log.debug("Request URL: {}", request.getURI());
ArrayList<JsonObject> results = new ArrayList<>(range);
boolean canNext = true;
for (int pageIndex = startPages; canNext && pageIndex <= endPages && count < range; pageIndex++) {
HttpGet request = createHttpGetRequest(PixivURL.getRankingLink(contentType, mode, time, pageIndex, true));
log.debug("RequestUri: {}", request.getURI());
HttpResponse response = httpClient.execute(request);
String content = EntityUtils.toString(response.getEntity());
log.debug("Content: " + content);
JsonObject contentObject = gson.fromJson(content, JsonObject.class);
if(contentObject.has("error")) {
log.warn("接口报错, 返回信息: {}", contentObject.get("error").getAsString());
break;
String responseBody = EntityUtils.toString(response.getEntity());
log.debug("ResponseBody: {}", responseBody);
if(response.getStatusLine().getStatusCode() != 200) {
throw new IOException("Http Response Error: '" + response.getStatusLine() + "', ResponseBody: '" + responseBody + '\'');
}
JsonArray rankingArray = contentObject.getAsJsonArray("contents");
log.debug("正在解析数据...");
//需要添加一个总量, 否则会完整跑完一次.
//检查是否为最后一次请求,和剩余量有多少
int firstRequestStartIndex = (rankStart % 50) - 1;
for (int rankIndex = firstRequest ? firstRequestStartIndex : 0; rankIndex < rankingArray.size() && surplusQuantity > 0; rankIndex++, surplusQuantity--) {
JsonElement jsonElement = rankingArray.get(rankIndex);
JsonObject rankInfo = jsonElement.getAsJsonObject();
int rank = rankInfo.get("rank").getAsInt();
int illustId = rankInfo.get("illust_id").getAsInt();
int authorId = rankInfo.get("user_id").getAsInt();
String authorName = rankInfo.get("user_name").getAsString();
String title = rankInfo.get("title").getAsString();
log.debug("Download-当前到第 {}/{} 名(总共 {} 名), IllustID: {}, Author: ({}) {}, Title: {}", rank, rankStart + range - 1, range, illustId, authorId, authorName, title);
results.add(rankInfo);
JsonObject resultObject = gson.fromJson(responseBody, JsonObject.class);
canNext = resultObject.get("next").getAsJsonPrimitive().isNumber();
JsonArray resultArray = resultObject.getAsJsonArray("contents");
for (int resultIndex = startIndex; resultIndex < resultArray.size() && count < range; resultIndex++, count++) {
results.add(resultArray.get(resultIndex).getAsJsonObject());
}
firstRequest = false;
log.debug("第 {} 到 {} 位排名榜完成. (第{}次请求)", rangeStart, rangeStart + 49, requestCount);
}
if(requestFrequency > 10) {
log.warn("请求的排名榜范围超出所支持的范围, 已终止请求.");
// 重置索引
startIndex = 0;
}
return results;
}
@ -463,13 +438,14 @@ public class PixivDownload {
THUMB_MINI
}
/**
* 获取帐号所有的收藏插画,并以输入流形式提供
* @return 获取所有链接的InputStream, 请注意关闭InputStream
* @throws IOException 当获取时发生异常则直接抛出
* @deprecated 该方法可能会导致已经打开的InputStream超时, 使图片获取失败,
* 请直接使用{@linkplain #getCollectionAsInputStream(PageQuality, BiConsumer)}
*/
@Deprecated
public Set<Map.Entry<String, InputStream>> getCollectionAsInputStream(PageQuality quality) throws IOException {
HashSet<Map.Entry<String, InputStream>> illustInputStreamSet = new HashSet<>();
getCollectionAsInputStream(quality, (link, inputStream) -> illustInputStreamSet.add(new AbstractMap.SimpleEntry<>(link, inputStream)));
@ -630,12 +606,29 @@ public class PixivDownload {
return null;
}
/**
* 通过预加载数据获取作品类型
* @param illustId 作品Id
* @param preLoadDataObject 预加载数据(IllustInfo也可以)
* @return 如果存在illustType属性, 则返回对应项, 如没有, 或数据内不存在指定作品id的数据, 返回null
*/
public static PixivIllustType getIllustTypeByPreLoadData(int illustId, JsonObject preLoadDataObject) {
JsonObject illustData;
JsonObject illustData = null;
if(preLoadDataObject.has("illust")) {
illustData = preLoadDataObject.getAsJsonObject("illust").getAsJsonObject(String.valueOf(illustId));
} else if(preLoadDataObject.has(String.valueOf(illustId))) {
illustData = preLoadDataObject.getAsJsonObject(String.valueOf(illustId));
} else if(preLoadDataObject.has("body")) { // 解析IllustInfo
for (JsonElement jsonElement : preLoadDataObject.getAsJsonObject("body").getAsJsonArray("illusts")) {
JsonObject illustInfo = jsonElement.getAsJsonObject();
if (illustInfo.get("illustId").getAsInt() == illustId) {
illustData = illustInfo;
break;
}
}
if(illustData == null) {
return null;
}
} else {
illustData = preLoadDataObject;
}

View File

@ -9,6 +9,13 @@ import java.util.Date;
import java.util.HashSet;
import java.util.Objects;
/**
* Pixiv搜索URL构造器
* <p>该构造器通过分析Pixiv搜索链接可用的参数而开发, 对搜索链接的构造有高度自定义能力.</p>
* @author LamGC
* @see PixivURL#PIXIV_SEARCH_CONTENT_URL
*/
@SuppressWarnings("ALL")
public class PixivSearchBuilder {
private final String content;

View File

@ -1,171 +0,0 @@
package net.lamgc.cgj.pixiv;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import org.apache.http.Header;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.CookieStore;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicHeader;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
public class PixivSession {
public static final String USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36";
/**
* 全登陆过程的关键,
* 保存会话用的cookieStore!
*/
private CookieStore cookieStore = new BasicCookieStore();
/**
* 可以直接使用的HttpClient对象
*/
private HttpClient httpClient;
/**
* 最后一次登录的错误信息
*/
private String errMsg;
public PixivSession(){
this(null);
}
public PixivSession(CookieStore cookieStore){
this(null, cookieStore);
}
/**
* 创建一个Pixiv登录会话
*/
public PixivSession(HttpHost proxy, CookieStore cookieStore) {
if(cookieStore != null){
this.cookieStore = cookieStore;
}
List<Header> defaultHeader = new ArrayList<>();
defaultHeader.add(new BasicHeader("accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"));
defaultHeader.add(new BasicHeader("user-agent", PixivSession.USER_AGENT));
defaultHeader.add(new BasicHeader("accept-encoding", "gzip, deflate, br"));
defaultHeader.add(new BasicHeader("accept-language", "zh-CN,zh;q=0.9"));
/*defaultHeader.add(new BasicHeader("sec-fetch-mode", "navigate"));
defaultHeader.add(new BasicHeader("sec-fetch-site", "same-origin"));
defaultHeader.add(new BasicHeader("upgrade-insecure-requests", "1"));*/
//创建一个Http访问器
httpClient = HttpClients.custom()
.setDefaultCookieStore(cookieStore)
.setDefaultHeaders(defaultHeader)
.setProxy(proxy)
.build();
}
/**
* 程序自行通过帐号密码登录Pixiv.
* @param PixivID Pixiv帐号
* @param Password Pixiv密码
* @return 登录成功返回true
* @throws IOException 当登录抛出异常时返回
* @deprecated Pixiv已经新增Google人机验证, 程序已无法自行登录Pixiv
*/
public boolean Login(String PixivID, String Password) throws IOException {
// 获取登录接口所需的PostKey
String post_key = getPostKey();
HttpPost postRequest = new HttpPost(PixivURL.PIXIV_LOGIN_URL); //https://accounts.pixiv.net/api/login?lang=zh
List<NameValuePair> params = new ArrayList<>();
params.add(new BasicNameValuePair("pixiv_id", PixivID));
params.add(new BasicNameValuePair("password", Password));
params.add(new BasicNameValuePair("post_key", post_key));
//Form编码表单,作为Post的数据
postRequest.setEntity(new UrlEncodedFormEntity(params, StandardCharsets.UTF_8));
//启动访问
HttpResponse response = httpClient.execute(postRequest);
//获取接口返回数据
String httpXML = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
System.out.println(httpXML);
JsonObject responseJson = (JsonObject) new JsonParser().parse(httpXML);
if(!responseJson.get("error").getAsBoolean() && !responseJson.get("body").getAsJsonObject().has("validation_errors")){
errMsg = null;
return true;
}else{
errMsg = responseJson.get("body").getAsJsonObject().get("validation_errors").toString();
//System.err.println("登录失败MSG: " + errMsg);
return false;
}
}
/**
* 登录前准备, 获取PostKey
* @return Post_Key
*/
private String getPostKey() throws IOException {
//创建请求,获取PostKey
HttpGet getRequest = new HttpGet(PixivURL.PIXIV_LOGIN_PAGE_URL);
//设置请求
//getRequest.setConfig(config);
getRequest.setHeader("User-Agent", USER_AGENT);
//启动访问
HttpResponse response = httpClient.execute(getRequest);
//获取网页内容
String pageAsXML = EntityUtils.toString(response.getEntity(),"utf-8");
//创建Http解析器
Document document = Jsoup.parse(pageAsXML);
//获取init-config内容
String init_config = document.getElementById("init-config").val();
//System.out.println(init_config);
//创建Json解析器解析init-config
JsonObject initConfigObj = (JsonObject) new JsonParser().parse(init_config);
//检查是否有postKey
if(!initConfigObj.has("pixivAccount.postKey")){
throw new RuntimeException("postKey获取失败!可能是Pixiv修改了登录过程!");
}
//获取postKey
return initConfigObj.get("pixivAccount.postKey").getAsString();
}
/**
* 获取CookieStore
* @return CookieStore
*/
public CookieStore getCookieStore(){
return cookieStore;
}
/**
* 获取可直接使用的HttpClient对象
* @return 已配置好的HttpClient对象
*/
public HttpClient getHttpClient(){
return this.httpClient;
}
public boolean hasError(){
return errMsg == null;
}
/**
* 获取错误信息
* @return 返回登录错误信息
* @deprecated {@link #Login(String, String)}已经废除, 故本接口废除
*/
public String getErrMsg(){
return errMsg;
}
}

View File

@ -1,16 +0,0 @@
package net.lamgc.cgj.pixiv;
public class PixivTag {
public static PixivTag create(String tagName) {
return null;
}
private PixivTag() {
}
}

View File

@ -9,6 +9,7 @@ import java.util.GregorianCalendar;
/**
* 目前已整理的一些Pixiv接口列表
*/
@SuppressWarnings("unused")
public class PixivURL {
@ -52,10 +53,10 @@ public class PixivURL {
/**
* P站用户插图列表获取API
* <p>所需数据在 body属性内的 illusts(属性名,属性值不重要), manga(多图) pickup(精选)</p>
* 需要替换的文本:
* {userId} - 用户ID
*/
//TODO: 所需数据在 body属性内的 illusts(属性名,属性值不重要), manga(多图) pickup(精选)
//{"error":false,"message":"","body":{"illusts":{"74369837":null,"70990542":null,"70608653":null,"69755191":null,"69729450":null,"69729416":null,"69503608":null,"69288766":null,"69083882":null,"69051458":null,"68484200":null,"68216927":null,"68216866":null,"68192333":null,"67915106":null,"67914932":null,"67854803":null,"67854745":null,"67854670":null,"67787211":null,"67772199":null,"67770637":null,"67754861":null,"67754804":null,"67754726":null,"67740486":null,"67740480":null,"67740450":null,"67740434":null,"67726337":null,"67499196":null,"67499163":null,"67499145":null,"67499111":null,"67499085":null,"67499038":null,"67498987":null,"67473178":null,"66271465":null,"63682753":null,"63682697":null,"59385148":null,"59383265":null,"59383240":null,"59383227":null,"59383173":null},"manga":[],"novels":[],"mangaSeries":[],"novelSeries":[],"pickup":[],"bookmarkCount":{"public":{"illust":1,"novel":0},"private":{"illust":0,"novel":0}}}}
public static final String PIXIV_USER_ILLUST_LIST_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/all";
@ -186,7 +187,7 @@ public class PixivURL {
public static String getRankingLink(RankingContentType contentType, RankingMode mode, Date time, int pageIndex, boolean json){
StringBuilder linkBuilder = new StringBuilder(PIXIV_RANKING_LINK);
linkBuilder.append("mode=").append(mode == null ? RankingMode.MODE_DAILY.modeParam : mode.modeParam);
if(contentType != null && !contentType.equals(RankingContentType.ALL)){
if(contentType != null && !contentType.equals(RankingContentType.TYPE_ALL)){
linkBuilder.append("&content=").append(contentType.typeName);
}
SimpleDateFormat format = new SimpleDateFormat("yyyyMMdd");
@ -278,7 +279,7 @@ public class PixivURL {
* 排名榜类型
*/
public enum RankingContentType{
ALL("", RankingMode.values()),
TYPE_ALL("", RankingMode.values()),
/**
* 插画
* 支持的时间类型: 每天, 每周, 每月, 新人
@ -336,7 +337,7 @@ public class PixivURL {
* @return 如果支持返回true
*/
public boolean isSupportedMode(RankingMode mode) {
return Arrays.binarySearch(supportedMode, mode) != -1;
return Arrays.binarySearch(supportedMode, mode) >= 0;
}
}

View File

@ -26,8 +26,7 @@ import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
/**
* Pixiv动图构建器.
* 可便捷的接收并处理动图.
* Pixiv动图构建器
*/
public final class PixivUgoiraBuilder {
@ -135,7 +134,7 @@ public final class PixivUgoiraBuilder {
image.getRGB(0, 0, image.getWidth(), image.getHeight(), rgb, 0, image.getWidth());
log.trace("帧解析完成, 正在插入...");
encoder.addImage(Image.fromRgb(rgb, image.getWidth()), new ImageOptions().setDelay(frameInfo.get("delay").getAsLong(), TimeUnit.MILLISECONDS));
log.debug("帧 {} 插入完成.", frameFileName);
log.trace("帧 {} 插入完成.", frameFileName);
} catch (IOException e) {
log.error("解析帧图片数据时发生异常", e);
}

View File

@ -1,31 +0,0 @@
package net.lamgc.cgj.util;
import org.apache.http.client.CookieStore;
import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.cookie.BasicClientCookie;
import java.util.Date;
public class CookieUtil {
/**
* 将{@link java.net.CookieStore}转换到{@link CookieStore}
* @param netCookieStore java.net.CookieStore
* @return org.apache.http.client.CookieStore
*/
public static CookieStore NAParse(java.net.CookieStore netCookieStore){
CookieStore apacheCookieStore = new BasicCookieStore();
netCookieStore.getCookies().forEach(netCookie -> {
BasicClientCookie aCookie = new BasicClientCookie(netCookie.getName(), netCookie.getValue());
aCookie.setComment(netCookie.getComment());
aCookie.setDomain(netCookie.getDomain());
aCookie.setExpiryDate(new Date(netCookie.getMaxAge()));
aCookie.setPath(netCookie.getPath());
aCookie.setSecure(netCookie.getSecure());
aCookie.setVersion(netCookie.getVersion());
apacheCookieStore.addCookie(aCookie);
});
return apacheCookieStore;
}
}

View File

@ -7,7 +7,7 @@ import net.lamgc.utils.base.runner.StringParameterParser;
public class PagesQualityParser implements StringParameterParser<PixivDownload.PageQuality> {
@Override
public PixivDownload.PageQuality parse(String strValue) throws Exception {
public PixivDownload.PageQuality parse(String strValue) {
return PixivDownload.PageQuality.valueOf(strValue.toUpperCase());
}

View File

@ -0,0 +1,151 @@
package net.lamgc.cgj.util;
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicLong;
/**
* 带有时间限制的线程池.
* 当线程超出了限制时间时, 将会对该线程发出中断.
*/
public class TimeLimitThreadPoolExecutor extends ThreadPoolExecutor {
/**
* 执行时间限制, 单位毫秒.
* 默认30s.
*/
private final AtomicLong executeTimeLimit = new AtomicLong();
/**
* 检查间隔时间.
* 默认100ms.
*/
private final AtomicLong timeoutCheckInterval = new AtomicLong(100);
private final Map<Thread, AtomicLong> workerThreadMap = new Hashtable<>();
private final Thread timeoutCheckThread = createTimeoutCheckThread();
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
setInitialTime(0, executeLimitTime);
timeoutCheckThread.start();
}
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory);
setInitialTime(0, executeLimitTime);
timeoutCheckThread.start();
}
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, RejectedExecutionHandler handler) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, handler);
setInitialTime(0, executeLimitTime);
timeoutCheckThread.start();
}
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory, RejectedExecutionHandler handler) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler);
setInitialTime(0, executeLimitTime);
timeoutCheckThread.start();
}
private void setInitialTime(long checkInterval, long executeLimitTime) {
if(checkInterval > 0) {
timeoutCheckInterval.set(checkInterval);
}
if(executeLimitTime > 0) {
executeTimeLimit.set(executeLimitTime);
}
}
/**
* 设置执行时间.
* <p>注意: 该修改仅在线程池完全停止后才有效</p>
* @see #isTerminated()
* @param time 新的限制时间(ms)
*/
public void setExecuteTimeLimit(long time) {
if(time <= 0) {
throw new IllegalArgumentException("Time is not allowed to be set to 0 or less");
}
if(this.isTerminated()) {
executeTimeLimit.set(time);
}
}
/**
* 设置超时检查间隔.
* <p>该方法仅会在当前检查后生效.</p>
* @param time 新的检查间隔(ms)
*/
public void setTimeoutCheckInterval(long time) {
if(time <= 0) {
throw new IllegalArgumentException("Time is not allowed to be set to 0 or less");
}
timeoutCheckInterval.set(time);
}
/**
* 获取当前设置的执行时间限制.
* @return 执行时间限制(ms).
*/
public long getExecuteTimeLimit() {
return executeTimeLimit.get();
}
/**
* 获取当前设定的超时检查间隔
* @return 间隔时间(ms).
*/
public long getTimeoutCheckInterval() {
return timeoutCheckInterval.get();
}
private Thread createTimeoutCheckThread() {
Thread checkThread = new Thread(() -> {
if(executeTimeLimit.get() <= 0) {
return;
}
while (true) {
try {
long interval = this.timeoutCheckInterval.get();
Thread.sleep(interval);
// 检查是否存在超时的任务
workerThreadMap.forEach((thread, time) -> {
long currentTime = time.getAndAdd(interval);
if(currentTime > executeTimeLimit.get()) {
if(!thread.isInterrupted()) {
thread.interrupt();
}
}
});
} catch(InterruptedException ignored) {
break;
}
}
});
checkThread.setName("ThreadPool-" + Integer.toHexString(this.hashCode()) +"-TimeoutCheck");
return checkThread;
}
@Override
protected void beforeExecute(Thread t, Runnable r) {
workerThreadMap.put(t, new AtomicLong());
super.beforeExecute(t, r);
}
@Override
protected void afterExecute(Runnable r, Throwable t) {
workerThreadMap.remove(Thread.currentThread());
super.afterExecute(r, t);
}
@Override
protected void terminated() {
this.timeoutCheckThread.interrupt();
super.terminated();
}
}

View File

@ -0,0 +1,40 @@
<?xml version="1.0" encoding="UTF-8"?>
<configuration status="WARN">
<properties>
<property name="logStorePath">./logs</property>
<property name="charset">UTF-8</property>
<property name="pattern">[%-d{HH:mm:ss.SSS} %5level][%logger.%method():%-3L][%thread]: %msg%n</property>
<property name="logsDir">${sys:cgj.logsPath:-logs}</property>
</properties>
<Appenders>
<Console name="CONSOLE_STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="${pattern}" charset="${charset}"/>
<Filters>
<ThresholdFilter level="WARN" onMatch="DENY" onMismatch="ACCEPT"/>
</Filters>
</Console>
<Console name="CONSOLE_STDERR" target="SYSTEM_ERR">
<PatternLayout pattern="${pattern}" charset="${charset}"/>
<Filters>
<ThresholdFilter level="WARN" onMatch="ACCEPT" onMismatch="DENY"/>
</Filters>
</Console>
<RollingFile name="rollingFile" fileName="${logsDir}/latest.log" filePattern="${logsDir}/running.%-d{yyyy-MM-dd_HH-mm-ss}.log.gz">
<PatternLayout pattern="${pattern}" charset="${charset}"/>
<Policies>
<OnStartupTriggeringPolicy />
</Policies>
</RollingFile>
</Appenders>
<Loggers>
<Logger level="INFO" name="org.apache.http"/>
<Root level="TRACE">
<AppenderRef ref="CONSOLE_STDOUT"/>
<AppenderRef ref="CONSOLE_STDERR"/>
<AppenderRef ref="rollingFile"/>
</Root>
</Loggers>
</configuration>

View File

@ -4,13 +4,14 @@
<property name="logStorePath">./logs</property>
<property name="charset">UTF-8</property>
<property name="pattern">[%-d{HH:mm:ss.SSS} %5level][%logger.%method():%-3L][%thread]: %msg%n</property>
<property name="logsDir">${sys:cgj.logsPath:-logs}</property>
</properties>
<Appenders>
<Console name="CONSOLE_STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="${pattern}" charset="${charset}"/>
<Filters>
<ThresholdFilter level="WARN" onMatch="DENY" onMismatch="ACCEPT"/>
<LevelRangeFilter minLevel="INFO" maxLevel="INFO" />
</Filters>
</Console>
<Console name="CONSOLE_STDERR" target="SYSTEM_ERR">
@ -20,7 +21,7 @@
</Filters>
</Console>
<RollingFile name="rollingFile" fileName="logs/latest.log" filePattern="logs/running.%-d{yyyy-MM-dd_HH-mm-ss}.log.gz">
<RollingFile name="rollingFile" fileName="${logsDir}/latest.log" filePattern="${logsDir}/running.%-d{yyyy-MM-dd_HH-mm-ss}.log.gz">
<PatternLayout pattern="${pattern}" charset="${charset}"/>
<Policies>
<OnStartupTriggeringPolicy />
@ -30,7 +31,7 @@
<Loggers>
<Logger level="INFO" name="org.apache.http"/>
<Root level="DEBUG">
<Root level="TRACE">
<AppenderRef ref="CONSOLE_STDOUT"/>
<AppenderRef ref="CONSOLE_STDERR"/>
<AppenderRef ref="rollingFile"/>

View File

@ -0,0 +1,22 @@
package net.lamgc.cgj.pixiv;
import org.junit.Ignore;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@Ignore
public class ModerateContentDetectorTest {
private final static AdultContentDetector acd = new ModerateContentDetector("d91b6c3fa2bba9ee8f9e68827ba0d937");
private final static Logger log = LoggerFactory.getLogger(ModerateContentDetector.class);
@Test
public void checkTest() throws Exception {
log.info("Detect: {}, isAdult: {}",
acd.detect(80840411, false, 0),
acd.isAdultContent(80840411, false, 0));
}
}

View File

@ -1,8 +1,15 @@
package net.lamgc.cgj.pixiv;
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import org.apache.commons.io.IOUtils;
import org.apache.http.HttpHost;
import org.apache.http.client.CookieStore;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Ignore;
@ -174,19 +181,6 @@ public class PixivDownloadTest {
log.info("正在调用方法...");
try {
pixivDownload.getRankingAsInputStream(null, null, queryDate, 5, 50, PixivDownload.PageQuality.ORIGINAL, (rank, link, rankInfo, inputStream) -> {
/*try {
ZipEntry entry = new ZipEntry("Rank" + rank + "-" + link.substring(link.lastIndexOf("/") + 1));
entry.setComment(rankInfo.toString());
log.info("正在写入: " + entry.getName());
zos.putNextEntry(entry);
IOUtils.copy(inputStream, zos);
zos.flush();
log.info("已成功写入 {}", entry.getName());
inputStream.close();
} catch (IOException e) {
log.error("写入文件项时发生异常", e);
}*/
log.info("空操作");
});
} finally {
@ -202,4 +196,17 @@ public class PixivDownloadTest {
log.info(new PixivDownload(cookieStore, proxy).getIllustPreLoadDataById(64076261).toString());
}
@Test
public void illustInfoTest() throws IOException {
CloseableHttpClient httpClient = HttpClientBuilder.create().setProxy(new HttpHost("127.0.0.1", 1001)).build();
HttpGet request = new HttpGet(PixivURL.getPixivIllustInfoAPI(80880547));
CloseableHttpResponse response = httpClient.execute(request);
String body = EntityUtils.toString(response.getEntity());
JsonObject illustInfoResult = new Gson().fromJson(body, JsonObject.class);
log.info("IllustInfoJsonResult: {}", illustInfoResult);
PixivDownload.PixivIllustType illustType = PixivDownload.PixivIllustType.getIllustTypeByPreLoadData(80880547, illustInfoResult);
log.info("IllustType: {}", illustType);
Assert.assertEquals(PixivDownload.PixivIllustType.UGOIRA, illustType);
}
}

View File

@ -0,0 +1,30 @@
package net.lamgc.cgj.util;
import org.junit.Assert;
import org.junit.Test;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.TimeUnit;
public class TimeLimitThreadPoolExecutorTest {
@Test
public void timeoutTest() throws InterruptedException {
TimeLimitThreadPoolExecutor executor = new TimeLimitThreadPoolExecutor(1000, 1, 1, 30, TimeUnit.SECONDS, new ArrayBlockingQueue<>(50));
System.out.println(executor.isTerminated());
System.out.println(executor.isShutdown());
executor.setTimeoutCheckInterval(150);
System.out.println("当前设定: ETL: " + executor.getExecuteTimeLimit() + "ms, TCI: " + executor.getTimeoutCheckInterval() + "ms");
executor.execute(() -> {
try {
Thread.sleep(5 * 1000);
} catch (InterruptedException e) {
System.out.println("线程 " + Thread.currentThread().getName() + " 被中断");
}
});
executor.shutdown();
Assert.assertTrue(executor.awaitTermination(5 * 1000, TimeUnit.MILLISECONDS));
}
}