Compare commits

...

49 Commits

Author SHA1 Message Date
a36eb713d7 [Version] 更新版本(2.5.0-20200505.1-SNAPSHOT -> 2.5.0-20200506.1-SNAPSHOT); 2020-05-06 11:09:18 +08:00
f452c1c128 Merge branch 'add-MultiProperties' 2020-05-06 11:08:16 +08:00
cb42aadb15 [Fix] SettingProperties 修复在初次设置群组配置时未创建该群组所属Properties导致NPE的问题;
[Update] BotAdminCommandProcess 更新日志内容及命令返回信息;
[Change] BotCommandProcess 调整配置项key格式;
2020-05-06 10:55:32 +08:00
b969bb29b2 Merge branch 'master' into add-MultiProperties 2020-05-06 10:18:00 +08:00
0d74007a98 [Change] search.txt 整理接口相关文档的存储目录; 2020-05-06 09:42:34 +08:00
edade24883 [Change] Issue #2 调整Logger Name以统一Logger Name格式; 2020-05-06 09:34:28 +08:00
34f57404ca [Update] 重构Setting Properties, 使其支持分群配置; 2020-05-06 00:57:48 +08:00
99b6e14ff7 [Change] RandomIntervalSendTimer 调整日志输出内容; 2020-05-05 17:09:45 +08:00
c8fe2c3fdd [Version] 更新版本(2.5.0-20200504.2-SNAPSHOT -> 2.5.0-20200505.1-SNAPSHOT);
[Update] net.lamgc:java-utils 更新依赖版本(1.1.0 -> 1.2.0_20200505.1-SNAPSHOT);
2020-05-05 01:29:11 +08:00
49a33d4078 [Add] MessageEventExecutionDebugger 添加对消息处理的调试器Enum;
[Add] BotEventHandler 添加对 MessageEventExecutionDebugger 的支持;
[Add] VirtualLoadMessageEvent 增加 toVirtualLoadMessageEvent(MessageEvent) 方法;
2020-05-05 01:27:10 +08:00
cd1d2316ee [Add] BotEventHandler 添加executeMessageEvent(MessageEvent)方法;
[Change] BotEventHandler 将BotEventHandler.executor设为private;
[Change] MiraiMain, CQPluginMain, RankingUpdateTimer 适配BotEventHandler的调整
2020-05-04 23:06:07 +08:00
cf08353ed9 [Change] BotEventHandler 调整事件处理线程池的线程数(min: 4, Max: 32); 2020-05-04 22:52:47 +08:00
bef6a684b9 [Change] BotEventHandler 调整事件处理线程池的线程数; 2020-05-04 22:46:06 +08:00
04960889b4 [Version] 更新版本(2.5.0-20200504.01-SNAPSHOT -> 2.5.0-20200504.2-SNAPSHOT);
[Update] net.lamgc:java-utils 更新依赖版本(1.1.0_5-SNAPSHOT -> 1.1.0);
[Update] net.lamgc:PixivLoginProxyServer 更新依赖版本(1.1.0 -> 1.1.1);
2020-05-04 19:44:11 +08:00
0dc8fc78b4 [Version] 更新版本(2.4.0 -> 2.5.0-20200504.01-SNAPSHOT); 2020-05-04 17:07:47 +08:00
40c5284be2 [Change] Dockerfile.sample 调整构建指令顺序以充分利用构建缓存;
[Fix] Main 修复SystemProperties设置null抛出NPE的问题;
2020-05-04 17:07:39 +08:00
46cb47c7d1 [Add] LICENSE 添加开源许可证(LGPLv3); 2020-05-04 16:55:16 +08:00
fe213deecb [Change] Main 调整参数的接收形式;
[Fix] Dockerfile.sample 修复Java应用无法接收stop命令发送信号的问题;
[Change] RankingUpdateTimer 调整更新投递形式, 将同步更新调整为异步更新;
[Change] BotEventHandler 实装TimeLimitThreadPoolExecutor;
2020-05-04 02:07:46 +08:00
f279d99fda [Add] TimeLimitThreadPoolExecutor 增加一个带有执行时间限制的线程池及对应单元测试类; 2020-05-04 02:04:59 +08:00
a09aef5be2 [Delete] 删除已完成或已经没有存在意义的TODO; 2020-05-02 10:40:11 +08:00
766411fa09 [Fix] Dockerfile.sample 修复应用无法接收容器退出信号的问题(现在应用将可以接受信号主动退出, 即使ExitCode不为0); 2020-05-02 00:56:05 +08:00
84b544cea9 [Change] Main 调整日志输出级别, 补充日志输出内容;
[Change] Dockerfile.sample 调整镜像内应用运行路径;
2020-05-01 22:28:15 +08:00
2f492f5b03 [Add] log4j2.xml, log4j2-test.xml 添加允许指定日志存储目录的功能(sys:cgj.logsPath); 2020-05-01 13:19:08 +08:00
15af939c3f [Fix] PixivDownload 修复因排行榜总量的不确定导致排行榜获取异常的问题;
[Change] BotEventHandler 调整事件处理完成后的日志输出形式;
2020-04-30 19:08:34 +08:00
a28cb142b4 [Change] 调整数据存储的路径设置及存储目录参数名; 2020-04-30 17:51:57 +08:00
e8fda3214b [Version] 更新版本(2.3.0 -> 2.4.0); 2020-04-30 02:24:21 +08:00
fba6d3532c [Delete] 删除不再使用的类和方法;
[Update] 优化Javadoc并设置编译注解, 优化代码;
[Change] 调整部分日志的输出级别;
[Change] RedisPoolCacheStore clear方法将根据返回信息确定是否清空成功;
[Change] Dockerfile.sample 调整镜像构建步骤;
2020-04-30 02:23:31 +08:00
0075446412 [Fix] MiraiMessageSender 修复因BotCode-Image中flashImage参数不存在导致NPE的问题; 2020-04-30 01:54:27 +08:00
2388cd419e [Add] Dockerfile.sample 增加一个用于构建CGJ-BotMode镜像的Dockerfile模板; 2020-04-30 00:45:41 +08:00
7f83b16118 [Add] Main 添加日志输出内容; 2020-04-29 23:44:38 +08:00
10cad32efa [Version] 更新版本(2.2.1 -> 2.3.0); 2020-04-28 23:16:10 +08:00
22a113ef38 [Change] 调整CGJ包结构, 将机器人平台相关类单独存放在framework包内; 2020-04-28 23:15:55 +08:00
6824b12e8f [Add] MiraiMessageSender 增加对闪照的支持; 2020-04-28 10:57:17 +08:00
1c556c5b94 [Update] MiraiMessageSender 对已上传的图片所属ImageId设置10天有效期; 2020-04-28 00:32:40 +08:00
dbc9f4c90b [Fix] BotCommandProcess 修复了通过缓存获取排行榜时排行榜从第2名开始读取的bug;
[Update] MiraiMain, MiraiMessageSender, MiraiMessageEvent 更新Mirai框架(0.32.0 -> 0.39.4)并适配框架新版本的变动;
2020-04-28 00:19:24 +08:00
d3c1975722 [Fix] PixivURL 修复由于智障修复导致的bug; 2020-04-27 23:26:22 +08:00
c09b750fe6 [Version] 更新版本(2.2.0 -> 2.2.1); 2020-04-27 14:13:51 +08:00
ca479ef1af [Change] log4j2.xml 调整SYSTEM_OUT控制台输出配置; 2020-04-27 14:06:33 +08:00
c2c49d2355 [Add] PixivDownload 增加对排行榜获取失败抛出异常的信息;
[Fix] PixivURL.RankingContentType 修复isSupportedMode判断错误的问题;
[Fix] RankingUpdateTimer 调整每日更新时间(12:10 -> 12:30)以尝试修复排行榜更新时机过早的问题;
2020-04-27 12:32:25 +08:00
22e74e8cd5 [Add] PixivIllustType 增加了对IllustInfo的解析; 2020-04-26 15:53:39 +08:00
d549c5674d [Update] BotCommandProcess 补充Javadoc;
[Change] PixivUgoiraBuilder 调整日志输出级别;
2020-04-26 15:45:23 +08:00
c6952de84c [Add] log4j2-test.xml 增加测试运行时的日志配置文件;
[Add] pom.xml 设置package goal的文件排除项, 将测试运行所使用的日志配置文件(log4j2-test.xml)排除;
2020-04-25 17:07:13 +08:00
410d6c0828 [Add] BotAdminCommandProcess, PixivDownload 增加日志输出内容;
[Change] RandomIntervalSendTimer 对AutoSender增加异常捕获以减少对其他Timer执行的影响;
2020-04-25 17:02:01 +08:00
aaa1bc932b [Add] RandomRankingArtworksSender 增加参数;
[Add] BotAdminCommandProcess 增加更多可自定义的参数项, 增加json配置项常量;
2020-04-25 02:29:38 +08:00
a1e54e70d9 [Change] PixivURL 调整RankingContentType.ALL名称(ALL -> TYPE_ALL);
[Add] PixivDownload 增加调试日志信息;
[Fix] RandomIntervalSendTimer 修复cancel方法无法停止定时器的问题;
2020-04-24 23:54:26 +08:00
16522155e1 [Add] 通过增加参数信息来为命令方法提供来源群和来源QQ; 2020-04-24 23:52:05 +08:00
5f796f7da0 [Update] BotCommandProcess 更新帮助信息, 补充Javadoc, 调整命令名;
[Change] RandomIntervalSendTimer 将下一执行时间从分钟调整为具体时间;
[Change] MessageEvent toString方法调整为输出具体实现类名而不是"MessageEvent";
2020-04-24 10:45:03 +08:00
5a52dd9208 [Add] 初步添加成人内容检测器, 尚未使用; 2020-04-24 01:42:47 +08:00
b53aafa81b [Fix] RandomRankingArtworksSender 修复排行榜选取错误的问题;
[Add] RandomRankingArtworksSender 增加对rankingStart, rankingStop的范围选取检查;
[Update] RandomRankingArtworksSender 更新Javadoc;
2020-04-24 01:41:53 +08:00
46 changed files with 1397 additions and 1000 deletions

10
Dockerfile.sample Normal file
View File

@ -0,0 +1,10 @@
FROM openjdk:8u252-jre
ENV jarFileName=ContentGrabbingJi-exec.jar
ENV CGJ_REDIS_URI="127.0.0.1:6379"
ENV CGJ_PROXY=""
RUN mkdir /data/
COPY ${jarFileName} /CGJ.jar
CMD ["java", "-Dcgj.logsPath=/data/logs", "-jar", "/CGJ.jar", "botMode", "-botDataDir=/data"]

165
LICENSE Normal file
View File

@ -0,0 +1,165 @@
GNU LESSER GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
This version of the GNU Lesser General Public License incorporates
the terms and conditions of version 3 of the GNU General Public
License, supplemented by the additional permissions listed below.
0. Additional Definitions.
As used herein, "this License" refers to version 3 of the GNU Lesser
General Public License, and the "GNU GPL" refers to version 3 of the GNU
General Public License.
"The Library" refers to a covered work governed by this License,
other than an Application or a Combined Work as defined below.
An "Application" is any work that makes use of an interface provided
by the Library, but which is not otherwise based on the Library.
Defining a subclass of a class defined by the Library is deemed a mode
of using an interface provided by the Library.
A "Combined Work" is a work produced by combining or linking an
Application with the Library. The particular version of the Library
with which the Combined Work was made is also called the "Linked
Version".
The "Minimal Corresponding Source" for a Combined Work means the
Corresponding Source for the Combined Work, excluding any source code
for portions of the Combined Work that, considered in isolation, are
based on the Application, and not on the Linked Version.
The "Corresponding Application Code" for a Combined Work means the
object code and/or source code for the Application, including any data
and utility programs needed for reproducing the Combined Work from the
Application, but excluding the System Libraries of the Combined Work.
1. Exception to Section 3 of the GNU GPL.
You may convey a covered work under sections 3 and 4 of this License
without being bound by section 3 of the GNU GPL.
2. Conveying Modified Versions.
If you modify a copy of the Library, and, in your modifications, a
facility refers to a function or data to be supplied by an Application
that uses the facility (other than as an argument passed when the
facility is invoked), then you may convey a copy of the modified
version:
a) under this License, provided that you make a good faith effort to
ensure that, in the event an Application does not supply the
function or data, the facility still operates, and performs
whatever part of its purpose remains meaningful, or
b) under the GNU GPL, with none of the additional permissions of
this License applicable to that copy.
3. Object Code Incorporating Material from Library Header Files.
The object code form of an Application may incorporate material from
a header file that is part of the Library. You may convey such object
code under terms of your choice, provided that, if the incorporated
material is not limited to numerical parameters, data structure
layouts and accessors, or small macros, inline functions and templates
(ten or fewer lines in length), you do both of the following:
a) Give prominent notice with each copy of the object code that the
Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the object code with a copy of the GNU GPL and this license
document.
4. Combined Works.
You may convey a Combined Work under terms of your choice that,
taken together, effectively do not restrict modification of the
portions of the Library contained in the Combined Work and reverse
engineering for debugging such modifications, if you also do each of
the following:
a) Give prominent notice with each copy of the Combined Work that
the Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the Combined Work with a copy of the GNU GPL and this license
document.
c) For a Combined Work that displays copyright notices during
execution, include the copyright notice for the Library among
these notices, as well as a reference directing the user to the
copies of the GNU GPL and this license document.
d) Do one of the following:
0) Convey the Minimal Corresponding Source under the terms of this
License, and the Corresponding Application Code in a form
suitable for, and under terms that permit, the user to
recombine or relink the Application with a modified version of
the Linked Version to produce a modified Combined Work, in the
manner specified by section 6 of the GNU GPL for conveying
Corresponding Source.
1) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (a) uses at run time
a copy of the Library already present on the user's computer
system, and (b) will operate properly with a modified version
of the Library that is interface-compatible with the Linked
Version.
e) Provide Installation Information, but only if you would otherwise
be required to provide such information under section 6 of the
GNU GPL, and only to the extent that such information is
necessary to install and execute a modified version of the
Combined Work produced by recombining or relinking the
Application with a modified version of the Linked Version. (If
you use option 4d0, the Installation Information must accompany
the Minimal Corresponding Source and Corresponding Application
Code. If you use option 4d1, you must provide the Installation
Information in the manner specified by section 6 of the GNU GPL
for conveying Corresponding Source.)
5. Combined Libraries.
You may place library facilities that are a work based on the
Library side by side in a single library together with other library
facilities that are not Applications and are not covered by this
License, and convey such a combined library under terms of your
choice, if you do both of the following:
a) Accompany the combined library with a copy of the same work based
on the Library, uncombined with any other library facilities,
conveyed under the terms of this License.
b) Give prominent notice with the combined library that part of it
is a work based on the Library, and explaining where to find the
accompanying uncombined form of the same work.
6. Revised Versions of the GNU Lesser General Public License.
The Free Software Foundation may publish revised and/or new versions
of the GNU Lesser General Public License from time to time. Such new
versions will be similar in spirit to the present version, but may
differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the
Library as you received it specifies that a certain numbered version
of the GNU Lesser General Public License "or any later version"
applies to it, you have the option of following the terms and
conditions either of that published version or of any later version
published by the Free Software Foundation. If the Library as you
received it does not specify a version number of the GNU Lesser
General Public License, you may choose any version of the GNU Lesser
General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide
whether future versions of the GNU Lesser General Public License shall
apply, that proxy's public statement of acceptance of any version is
permanent authorization for you to choose that version for the
Library.

22
pom.xml
View File

@ -6,7 +6,7 @@
<groupId>net.lamgc</groupId>
<artifactId>ContentGrabbingJi</artifactId>
<version>2.2.0</version>
<version>2.5.0-20200506.1-SNAPSHOT</version>
<repositories>
<repository>
@ -19,7 +19,7 @@
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<maven.compiler.encoding>UTF-8</maven.compiler.encoding>
<mirai.CoreVersion>0.32.0</mirai.CoreVersion>
<mirai.CoreVersion>0.39.4</mirai.CoreVersion>
<mirai.JaptVersion>1.1.1</mirai.JaptVersion>
<kotlin.version>1.3.71</kotlin.version>
<ktor.version>1.3.2</ktor.version>
@ -36,6 +36,16 @@
<target>8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.0.2</version>
<configuration>
<excludes>
<exclude>log4j2-test.xml</exclude>
</excludes>
</configuration>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
@ -76,7 +86,7 @@
<dependency>
<groupId>net.lamgc</groupId>
<artifactId>java-utils</artifactId>
<version>1.1.0_5-SNAPSHOT</version>
<version>1.2.0_20200505.1-SNAPSHOT</version>
</dependency>
<dependency>
@ -141,12 +151,12 @@
</dependency>
<dependency>
<groupId>net.mamoe</groupId>
<artifactId>mirai-core-jvm</artifactId>
<artifactId>mirai-core</artifactId>
<version>${mirai.CoreVersion}</version>
</dependency>
<dependency>
<groupId>net.mamoe</groupId>
<artifactId>mirai-core-qqandroid-jvm</artifactId>
<artifactId>mirai-core-qqandroid</artifactId>
<version>${mirai.CoreVersion}</version>
</dependency>
<dependency>
@ -162,7 +172,7 @@
<dependency>
<groupId>net.lamgc</groupId>
<artifactId>PixivLoginProxyServer</artifactId>
<version>1.1.0</version>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>com.squareup</groupId>

View File

@ -7,10 +7,11 @@ import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.CQConfig;
import net.lamgc.cgj.bot.MiraiMain;
import net.lamgc.cgj.pixiv.*;
import net.lamgc.cgj.proxy.PixivAccessProxyServer;
import net.lamgc.cgj.bot.framework.coolq.CQConfig;
import net.lamgc.cgj.bot.framework.mirai.MiraiMain;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivSearchBuilder;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.plps.PixivLoginProxyServer;
import net.lamgc.utils.base.ArgumentsProperties;
import net.lamgc.utils.base.runner.Argument;
@ -20,9 +21,7 @@ import org.apache.commons.io.IOUtils;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.client.CookieStore;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.message.BasicHeader;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@ -43,7 +42,7 @@ import java.util.zip.ZipOutputStream;
@SpringBootApplication
public class Main {
private final static Logger log = LoggerFactory.getLogger("Main");
private final static Logger log = LoggerFactory.getLogger(Main.class.getName());
private final static File storeDir = new File("store/");
@ -52,9 +51,19 @@ public class Main {
public static HttpHost proxy;
public static void main(String[] args) throws IOException, ClassNotFoundException {
log.trace("ContentGrabbingJi 正在启动...");
log.debug("Args: {}, LogsPath: {}", Arrays.toString(args), System.getProperty("cgj.logsPath"));
log.debug("运行目录: {}", System.getProperty("user.dir"));
ArgumentsProperties argsProp = new ArgumentsProperties(args);
if(argsProp.containsKey("proxy")) {
URL proxyUrl = new URL(argsProp.getValue("proxy"));
if(!getSettingToSysProp(argsProp, "proxy", null)) {
getEnvSettingToSysProp("CGJ_PROXY", "proxy", null);
}
String proxyAddress = System.getProperty("cgj.proxy");
if(!Strings.isNullOrEmpty(proxyAddress)) {
URL proxyUrl = new URL(proxyAddress);
proxy = new HttpHost(proxyUrl.getHost(), proxyUrl.getPort());
log.info("已启用Http协议代理{}", proxy.toHostString());
} else {
@ -65,28 +74,20 @@ public class Main {
log.error("创建文件夹失败!");
}
// TODO: 需要修改参数名了, 大概改成类似于 workerDir这样的吧
if(argsProp.containsKey("cqRootDir")) {
log.info("cqRootDir: {}", argsProp.getValue("cqRootDir"));
System.setProperty("cgj.cqRootDir", argsProp.getValue("cqRootDir"));
} else {
log.warn("未设置cqRootDir, 当前运行目录将作为酷Q机器人所在目录.");
System.setProperty("cgj.cqRootDir", "./");
if(!getSettingToSysProp(argsProp, "botDataDir", "./") &&
!getEnvSettingToSysProp("CGJ_BOT_DATA_DIR", "botDataDir", "./")) {
log.warn("未设置botDataDir, 当前运行目录将作为酷Q机器人所在目录.");
}
if(!getSettingToSysProp(argsProp, "redisAddress", "127.0.0.1") &&
!getEnvSettingToSysProp("CGJ_REDIS_URI", "redisAddress", "127.0.0.1")) {
log.warn("未设置RedisAddress, 将使用默认值连接Redis服务器(127.0.0.1:6379)");
}
if(argsProp.containsKey("redisAddr")) {
log.info("redisAddress: {}", argsProp.getValue("redisAddr"));
System.setProperty("cgj.redisAddress", argsProp.getValue("redisAddr"));
} else {
log.info("未设置RedisAddress, 将使用默认值连接Redis服务器(127.0.0.1:6379)");
System.setProperty("cgj.redisAddress", "127.0.0.1");
}
File cookieStoreFile = new File("cookies.store");
File cookieStoreFile = new File(System.getProperty("cgj.botDataDir"), "cookies.store");
if(!cookieStoreFile.exists()) {
log.error("未找到cookies.store文件, 是否启动PixivLoginProxyServer? (yes/no)");
log.warn("未找到cookies.store文件, 是否启动PixivLoginProxyServer? (yes/no)");
Scanner scanner = new Scanner(System.in);
if(scanner.nextLine().equalsIgnoreCase("yes")) {
if(scanner.nextLine().trim().equalsIgnoreCase("yes")) {
startPixivLoginProxyServer();
} else {
System.exit(1);
@ -103,6 +104,37 @@ public class Main {
ArgumentsRunner.run(Main.class, args);
}
/**
* 从ArgumentsProperties获取设置项到System Properties
* @param prop ArgumentsProperties对象
* @param key 设置项key
* @param defaultValue 默认值
* @return 如果成功从ArgumentsProperties获得设置项, 返回true, 如未找到(使用了defaultValue或null), 返回false;
*/
private static boolean getSettingToSysProp(ArgumentsProperties prop, String key, String defaultValue) {
if(prop.containsKey(key)) {
log.info("{}: {}", key, prop.getValue(key));
System.setProperty("cgj." + key, prop.getValue(key));
return true;
} else {
if(defaultValue != null) {
System.setProperty("cgj." + key, defaultValue);
}
return false;
}
}
private static boolean getEnvSettingToSysProp(String envKey, String sysPropKey, String defaultValue) {
String env = System.getenv(envKey);
if(env != null) {
System.setProperty("cgj." + sysPropKey, env);
return true;
} else if(defaultValue != null) {
System.setProperty("cgj." + sysPropKey, defaultValue);
}
return false;
}
@Command
public static void botMode(@Argument(name = "args", force = false) String argsStr) {
new MiraiMain().init();
@ -110,10 +142,10 @@ public class Main {
@Command
public static void pluginMode(@Argument(name = "args", force = false) String argsStr) {
if(!System.getProperty("cgj.cqRootDir").endsWith("\\") && !System.getProperty("cgj.cqRootDir").endsWith("/")) {
System.setProperty("cgj.cqRootDir", System.getProperty("cgj.cqRootDir") + "/");
if(!System.getProperty("cgj.botDataDir").endsWith("\\") && !System.getProperty("cgj.botDataDir").endsWith("/")) {
System.setProperty("cgj.botDataDir", System.getProperty("cgj.botDataDir") + "/");
}
log.info("酷Q机器人根目录: {}", System.getProperty("cgj.cqRootDir"));
log.info("酷Q机器人根目录: {}", System.getProperty("cgj.botDataDir"));
CQConfig.init();
Pattern pattern = Pattern.compile("/\\s*(\".+?\"|[^:\\s])+((\\s*:\\s*(\".+?\"|[^\\s])+)|)|(\".+?\"|[^\"\\s])+");
Matcher matcher = pattern.matcher(Strings.nullToEmpty(argsStr));
@ -353,89 +385,19 @@ public class Main {
builder,
PixivURL.getPixivRefererLink(illustId)
);
/*log.info("正在下载...");
List<String> list = PixivDownload.getIllustAllPageDownload(
HttpClientBuilder.create()
.setProxy(proxy)
.build(),
illustId, PixivDownload.PageQuality.ORIGINAL);*/
}
}
/*
if(searchBuilder.getSearchArea().equals(PixivSearchBuilder.SearchArea.TOP)) {
} else {
JsonArray illustsArray = resultBody
.getAsJsonObject(searchBuilder.getSearchArea().jsonKey).getAsJsonArray("data");
log.info("已找到与 {} 相关插图信息:", content);
int count = 1;
for (JsonElement jsonElement : illustsArray) {
JsonObject illustObj = jsonElement.getAsJsonObject();
//TODO: 防止数据内混入无效内容, 需要检查对象是否有illustId
if(!illustObj.has("illustId")) {
continue;
}
int illustId = illustObj.get("illustId").getAsInt();
StringBuilder builder = new StringBuilder("[");
illustObj.get("tags").getAsJsonArray().forEach(el -> builder.append(el.getAsString()).append(", "));
builder.replace(builder.length() - 2, builder.length(), "]");
log.info("({} / {})\n\t作品id: {}, \n\t作者名(作者id): {} ({}), \n\t作品标题: {}, \n\t作品Tags: {}, \n\t作品链接: {}",
count++,
illustsArray.size(),
illustId,
illustObj.get("userName").getAsString(),
illustObj.get("userId").getAsInt(),
illustObj.get("illustTitle").getAsString(),
builder,
PixivURL.getPixivRefererLink(illustId)
);
}
}
*/
}
@Command(defaultCommand = true)
public static void testRun() {
/*loadCookieStoreFromFile();
if(cookieStore == null){
startPixivLoginProxyServer();
}*/
//accessPixivToFile();
//startPixivAccessProxyServer();
//saveCookieStoreToFile();
log.info("这里啥都没有哟w");
}
private static void loadCookieStoreFromFile() throws IOException {
log.info("正在加载CookieStore...");
File storeFile = new File("./cookies.store");
if(!storeFile.exists()){
log.info("未找到CookieStore, 跳过加载.");
return;
}
ObjectInputStream stream = new ObjectInputStream(new FileInputStream(storeFile));
Object result;
try {
result = stream.readObject();
} catch (ClassNotFoundException e) {
log.error("加载出错", e);
return;
}
cookieStore = (CookieStore) result;
cookieStore.getCookies().forEach(cookie -> log.debug(cookie.getName() + ": " + cookie.getValue() + ", isExpired: " + cookie.isExpired(new Date())));
log.info("CookieStore加载完成.");
}
private static void saveCookieStoreToFile() throws IOException {
log.info("正在保存CookieStore...");
File outputFile = new File("./cookies.store");
File outputFile = new File(System.getProperty("cgj.botDataDir"), "cookies.store");
if(!outputFile.exists() && !outputFile.delete() && !outputFile.createNewFile()){
log.error("保存CookieStore失败.");
return;
@ -482,75 +444,4 @@ public class Main {
}
}
private static void accessPixivToFile() throws IOException {
File cookieStoreFile = new File("./cookie.txt");
if (!cookieStoreFile.exists() && !cookieStoreFile.createNewFile()) {
log.info("Cookie文件存储失败");
}
/*log.info("正在写出Cookie, Cookie count: " + cookieStore.getCookies().size());
FileWriter cookieWriter = new FileWriter(cookieStoreFile);
cookieStore.getCookies().forEach(cookie -> {
try {
StringBuilder sb = new StringBuilder().append(cookie.getName()).append(" = ").append(cookie.getValue());
log.info("正在导出Cookie: " + sb.toString());
cookieWriter.append(sb.toString()).append("\n").flush();
} catch (IOException e) {
e.printStackTrace();
}
});
log.info("Cookie写出完成");*/
log.info("尝试通过捕获的Cookie访问Pixiv...");
HttpClient httpClient = new PixivSession(new HttpHost("127.0.0.1", 1080), cookieStore).getHttpClient();
HttpGet request = new HttpGet("https://www.pixiv.net");
request.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:73.0) Gecko/20100101 Firefox/73.0");
request.addHeader(new BasicHeader("accept-encoding", "gzip, deflate, br"));
request.addHeader(new BasicHeader("accept-language", "zh-CN,zh;q=0.9"));
StringBuilder cookieBuilder = new StringBuilder();
cookieStore.getCookies().forEach(cookie -> {
if(cookie.isExpired(new Date())){
return;
}
cookieBuilder.append(cookie.getName()).append("=").append(cookie.getValue()).append("; ");
});
request.addHeader("cookie", cookieBuilder.toString());
HttpResponse response = httpClient.execute(request);
log.info("正在写入文件...");
File outFile = new File("./pixiv.html");
if (outFile.createNewFile() && !outFile.exists()) {
log.info("文件创建失败!");
}else {
new FileWriter(outFile).append(EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8)).flush();
}
Pixiv pixiv = new Pixiv(httpClient);
pixiv.getRecommend().forEach(illustMap -> {
StringBuilder builder = new StringBuilder();
illustMap.forEach((key, value) -> builder.append(key).append(": ").append(value).append("\n"));
try {
builder.append("download Link: ").append(Arrays.toString(pixiv.getAllDownloadLink(Integer.parseInt(illustMap.get(Pixiv.ATTR_ILLUST_ID)))));
} catch (IOException e) {
log.error("获取下载链接时出错!", e);
}
log.info(builder.append("\n").toString());
});
}
private static void startPixivAccessProxyServer(){
log.info("正在启动访问代理服务器, 将浏览器相关缓存清空后, 使用浏览器进行访问以尝试Cookie正确性.");
final PixivAccessProxyServer accessProxyServer = new PixivAccessProxyServer(cookieStore, new ProxyConfig(ProxyType.SOCKS5, "127.0.0.1", 1080));
Thread accessProxyServerThread = new Thread(() -> {
log.info("正在启动PAPS...");
accessProxyServer.start(1007);
log.info("PAPS已关闭.");
});
accessProxyServerThread.setName("AccessProxyServerThread");
accessProxyServerThread.start();
new Scanner(System.in).nextLine();
log.info("关闭PAPS服务器...");
accessProxyServer.close();
}
}

View File

@ -8,6 +8,7 @@ import com.google.gson.reflect.TypeToken;
import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.utils.base.runner.Argument;
import net.lamgc.utils.base.runner.Command;
import org.slf4j.Logger;
@ -19,11 +20,9 @@ import java.util.*;
public class BotAdminCommandProcess {
private final static Logger log = LoggerFactory.getLogger(BotAdminCommandProcess.class.getSimpleName());
private final static Logger log = LoggerFactory.getLogger(BotAdminCommandProcess.class.getName());
private final static File globalPropFile = new File("global.properties");
private final static File pushListFile = new File("pushList.json");
private final static File pushListFile = new File(System.getProperty("cgj.botDataDir"), "pushList.json");
private final static Hashtable<Long, JsonObject> pushInfoMap = new Hashtable<>();
@ -38,59 +37,47 @@ public class BotAdminCommandProcess {
}
@Command
public static String setGlobalProperty(@Argument(name = "key") String key, @Argument(name = "value") String value, @Argument(name = "save", force = false) boolean saveNow) {
String lastValue = BotCommandProcess.globalProp.getProperty(key);
BotCommandProcess.globalProp.setProperty(key, Strings.nullToEmpty(value));
if(saveNow) {
saveGlobalProperties();
public static String setProperty(
@Argument(name = "group", force = false) long groupId,
@Argument(name = "key") String key,
@Argument(name = "value") String value
) {
if(Strings.isNullOrEmpty(key)) {
return "未选择配置项key.";
}
return "全局配置项 " + key + " 现已设置为: " + value + " (设置前的值: " + lastValue + ")";
String lastValue = SettingProperties.setProperty(groupId, key, value.equals("null") ? null : value);
return (groupId <= 0 ? "已更改全局配置 " : "已更改群组 " + groupId + " 配置 ") +
key + " 的值: '" + value + "' (原配置值: '" + lastValue + "')";
}
@Command
public static String getGlobalProperty(@Argument(name = "key") String key) {
return "全局配置项 " + key + " 当前值: " + BotCommandProcess.globalProp.getProperty(key, "(Empty)");
public static String getProperty(
@Argument(name = "group", force = false) long groupId,
@Argument(name = "key") String key
) {
if(Strings.isNullOrEmpty(key)) {
return "未选择配置项key.";
}
return (groupId <= 0 ? "全局配置 " : "群组 " + groupId + " 配置 ") +
key + " 设定值: '" + SettingProperties.getProperty(groupId, key, "(empty)") + "'";
}
@Command
public static String saveGlobalProperties() {
log.info("正在保存全局配置文件...");
try {
if(!globalPropFile.exists()) {
if(!globalPropFile.createNewFile()) {
log.error("全局配置项文件保存失败!({})", "文件创建失败");
return "全局配置项文件保存失败!";
}
}
BotCommandProcess.globalProp.store(new FileOutputStream(globalPropFile), "");
log.info("全局配置文件保存成功!");
return "保存全局配置文件 - 操作已完成.";
} catch (IOException e) {
log.error("全局配置项文件保存失败!", e);
return "全局配置项文件保存失败!";
}
public static String saveProperties() {
log.info("正在保存配置文件...");
SettingProperties.saveProperties();
log.info("配置文件保存操作已完成.");
return "保存配置 - 操作已完成.";
}
@Command
public static String loadGlobalProperties(@Argument(name = "reload", force = false) boolean reload) {
Properties cache = new Properties();
if(!globalPropFile.exists()) {
return "未找到全局配置文件, 无法重载";
}
try(Reader reader = new BufferedReader(new FileReader(globalPropFile))) {
cache.load(reader);
} catch (IOException e) {
log.error("重载全局配置文件时发生异常", e);
return "加载全局配置文件时发生错误!";
}
public static String loadProperties(@Argument(name = "reload", force = false) boolean reload) {
if(reload) {
BotCommandProcess.globalProp.clear();
SettingProperties.clearProperties();
}
BotCommandProcess.globalProp.putAll(cache);
return "全局配置文件重载完成.";
SettingProperties.loadProperties();
return "操作已完成.";
}
@Command
@ -104,29 +91,46 @@ public class BotAdminCommandProcess {
return "操作已完成.";
}
private final static String RANKING_SETTING_TIME_MIN = "time.min";
private final static String RANKING_SETTING_TIME_FLOAT = "time.float";
private final static String RANKING_SETTING_RANKING_START = "ranking.start";
private final static String RANKING_SETTING_RANKING_END = "ranking.end";
private final static String RANKING_SETTING_RANKING_MODE = "ranking.mode";
private final static String RANKING_SETTING_RANKING_CONTENT_TYPE = "ranking.contentType";
private final static String RANKING_SETTING_PAGE_QUALITY = "page.quality";
@Command
public static String addPushGroup(@Argument(name = "group") long groupId,
@Argument(name = "minTime", force = false, defaultValue = "21600000") long minTime,
@Argument(name = "floatTime", force = false, defaultValue = "10800000") int floatTime,
@Argument(name = "rankingStart", force = false, defaultValue = "1") int rankingStart,
@Argument(name = "rankingStop", force = false, defaultValue = "150") int rankingStop,
@Argument(name = "original", force = false, defaultValue = "false") boolean original
public static String addPushGroup(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "group", force = false, defaultValue = "0") long groupId,
@Argument(name = "minTime", force = false, defaultValue = "21600000") long minTime,
@Argument(name = "floatTime", force = false, defaultValue = "10800000") int floatTime,
@Argument(name = "rankingStart", force = false, defaultValue = "1") int rankingStart,
@Argument(name = "rankingStop", force = false, defaultValue = "150") int rankingStop,
@Argument(name = "mode", force = false, defaultValue = "DAILY") String rankingMode,
@Argument(name = "type", force = false, defaultValue = "ILLUST") String rankingContentType,
@Argument(name = "original", force = false, defaultValue = "false") boolean original
) {
long group = groupId <= 0 ? fromGroup : groupId;
JsonObject setting = new JsonObject();
setting.addProperty("time.min", minTime);
setting.addProperty("time.float", floatTime);
setting.addProperty("ranking.start", rankingStart);
setting.addProperty("ranking.end", rankingStop);
setting.addProperty("pageQuality.original", original);
if(pushInfoMap.containsKey(groupId)) {
log.info("群 {} 已存在Timer, 删除Timer...", groupId);
removePushGroup(groupId);
setting.addProperty(RANKING_SETTING_TIME_MIN, minTime);
setting.addProperty(RANKING_SETTING_TIME_FLOAT, floatTime);
setting.addProperty(RANKING_SETTING_RANKING_START, rankingStart);
setting.addProperty(RANKING_SETTING_RANKING_END, rankingStop);
setting.addProperty(RANKING_SETTING_RANKING_MODE, rankingMode);
setting.addProperty(RANKING_SETTING_RANKING_CONTENT_TYPE, rankingContentType);
setting.addProperty(RANKING_SETTING_PAGE_QUALITY, original ?
PixivDownload.PageQuality.ORIGINAL.name() :
PixivDownload.PageQuality.REGULAR.name());
if(pushInfoMap.containsKey(group)) {
log.info("群 {} 已存在Timer, 删除Timer...", group);
removePushGroup(fromGroup, groupId);
}
log.info("正在增加Timer...(Setting: {})", setting);
pushInfoMap.put(groupId, setting);
addPushTimer(groupId, setting);
return "已在 " + groupId + " 开启定时推送功能。";
pushInfoMap.put(group, setting);
addPushTimer(group, setting);
return "已在 " + group + " 开启定时推送功能。";
}
/**
@ -170,14 +174,24 @@ public class BotAdminCommandProcess {
}
}
/**
* 加载所有推送Timer
* @param flush 是否完全重载, 如为true则加载前会删除所有已加载的Timer
*/
public static void loadAllPushTimer(boolean flush) {
if(flush) {
RandomIntervalSendTimer.timerIdSet().forEach(id -> RandomIntervalSendTimer.getTimerById(id).destroy());
} else {
cleanPushTimer();
}
cleanPushTimer();
pushInfoMap.forEach(BotAdminCommandProcess::addPushTimer);
}
/**
* 根据设置增加Timer
* @param id 群组id
* @param setting jsonObject设置集
*/
private static void addPushTimer(long id, JsonObject setting) {
try {
RandomIntervalSendTimer.getTimerById(id);
@ -185,19 +199,65 @@ public class BotAdminCommandProcess {
} catch(NoSuchElementException ignored) {
}
int rankingStart = setting.has(RANKING_SETTING_RANKING_START) ? setting.get(RANKING_SETTING_RANKING_START).getAsInt() : 1;
int rankingEnd = setting.has(RANKING_SETTING_RANKING_END) ? setting.get(RANKING_SETTING_RANKING_END).getAsInt() : 150;
PixivURL.RankingMode rankingMode = PixivURL.RankingMode.MODE_DAILY;
PixivURL.RankingContentType rankingContentType = PixivURL.RankingContentType.TYPE_ILLUST;
PixivDownload.PageQuality pageQuality = PixivDownload.PageQuality.REGULAR;
if(rankingStart <= 0 || rankingStart > 500) {
log.warn("群组 [{}] - 无效的RankingStart设定值, 将重置为默认设定值(1): {}", id, rankingStart);
rankingStart = 1;
} else if(rankingEnd > 500 || rankingEnd <= 0) {
log.warn("群组 [{}] - 无效的RankingEnd设定值, 将重置为默认设定值(150): {}", id, rankingEnd);
rankingEnd = 150;
} else if(rankingStart > rankingEnd) {
log.warn("群组 [{}] - 无效的排行榜选取范围, 将重置为默认设定值(1 ~ 150): start={}, end={}", id, rankingStart, rankingEnd);
rankingStart = 1;
rankingEnd = 150;
}
if(setting.has(RANKING_SETTING_RANKING_MODE)) {
String value = setting.get(RANKING_SETTING_RANKING_MODE).getAsString().trim().toUpperCase();
try {
rankingMode = PixivURL.RankingMode.valueOf(value.startsWith("MODE_") ? value : "MODE_" + value);
} catch(IllegalArgumentException e) {
log.warn("群组ID [{}] - 无效的RankingMode设定值, 将重置为默认值: {}", id, value);
}
}
if(setting.has(RANKING_SETTING_RANKING_CONTENT_TYPE)) {
String value = setting.get(RANKING_SETTING_RANKING_CONTENT_TYPE).getAsString().trim().toUpperCase();
try {
rankingContentType = PixivURL.RankingContentType.valueOf(value.startsWith("TYPE_") ? value : "TYPE_" + value);
} catch(IllegalArgumentException e) {
log.warn("群组ID [{}] - 无效的RankingContentType设定值: {}", id, value);
}
}
if(setting.has(RANKING_SETTING_PAGE_QUALITY)) {
String value = setting.get(RANKING_SETTING_PAGE_QUALITY).getAsString().trim().toUpperCase();
try {
pageQuality = PixivDownload.PageQuality.valueOf(value);
} catch(IllegalArgumentException e) {
log.warn("群组ID [{}] - 无效的PageQuality设定值: {}", id, value);
}
}
AutoSender sender = new RandomRankingArtworksSender(
MessageSenderBuilder.getMessageSender(MessageSource.Group, id),
setting.get("ranking.start").getAsInt(),
setting.get("ranking.end").getAsInt(),
setting.get("pageQuality.original").getAsBoolean() ? PixivDownload.PageQuality.ORIGINAL : PixivDownload.PageQuality.REGULAR
rankingStart,
rankingEnd,
rankingMode, rankingContentType,
pageQuality
);
RandomIntervalSendTimer.createTimer(
RandomIntervalSendTimer timer = RandomIntervalSendTimer.createTimer(
id,
sender,
setting.get("time.min").getAsLong(),
setting.get("time.float").getAsInt(),
true, true);
log.info("群组 {} 已创建对应Timer: {}", id, Integer.toHexString(timer.hashCode()));
}
/**
@ -206,10 +266,11 @@ public class BotAdminCommandProcess {
* @throws NoSuchElementException 当这个群号没有定时器的时候抛出异常
*/
@Command
public static String removePushGroup(@Argument(name = "group") long id) {
RandomIntervalSendTimer.getTimerById(id).destroy();
pushInfoMap.remove(id);
return "已关闭群 " + id + " 的美图推送功能。";
public static String removePushGroup(@Argument(name = "$fromGroup") long fromGroup, @Argument(name = "group", force = false) long id) {
long group = id <= 0 ? fromGroup : id;
RandomIntervalSendTimer.getTimerById(group).destroy();
pushInfoMap.remove(group);
return "已关闭群 " + group + " 的美图推送功能。";
}
/**
@ -229,14 +290,17 @@ public class BotAdminCommandProcess {
Set<String> keys = BotCommandProcess.reportStore.keys();
StringBuilder msgBuilder = new StringBuilder();
msgBuilder.append("当前被报告的作品列表:\n");
int count = 1;
for(String key : keys) {
String illustIdStr = key.substring(key.indexOf(".") + 1);
JsonObject report = BotCommandProcess.reportStore.getCache(illustIdStr).getAsJsonObject();
log.debug("{} - Report: {}", illustIdStr, report);
String reason = report.get("reason").isJsonNull() ? "" : report.get("reason").getAsString();
msgBuilder.append(illustIdStr)
.append("(").append(dateFormat.format(new Date(report.get("reportTime").getAsLong()))).append(")")
.append(reason).append("\n");
msgBuilder.append(count).append(". 作品Id: ").append(illustIdStr)
.append("(").append(dateFormat.format(new Date(report.get("reportTime").getAsLong()))).append(")\n")
.append("报告者QQ").append(report.get("fromQQ").getAsLong()).append("\n")
.append("报告所在群:").append(report.get("fromGroup").getAsLong()).append("\n")
.append("报告原因:\n").append(reason).append("\n");
}
return msgBuilder.toString();
}

View File

@ -25,7 +25,6 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.text.SimpleDateFormat;
@ -35,14 +34,14 @@ import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
@SuppressWarnings({"SynchronizationOnLocalVariableOrMethodParameter", "SameParameterValue"})
public class BotCommandProcess {
private final static PixivDownload pixivDownload = new PixivDownload(Main.cookieStore, Main.proxy);
private final static Logger log = LoggerFactory.getLogger(BotCommandProcess.class.getSimpleName());
private final static Logger log = LoggerFactory.getLogger(BotCommandProcess.class.getName());
private final static File imageStoreDir = new File(System.getProperty("cgj.cqRootDir"), "data/image/cgj/");
public final static Properties globalProp = new Properties();
private final static File imageStoreDir = new File(System.getProperty("cgj.botDataDir"), "data/image/cgj/");
private final static Gson gson = new GsonBuilder()
.serializeNulls()
.create();
@ -76,18 +75,7 @@ public class BotCommandProcess {
public static void initialize() {
log.info("正在初始化...");
File globalPropFile = new File("./global.properties");
if(globalPropFile.exists() && globalPropFile.isFile()) {
log.info("正在加载全局配置文件...");
try {
globalProp.load(new FileInputStream(globalPropFile));
log.info("全局配置文件加载完成.");
} catch (IOException e) {
log.error("加载全局配置文件时发生异常", e);
}
} else {
log.info("未找到全局配置文件,跳过加载.");
}
SettingProperties.loadProperties();
try {
imageCacheExecutor.addHandler(new ImageCacheHandler());
@ -113,24 +101,42 @@ public class BotCommandProcess {
public static String help() {
StringBuilder helpStrBuilder = new StringBuilder();
helpStrBuilder.append("CGJ Bot使用指南").append("\n");
helpStrBuilder.append("使用方法:.cgj <命令> [参数...]").append("\n");
helpStrBuilder.append("例如查询作品信息功能:").append("\n");
helpStrBuilder.append(".cgj artwork -id 80846159").append("\n");
helpStrBuilder.append("目前可用的命令:").append("\n");
helpStrBuilder.append("\t").append("ranking - 获取今天或指定日期排行榜的前10名作品").append("\n");
helpStrBuilder.append("\t\t").append("-date - 指定查询日期(年-月-日)").append("\n");
helpStrBuilder.append("\t\t").append("-type - 排行榜类型(illust/插画, ugoira/动图, manga/漫画)").append("\n");
helpStrBuilder.append("\t").append("search - 搜索指定关键词并显示前10个作品").append("\n");
helpStrBuilder.append("\t\t").append("-content - 搜索内容").append("\n");
helpStrBuilder.append("\t").append("artworks - 获取作品的Pixiv页面").append("\n");
helpStrBuilder.append("\t").append("link - 获取作品的Pixiv页面").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品id").append("\n");
helpStrBuilder.append("\t").append("info - 获取Pixiv作品信息").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品id").append("\n");
helpStrBuilder.append("\t").append("image - 获取指定作品的图片").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品id").append("\n");
helpStrBuilder.append("\t\t").append("-quality - 图片质量(original/原图 regular/预览图)").append("\n");
helpStrBuilder.append("\t\t").append("-page - 页数").append("\n");
helpStrBuilder.append("\t").append("report - 报告不当作品").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品Id").append("\n");
helpStrBuilder.append("\t\t").append("-msg - 报告原因").append("\n");
return helpStrBuilder.toString();
}
@Command(commandName = "info")
public static String artworkInfo(@Argument(name = "id") int illustId) {
public static String artworkInfo(@Argument(name = "$fromGroup") long fromGroup, @Argument(name = "id") int illustId) {
if(illustId <= 0) {
return "错误的作品id";
}
try {
if(isNoSafe(illustId, globalProp, false) || isReported(illustId)) {
if(isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false) || isReported(illustId)) {
return "阅览禁止:该作品已被封印!!";
}
@ -146,7 +152,7 @@ public class BotCommandProcess {
builder.append("评论数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.COMMENT.attrName).getAsInt()).append("\n");
builder.append("页数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.PAGE.attrName).getAsInt()).append("\n");
builder.append("---------------- 作品图片 ----------------\n");
builder.append(getImageById(illustId, PixivDownload.PageQuality.REGULAR, 1)).append("\n");
builder.append(getImageById(fromGroup, illustId, PixivDownload.PageQuality.REGULAR, 1)).append("\n");
builder.append("使用 \".cgj image -id ")
.append(illustId)
.append("\" 获取原图。\n如有不当作品可使用\".cgj report -id ")
@ -160,6 +166,7 @@ public class BotCommandProcess {
@Command
public static String ranking(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(force = false, name = "date") Date queryTime,
@Argument(force = false, name = "mode", defaultValue = "DAILY") String contentMode,
@Argument(force = false, name = "type", defaultValue = "ILLUST") String contentType
@ -211,9 +218,10 @@ public class BotCommandProcess {
try {
int index = 0;
int itemLimit = 10;
String itemLimitPropertyKey = "ranking.ItemCountLimit";
String itemLimitPropertyKey = "ranking.itemCountLimit";
try {
itemLimit = Integer.parseInt(globalProp.getProperty(itemLimitPropertyKey, "10"));
itemLimit = Integer.parseInt(SettingProperties
.getProperty(SettingProperties.GLOBAL, itemLimitPropertyKey, "10"));
} catch(NumberFormatException e) {
log.warn("配置项 {} 的参数值格式有误!", itemLimitPropertyKey);
}
@ -221,12 +229,12 @@ public class BotCommandProcess {
int imageLimit = 3;
String imageLimitPropertyKey = "ranking.imageCountLimit";
try {
imageLimit = Integer.parseInt(globalProp.getProperty(imageLimitPropertyKey, "3"));
imageLimit = Integer.parseInt(
SettingProperties.getProperty(SettingProperties.GLOBAL, imageLimitPropertyKey, "3"));
} catch(NumberFormatException e) {
log.warn("配置项 {} 的参数值格式有误!", imageLimitPropertyKey);
}
//TODO(LamGC, 2020.4.11): 将JsonRedisCacheStore更改为使用Redis的List集合, 以提高性能
List<JsonObject> rankingInfoList = getRankingInfoByCache(type, mode, queryDate, 1, Math.max(0, itemLimit), false);
if(rankingInfoList.isEmpty()) {
return "无法查询排行榜,可能排行榜尚未更新。";
@ -243,7 +251,7 @@ public class BotCommandProcess {
resultBuilder.append(rank).append(". (id: ").append(illustId).append(") ").append(title)
.append("(Author: ").append(authorName).append(",").append(authorId).append(") ").append(pagesCount).append("p.\n");
if (index <= imageLimit) {
resultBuilder.append(getImageById(illustId, PixivDownload.PageQuality.REGULAR, 1)).append("\n");
resultBuilder.append(getImageById(fromGroup, illustId, PixivDownload.PageQuality.REGULAR, 1)).append("\n");
}
}
} catch (IOException e) {
@ -259,14 +267,28 @@ public class BotCommandProcess {
return "功能未完成";
}
/**
* 搜索命令
* @param content 搜索内容
* @param type 搜索类型
* @param area 搜索区域
* @param includeKeywords 包括关键字
* @param excludeKeywords 排除关键字
* @param contentOption 搜索选项
* @param pagesIndex 搜索页索引
* @return 返回搜索内容消息
* @throws IOException 当搜索发生异常时抛出
*/
@Command
public static String search(@Argument(name = "content") String content,
@Argument(name = "type", force = false) String type,
@Argument(name = "area", force = false) String area,
@Argument(name = "in", force = false) String includeKeywords,
@Argument(name = "ex", force = false) String excludeKeywords,
@Argument(name = "contentOption", force = false) String contentOption,
@Argument(name = "page", force = false, defaultValue = "1") int pagesIndex
public static String search(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "content") String content,
@Argument(name = "type", force = false) String type,
@Argument(name = "area", force = false) String area,
@Argument(name = "in", force = false) String includeKeywords,
@Argument(name = "ex", force = false) String excludeKeywords,
@Argument(name = "contentOption", force = false) String contentOption,
@Argument(name = "page", force = false, defaultValue = "1") int pagesIndex
) throws IOException {
log.info("正在执行搜索...");
PixivSearchBuilder searchBuilder = new PixivSearchBuilder(Strings.isNullOrEmpty(content) ? "" : content);
@ -330,7 +352,8 @@ public class BotCommandProcess {
}
long expire = 7200 * 1000;
String propValue = globalProp.getProperty("cache.searchBody.expire", "7200000");
String propValue = SettingProperties
.getProperty(SettingProperties.GLOBAL, "cache.searchBody.expire", "7200000");
try {
expire = Long.parseLong(propValue);
} catch (Exception e) {
@ -355,7 +378,8 @@ public class BotCommandProcess {
log.debug("正在处理信息...");
int limit = 8;
try {
limit = Integer.parseInt(globalProp.getProperty("search.ItemCountLimit", "8"));
limit = Integer.parseInt(SettingProperties.
getProperty(SettingProperties.GLOBAL, "search.itemCountLimit", "8"));
} catch (Exception e) {
log.warn("参数转换异常!将使用默认值(" + limit + ")", e);
}
@ -399,8 +423,8 @@ public class BotCommandProcess {
//pageCount
String imageMsg = getImageById(illustId, PixivDownload.PageQuality.REGULAR, 1);
if (isNoSafe(illustId, globalProp, true)) {
String imageMsg = getImageById(fromGroup, illustId, PixivDownload.PageQuality.REGULAR, 1);
if (isNoSafe(illustId, SettingProperties.getProperties(fromGroup), true)) {
log.warn("作品Id {} 为R-18作品, 跳过.", illustId);
continue;
} else if(isReported(illustId)) {
@ -423,7 +447,9 @@ public class BotCommandProcess {
}
@Command(commandName = "pages")
public static String getPagesList(@Argument(name = "id") int illustId, @Argument(name = "quality", force = false) PixivDownload.PageQuality quality) {
public static String getPagesList(
@Argument(name = "id") int illustId,
@Argument(name = "quality", force = false) PixivDownload.PageQuality quality) {
try {
List<String> pagesList = PixivDownload.getIllustAllPageDownload(pixivDownload.getHttpClient(), pixivDownload.getCookieStore(), illustId, quality);
StringBuilder builder = new StringBuilder("作品ID ").append(illustId).append(" 共有").append(pagesList.size()).append("页:").append("\n");
@ -438,10 +464,10 @@ public class BotCommandProcess {
}
}
@Command(commandName = "artwork")
public static String artworksLink(@Argument(name = "id") int illustId) {
@Command(commandName = "link")
public static String artworksLink(@Argument(name = "$fromGroup") long fromGroup, @Argument(name = "id") int illustId) {
try {
if (isNoSafe(illustId, globalProp, false)) {
if (isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) {
log.warn("作品Id {} 已被屏蔽.", illustId);
return "由于相关设置,该作品已被屏蔽!";
} else if(isReported(illustId)) {
@ -463,9 +489,11 @@ public class BotCommandProcess {
* @return 如果成功, 返回BotCode, 否则返回错误信息.
*/
@Command(commandName = "image")
public static String getImageById(@Argument(name = "id") int illustId,
@Argument(name = "quality", force = false) PixivDownload.PageQuality quality,
@Argument(name = "page", force = false, defaultValue = "1") int pageIndex) {
public static String getImageById(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "id") int illustId,
@Argument(name = "quality", force = false) PixivDownload.PageQuality quality,
@Argument(name = "page", force = false, defaultValue = "1") int pageIndex) {
log.debug("IllustId: {}, Quality: {}, PageIndex: {}", illustId, quality.name(), pageIndex);
List<String> pagesList;
try {
@ -488,7 +516,7 @@ public class BotCommandProcess {
}
try {
if (isNoSafe(illustId, globalProp, false)) {
if (isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) {
log.warn("作品 {} 存在R-18内容且设置\"image.allowR18\"为false将屏蔽该作品不发送.", illustId);
return "(根据设置,该作品已被屏蔽!)";
} else if(isReported(illustId)) {
@ -538,6 +566,12 @@ public class BotCommandProcess {
return getImageToBotCode(imageCache.get(fileName), false).toString();
}
/**
* 通过文件获取图片的BotCode代码
* @param targetFile 图片文件
* @param updateCache 是否刷新缓存(只是让机器人重新上传, 如果上传接口有重复检测的话是无法处理的)
* @return 返回设定好参数的BotCode
*/
private static BotCode getImageToBotCode(File targetFile, boolean updateCache) {
String fileName = targetFile.getName();
BotCode code = BotCode.parse(CQCode.image(getImageStoreDir().getName() + "/" + fileName));
@ -554,7 +588,7 @@ public class BotCommandProcess {
illustPreLoadDataCache.clear();
pagesCache.clear();
searchBodyCache.clear();
File imageStoreDir = new File(System.getProperty("cgj.cqRootDir") + "data/image/cgj/");
File imageStoreDir = new File(System.getProperty("cgj.botDataDir") + "data/image/cgj/");
File[] listFiles = imageStoreDir.listFiles();
if (listFiles == null) {
log.debug("图片缓存目录为空或内部文件获取失败!");
@ -574,11 +608,13 @@ public class BotCommandProcess {
* @return 返回提示信息
*/
@Command
public static String report(@Argument(name = "id") int illustId, @Argument(name = "msg", force = false) String reason) {
public static String report(@Argument(name = "$fromGroup") long fromGroup, @Argument(name = "$fromQQ") long fromQQ, @Argument(name = "id") int illustId, @Argument(name = "msg", force = false) String reason) {
log.warn("收到作品反馈(IllustId: {}, 原因: {})", illustId, reason);
JsonObject reportJson = new JsonObject();
reportJson.addProperty("illustId", illustId);
reportJson.addProperty("reportTime", new Date().getTime());
reportJson.addProperty("fromGroup", fromGroup);
reportJson.addProperty("fromQQ", fromQQ);
reportJson.addProperty("reason", reason);
reportStore.update(String.valueOf(illustId), reportJson, 0);
return "色图姬收到了你的报告,将屏蔽该作品并对作品违规情况进行核实,感谢你的反馈!";
@ -617,7 +653,7 @@ public class BotCommandProcess {
String illustIdStr = buildSyncKey(Integer.toString(illustId));
JsonObject illustInfoObj = null;
if (!illustInfoCache.exists(illustIdStr) || flushCache) {
synchronized (illustIdStr) { // TODO: 这里要不做成HashMap存储key而避免使用常量池?
synchronized (illustIdStr) {
if (!illustInfoCache.exists(illustIdStr) || flushCache) {
illustInfoObj = pixivDownload.getIllustInfoByIllustId(illustId);
illustInfoCache.update(illustIdStr, illustInfoObj, null);
@ -646,13 +682,14 @@ public class BotCommandProcess {
if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) {
synchronized (illustIdStr) {
if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) {
log.info("IllustId {} 缓存失效, 正在更新...", illustId);
log.debug("IllustId {} 缓存失效, 正在更新...", illustId);
JsonObject preLoadDataObj = pixivDownload.getIllustPreLoadDataById(illustId)
.getAsJsonObject("illust")
.getAsJsonObject(Integer.toString(illustId));
long expire = 7200 * 1000;
String propValue = globalProp.getProperty("cache.illustPreLoadData.expire", "7200000");
String propValue = SettingProperties.
getProperty(SettingProperties.GLOBAL, "cache.illustPreLoadData.expire", "7200000");
log.debug("PreLoadData有效时间设定: {}", propValue);
try {
expire = Long.parseLong(propValue);
@ -662,7 +699,7 @@ public class BotCommandProcess {
result = preLoadDataObj;
illustPreLoadDataCache.update(illustIdStr, preLoadDataObj, expire);
log.info("作品Id {} preLoadData缓存已更新(有效时间: {})", illustId, expire);
log.debug("作品Id {} preLoadData缓存已更新(有效时间: {})", illustId, expire);
}
}
}
@ -734,21 +771,22 @@ public class BotCommandProcess {
if(!rankingCache.exists(requestSign) || flushCache) {
synchronized(requestSign) {
if(!rankingCache.exists(requestSign) || flushCache) {
log.info("Ranking缓存失效, 正在更新...(RequestSign: {})", requestSign);
log.debug("Ranking缓存失效, 正在更新...(RequestSign: {})", requestSign);
List<JsonObject> rankingResult = pixivDownload.getRanking(contentType, mode, queryDate, 1, 500);
long expireTime = 0;
if(rankingResult.size() == 0) {
log.info("数据获取失败, 将设置浮动有效时间以准备下次更新.");
expireTime = 5400000 + expireTimeFloatRandom.nextInt(1800000);
log.warn("数据获取失败, 将设置浮动有效时间以准备下次更新. (ExpireTime: {}ms)", expireTime);
}
result = new ArrayList<>(rankingResult).subList(start - 1, start + range - 1);
rankingCache.update(requestSign, rankingResult,
rankingResult.size() == 0 ? 5400000 + expireTimeFloatRandom.nextInt(1800000) : 0);
log.info("Ranking缓存更新完成.(RequestSign: {})", requestSign);
rankingCache.update(requestSign, rankingResult, expireTime);
log.debug("Ranking缓存更新完成.(RequestSign: {})", requestSign);
}
}
}
if (Objects.isNull(result)) {
result = rankingCache.getCache(requestSign, start, range);
result = rankingCache.getCache(requestSign, start - 1, range);
log.debug("RequestSign [{}] 缓存命中.", requestSign);
}
log.debug("Result-Length: {}", result.size());

View File

@ -0,0 +1,77 @@
package net.lamgc.cgj.bot;
import net.lamgc.cgj.bot.event.MessageEvent;
import net.lamgc.cgj.bot.event.VirtualLoadMessageEvent;
import net.lamgc.utils.event.EventExecutor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Properties;
/**
* 消息事件处理调试器.
* <p>当启用了消息事件处理调试后, 将会根据调试器代号调用指定调试器</p>
*/
@SuppressWarnings("unused")
public enum MessageEventExecutionDebugger {
/**
* PM - 压力测试
*/
PM ((executor, event, properties, log) -> {
MessageEvent virtualLoadEvent = VirtualLoadMessageEvent.toVirtualLoadMessageEvent(event, false);
int rotation = 5;
int number = 50;
int interval = 2500;
try {
rotation = Integer.parseInt(properties.getProperty("debug.pm.rotation", "5"));
} catch(NumberFormatException ignored) {}
try {
number = Integer.parseInt(properties.getProperty("debug.pm.number", "50"));
} catch(NumberFormatException ignored) {}
try {
interval = Integer.parseInt(properties.getProperty("debug.pm.interval", "2500"));
} catch(NumberFormatException ignored) {}
boolean interrupted = false;
Thread currentThread = Thread.currentThread();
for(int rotationCount = 0; rotationCount < rotation && !interrupted; rotationCount++) {
for(int sendCount = 0; sendCount < number; sendCount++) {
if(currentThread.isInterrupted()) {
interrupted = true;
break;
}
executor.executor(virtualLoadEvent);
}
try {
Thread.sleep(interval);
} catch (InterruptedException e) {
break;
}
}
});
public final MessageExecuteDebugger debugger;
MessageEventExecutionDebugger(MessageExecuteDebugger debugger) {
this.debugger = debugger;
}
public static Logger getDebuggerLogger(MessageEventExecutionDebugger debugger) {
return LoggerFactory.getLogger(MessageEventExecutionDebugger.class.getName() + "." + debugger.name());
}
@FunctionalInterface
public interface MessageExecuteDebugger {
/**
* 接收事件并根据指定需求处理
* @param executor 事件执行器
* @param event 消息事件对象
* @param properties 配置项, 调试器应按'debug.[debuggerName].'为前缀存储相应调试信息
* @throws Exception 当抛出异常则打断调试, 并输出至日志
*/
void accept(EventExecutor executor, MessageEvent event, Properties properties, Logger logger) throws Exception;
}
}

View File

@ -1,5 +1,6 @@
package net.lamgc.cgj.bot;
import com.google.common.base.Throwables;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@ -13,7 +14,7 @@ import java.util.concurrent.atomic.AtomicBoolean;
public class RandomIntervalSendTimer extends TimerTask {
private final static Timer timer = new Timer("Thread-RIST");
private final static Logger log = LoggerFactory.getLogger("RandomIntervalSendTimer");
private final static Logger log = LoggerFactory.getLogger(RandomIntervalSendTimer.class.getName());
private final static Map<Long, RandomIntervalSendTimer> timerMap = new HashMap<>();
private final long timerId;
@ -23,6 +24,8 @@ public class RandomIntervalSendTimer extends TimerTask {
private final int floatTime;
private AtomicBoolean loop = new AtomicBoolean();
private final AtomicBoolean start = new AtomicBoolean();
private final String hashId = Integer.toHexString(this.hashCode());
/**
* 创建一个随机延迟发送器
@ -96,7 +99,9 @@ public class RandomIntervalSendTimer extends TimerTask {
public void start(boolean loop) {
this.loop.set(loop);
long nextDelay = time + timeRandom.nextInt(floatTime);
log.info("定时器 {} 下一延迟: {}ms ({}min)", Integer.toHexString(this.hashCode()), nextDelay, nextDelay / 1000F / 60F);
Date nextDate = new Date();
nextDate.setTime(nextDate.getTime() + nextDelay);
log.info("定时器 {} 下一延迟: {}ms ({})", hashId, nextDelay, nextDate);
if(start.get()) {
try {
Field state = this.getClass().getSuperclass().getDeclaredField("state");
@ -114,7 +119,11 @@ public class RandomIntervalSendTimer extends TimerTask {
@Override
public void run() {
sender.send();
try {
sender.send();
} catch (Exception e) {
log.error("定时器 {} 执行时发生异常:\n{}", Integer.toHexString(this.hashCode()), Throwables.getStackTraceAsString(e));
}
if (this.loop.get()) {
start();
}
@ -127,6 +136,7 @@ public class RandomIntervalSendTimer extends TimerTask {
@Override
public boolean cancel() {
start.set(false);
loop.set(false);
return super.cancel();
}

View File

@ -21,20 +21,30 @@ public class RandomRankingArtworksSender extends AutoSender {
private final Logger log;
private final int rankingStart;
private final int rankingStop;
private final PixivURL.RankingMode mode;
private final PixivURL.RankingContentType contentType;
private final PixivDownload.PageQuality quality;
/**
* 构造一个推荐作品发送器
* @param messageSender 消息发送器
* @param rankingStart 排行榜开始范围(从1开始, 名次)
* @param rankingStop 排名榜结束范围(包括该名次)
* @param quality 图片质量, 详见{@link net.lamgc.cgj.pixiv.PixivDownload.PageQuality}
* @param rankingStart 排行榜开始范围(从1开始, 名次)如传入0或负数则为默认值默认为1
* @param rankingStop 排名榜结束范围(包括该名次)如传入0或负数则为默认值默认为150
* @param mode 排行榜模式
* @param contentType 排行榜内容类型
* @param quality 图片质量, 详见{@link PixivDownload.PageQuality}
* @throws IndexOutOfBoundsException 当 rankingStart > rankingStop时抛出
*/
public RandomRankingArtworksSender(MessageSender messageSender, int rankingStart, int rankingStop, PixivDownload.PageQuality quality) {
public RandomRankingArtworksSender(MessageSender messageSender, int rankingStart, int rankingStop, PixivURL.RankingMode mode, PixivURL.RankingContentType contentType, PixivDownload.PageQuality quality) {
super(messageSender);
log = LoggerFactory.getLogger("RecommendArtworksSender@" + Integer.toHexString(this.hashCode()));
this.mode = mode;
this.contentType = contentType;
log = LoggerFactory.getLogger(this.toString());
this.rankingStart = rankingStart > 0 ? rankingStart : 1;
this.rankingStop = rankingStop > 0 ? rankingStop : 150;
if(this.rankingStart > this.rankingStop) {
throw new IndexOutOfBoundsException("rankingStart=" + this.rankingStart + ", rankingStop=" + this.rankingStop);
}
this.quality = quality == null ? PixivDownload.PageQuality.REGULAR : quality;
}
@ -50,16 +60,16 @@ public class RandomRankingArtworksSender extends AutoSender {
}
queryDate = calendar.getTime();
int selectRanking = rankingStart + new Random().nextInt(rankingStop + 1);
int selectRanking = rankingStart + new Random().nextInt(rankingStop - rankingStart + 1);
try {
List<JsonObject> rankingList = BotCommandProcess.getRankingInfoByCache(
PixivURL.RankingContentType.TYPE_ILLUST,
PixivURL.RankingMode.MODE_DAILY,
contentType,
mode,
queryDate,
selectRanking,
1, false);
log.info("RankingResult.size: {}", rankingList.size());
log.debug("RankingResult.size: {}", rankingList.size());
if(rankingList.size() != 1) {
log.error("排行榜选取失败!(获取到了多个结果)");
return;
@ -67,7 +77,7 @@ public class RandomRankingArtworksSender extends AutoSender {
JsonObject rankingInfo = rankingList.get(0);
int illustId = rankingInfo.get("illust_id").getAsInt();
if(BotCommandProcess.isNoSafe(illustId, BotCommandProcess.globalProp, false)) {
if(BotCommandProcess.isNoSafe(illustId, SettingProperties.getProperties(SettingProperties.GLOBAL), false)) {
log.warn("作品为r18作品, 取消本次发送.");
return;
} else if(BotCommandProcess.isReported(illustId)) {
@ -79,7 +89,7 @@ public class RandomRankingArtworksSender extends AutoSender {
message.append("#美图推送 - 今日排行榜 第 ").append(rankingInfo.get("rank").getAsInt()).append("\n");
message.append("标题:").append(rankingInfo.get("title").getAsString()).append("(").append(illustId).append(")\n");
message.append("作者:").append(rankingInfo.get("user_name").getAsString()).append("\n");
message.append(BotCommandProcess.getImageById(illustId, quality, 1));
message.append(BotCommandProcess.getImageById(0, illustId, quality, 1));
message.append("\n如有不当作品可使用\".cgj report -id ").append(illustId).append("\"向色图姬反馈。");
getMessageSender().sendMessage(message.toString());
} catch (IOException e) {

View File

@ -1,6 +1,5 @@
package net.lamgc.cgj.bot;
import com.google.common.base.Throwables;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.event.VirtualLoadMessageEvent;
import net.lamgc.cgj.pixiv.PixivURL;
@ -28,10 +27,10 @@ public class RankingUpdateTimer {
cal.setTime(firstRunDate == null ? new Date() : firstRunDate);
LocalDate currentLocalDate = LocalDate.now();
if(cal.get(Calendar.DAY_OF_YEAR) <= currentLocalDate.getDayOfYear() && cal.get(Calendar.HOUR_OF_DAY) >= 12) {
cal.set(Calendar.DAY_OF_YEAR, cal.get(Calendar.DAY_OF_YEAR) + 1);
cal.set(Calendar.DAY_OF_YEAR, currentLocalDate.getDayOfYear() + 1);
}
cal.set(Calendar.HOUR_OF_DAY, 12);
cal.set(Calendar.MINUTE, 10);
cal.set(Calendar.MINUTE, 0);
cal.set(Calendar.SECOND, 0);
cal.set(Calendar.MILLISECOND, 0);
@ -66,15 +65,9 @@ public class RankingUpdateTimer {
log.debug("不支持的类型, 填空值跳过...(类型: {}.{})", rankingMode.name(), contentType.name());
}
log.info("当前排行榜类型: {}.{}, 正在更新...", rankingMode.name(), contentType.name());
try {
//BotCommandProcess.getRankingInfoByCache(contentType, rankingMode, calendar.getTime(), 1, 0, true);
BotEventHandler.executor.executorSync(
new VirtualLoadMessageEvent(0,0,
".cgj ranking -type=" + contentType.name() + " -mode=" + rankingMode.name()));
log.info("排行榜 {}.{} 更新完成.", rankingMode.name(), contentType.name());
} catch (InterruptedException e) {
log.error("排行榜 {}.{} 更新时发生异常. \n{}", rankingMode.name(), contentType.name(), Throwables.getStackTraceAsString(e));
}
BotEventHandler.executeMessageEvent(new VirtualLoadMessageEvent(0,0,
".cgj ranking -type=" + contentType.name() + " -mode=" + rankingMode.name()));
log.info("排行榜 {}.{} 负载指令已投递.", rankingMode.name(), contentType.name());
}
}
log.warn("定时任务更新完成.");

View File

@ -0,0 +1,241 @@
package net.lamgc.cgj.bot;
import com.google.common.base.Throwables;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.*;
import java.nio.charset.StandardCharsets;
import java.util.*;
public final class SettingProperties {
private final static Logger log = LoggerFactory.getLogger("SettingProperties");
private final static File globalPropFile = new File(getPropertiesDir(), "global.properties");
private final static Properties globalProp = new Properties();
private final static Map<Long, Properties> groupPropMap = new HashMap<>();
private final static Set<Long> changeList = Collections.synchronizedSet(new HashSet<>());
/**
* 全局配置项
*/
public final static long GLOBAL = 0;
/**
* 清空所有Properties.
*/
public static void clearProperties() {
groupPropMap.clear();
globalProp.clear();
}
/**
* 加载配置文件
*/
public static void loadProperties() {
loadGlobalProperties();
File[] files = getPropertiesDir()
.listFiles((dir, fileName) -> fileName.startsWith("group.") && fileName.endsWith(".properties"));
if(files == null) {
log.error("检索群组配置文件失败, 可能是被拒绝访问.");
return;
}
for (File file : files) {
String name = file.getName();
long groupId;
try {
groupId = Long.parseLong(name.substring(name.indexOf("group.") + 6, name.lastIndexOf(".properties")));
} catch (NumberFormatException e) {
log.error("非法的配置文件名: {}", name);
continue;
}
if(!groupPropMap.containsKey(groupId)) {
groupPropMap.put(groupId, new Properties(globalProp));
}
loadGroupProperties(groupId, groupPropMap.get(groupId));
}
}
/**
* 保存配置项
*/
public static void saveProperties() {
log.info("正在保存所有配置...");
saveGlobalProperties();
for (Long groupId : groupPropMap.keySet()) {
if(!changeList.contains(groupId)) {
log.debug("群组 {} 配置无改动, 忽略保存.", groupId);
return;
}
log.debug("正在保存群组 {} 配置文件...", groupId);
saveGroupProperties(groupId);
}
log.info("配置保存完成.");
}
/**
* 保存指定群组的配置文件
* @param groupId 要保存配置的群组Id
*/
private static void saveGroupProperties(long groupId) {
try {
saveGroupProperties(groupId, getGroupProperties(groupId));
} catch (IOException e) {
log.error("群组 {} 配置保存失败\n{}", groupId, Throwables.getStackTraceAsString(e));
}
}
private static void saveGroupProperties(Long groupId, Properties properties) throws IOException {
File groupPropFile = new File(getPropertiesDir(), "group." + groupId + ".properties");
if((!groupPropFile.exists() || !groupPropFile.isFile()) && (!groupPropFile.delete() || !groupPropFile.createNewFile())) {
log.error("群组 {} 配置文件创建失败!", groupId);
return;
}
saveProperties(properties, new FileOutputStream(groupPropFile));
}
private static void loadGlobalProperties() {
if(globalPropFile.exists() && globalPropFile.isFile()) {
log.info("正在加载全局配置文件...");
try (Reader reader = new InputStreamReader(new FileInputStream(globalPropFile), StandardCharsets.UTF_8)) {
globalProp.load(reader);
log.info("全局配置文件加载完成.");
} catch (IOException e) {
log.error("加载全局配置文件时发生异常", e);
}
} else {
log.info("未找到全局配置文件,跳过加载.");
}
}
/**
* 保存全局配置项
*/
private static void saveGlobalProperties() {
try {
if((!globalPropFile.exists() || !globalPropFile.isFile()) && (!globalPropFile.delete() || !globalPropFile.createNewFile())) {
log.error("创建全局配置文件失败.");
return;
}
saveProperties(globalProp, new FileOutputStream(globalPropFile));
} catch (IOException e) {
log.error("全局配置文件保存时发生异常", e);
}
}
private static void loadGroupProperties(long groupId, Properties properties) {
File propFile = new File(getPropertiesDir(), "group." + groupId + ".properties");
Properties groupProp = Objects.requireNonNull(properties);
if(!propFile.exists() || !propFile.isFile()) {
log.warn("群组 {} 配置文件不存在, 或不是一个文件.({})", groupId, propFile.getAbsolutePath());
return;
}
try (Reader reader = new InputStreamReader(new FileInputStream(propFile), StandardCharsets.UTF_8)) {
groupProp.load(reader);
} catch (IOException e) {
log.error("读取群组 {} 群配置文件时发生异常:\n{}", groupId, Throwables.getStackTraceAsString(e));
}
}
private static void saveProperties(Properties properties, OutputStream stream) throws IOException {
properties.store(new OutputStreamWriter(stream, StandardCharsets.UTF_8), null);
}
/**
* 获取配置文件目录
* @return 返回目录File对象.
*/
private static File getPropertiesDir() {
File propDir = new File(System.getProperty("cgj.botDataDir"), "/setting/");
if((!propDir.exists() || !propDir.isDirectory()) && (!propDir.delete() || !propDir.mkdirs())) {
log.warn("Setting文件夹创建失败!");
}
return propDir;
}
public static String getProperty(long groupId, String key) {
return getProperty(groupId, key, null);
}
public static String getProperty(long groupId, String key, String defaultValue) {
if(groupId <= 0) {
return globalProp.getProperty(key, defaultValue);
} else {
Properties properties = groupPropMap.get(groupId);
return properties == null ? defaultValue : properties.getProperty(key, defaultValue);
}
}
/**
* 设置配置项
* @param groupId 群组Id, 如为0或负数则为全局配置
* @param key 配置项key名
* @param value 欲设置的新值, 如为null则删除该配置项
* @return 返回上一次设定值
*/
public static String setProperty(long groupId, String key, String value) {
Objects.requireNonNull(key);
Properties targetProperties;
if(groupId <= 0) {
changeList.add(groupId);
targetProperties = getGroupProperties(groupId);
} else {
targetProperties = globalProp;
}
String lastValue = targetProperties.getProperty(key);
if(value != null) {
targetProperties.setProperty(key, value);
} else {
targetProperties.remove(key);
}
return lastValue;
}
/**
* 获取GlobalProperties
* @return 全局Properties
*/
private static Properties getGlobalProperties() {
return globalProp;
}
/**
* 获取群组Properties
* @param groupId 群组Id
* @return 如果存在, 返回Properties, 不存在返回null.
* @throws IllegalArgumentException 当群组Id 小于或等于0 时抛出.
*/
private static Properties getGroupProperties(long groupId) {
if (groupId <= 0) {
throw new IllegalArgumentException("Group number cannot be 0 or negative: " + groupId);
}
if(!groupPropMap.containsKey(groupId)) {
groupPropMap.put(groupId, new Properties(globalProp));
}
return groupPropMap.get(groupId);
}
/**
* 获取群组 Properties, 如果指定群组没有 Properties, 则使用GlobalProperties.
* @param groupId 指定群组Id
* @return 如果群组存在所属Properties, 则返回群组Properties, 否则返回GlobalProperties.
*/
public static Properties getProperties(long groupId) {
if(groupPropMap.containsKey(groupId)) {
return groupPropMap.get(groupId);
}
return getGlobalProperties();
}
}

View File

@ -17,7 +17,7 @@ public class HotDataCacheStore<T> implements CacheStore<T> {
private final long expireTime;
private final int expireFloatRange;
private final Random random = new Random();
private final Logger log = LoggerFactory.getLogger(HotDataCacheStore.class.getSimpleName() + "@" + Integer.toHexString(this.hashCode()));
private final Logger log = LoggerFactory.getLogger(this.toString());
/**
* 构造热点缓存存储对象

View File

@ -21,7 +21,7 @@ import java.util.Set;
public class ImageCacheHandler implements EventHandler {
private final static Logger log = LoggerFactory.getLogger("ImageCacheHandler");
private final static Logger log = LoggerFactory.getLogger(ImageCacheHandler.class.getName());
private final static HttpClient httpClient = HttpClientBuilder.create().setProxy(Main.proxy).build();
@ -30,7 +30,7 @@ public class ImageCacheHandler implements EventHandler {
@SuppressWarnings("unused")
public void getImageToCache(ImageCacheObject event) {
if(cacheQueue.contains(event)) {
log.info("图片 {} 已存在相同缓存任务, 跳过.", event.getStoreFile().getName());
log.debug("图片 {} 已存在相同缓存任务, 跳过.", event.getStoreFile().getName());
return;
} else {
cacheQueue.add(event);
@ -39,7 +39,7 @@ public class ImageCacheHandler implements EventHandler {
try {
log.info("图片 {} Event正在进行...({})", event.getStoreFile().getName(), Integer.toHexString(event.hashCode()));
File storeFile = event.getStoreFile();
log.info("正在缓存图片 {} (Path: {})", storeFile.getName(), storeFile.getAbsolutePath());
log.debug("正在缓存图片 {} (Path: {})", storeFile.getName(), storeFile.getAbsolutePath());
try {
if(!storeFile.exists() && !storeFile.createNewFile()) {
log.error("无法创建文件(Path: {})", storeFile.getAbsolutePath());
@ -64,7 +64,7 @@ public class ImageCacheHandler implements EventHandler {
return;
}
log.info("正在下载...(Content-Length: {}KB)", response.getEntity().getContentLength() / 1024);
log.debug("正在下载...(Content-Length: {}KB)", response.getEntity().getContentLength() / 1024);
try(FileOutputStream fos = new FileOutputStream(storeFile)) {
IOUtils.copy(response.getEntity().getContent(), fos);
} catch (IOException e) {

View File

@ -36,7 +36,7 @@ public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
if(jedisPool.isClosed()) {
throw new IllegalStateException("JedisPool is closed");
}
log = LoggerFactory.getLogger(this.getClass().getSimpleName() + "@" + Integer.toHexString(jedisPool.hashCode()));
log = LoggerFactory.getLogger(this.getClass().getName() + "@" + Integer.toHexString(jedisPool.hashCode()));
if(!Strings.isNullOrEmpty(keyPrefix)) {
this.keyPrefix = keyPrefix.endsWith(".") ? keyPrefix : keyPrefix + ".";
} else {
@ -82,9 +82,7 @@ public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
@Override
public boolean clear() {
try (Jedis jedis = jedisPool.getResource()) {
String result = jedis.flushDB();
log.info("flushDB返回结果: {}", result);
return true;
return jedis.flushDB().equalsIgnoreCase("ok");
}
}
@ -158,13 +156,4 @@ public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
return false;
}
/**
* 替换原本的分隔符(.)为(:).<br/>
* 即将启用
* @param key 要处理的键名
* @return 处理后的键名
*/
public static String replaceKey(String key) {
return key.replaceAll("\\.", ":");
}
}

View File

@ -5,17 +5,18 @@ import com.google.common.base.Throwables;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import net.lamgc.cgj.bot.BotAdminCommandProcess;
import net.lamgc.cgj.bot.BotCommandProcess;
import net.lamgc.cgj.bot.MessageEventExecutionDebugger;
import net.lamgc.cgj.bot.SettingProperties;
import net.lamgc.cgj.util.DateParser;
import net.lamgc.cgj.util.PagesQualityParser;
import net.lamgc.cgj.util.TimeLimitThreadPoolExecutor;
import net.lamgc.utils.base.runner.ArgumentsRunner;
import net.lamgc.utils.base.runner.ArgumentsRunnerConfig;
import net.lamgc.utils.base.runner.exception.DeveloperRunnerException;
import net.lamgc.utils.base.runner.exception.NoSuchCommandException;
import net.lamgc.utils.base.runner.exception.ParameterNoFoundException;
import net.lamgc.utils.event.EventExecutor;
import net.lamgc.utils.event.EventHandler;
import net.lamgc.utils.event.*;
import net.lamgc.utils.event.EventObject;
import net.lamgc.utils.event.EventUncaughtExceptionHandler;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.JedisPool;
@ -23,12 +24,8 @@ import redis.clients.jedis.JedisPool;
import java.lang.reflect.Method;
import java.net.URI;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Date;
import java.util.Objects;
import java.util.*;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.regex.Matcher;
@ -42,7 +39,7 @@ public class BotEventHandler implements EventHandler {
private final ArgumentsRunner processRunner;
private final ArgumentsRunner adminRunner;
private final static Logger log = LoggerFactory.getLogger("BotEventHandler");
private final static Logger log = LoggerFactory.getLogger(BotEventHandler.class.getName());
/**
* 所有缓存共用的JedisPool
@ -53,9 +50,10 @@ public class BotEventHandler implements EventHandler {
/**
* 消息事件执行器
*/
public final static EventExecutor executor = new EventExecutor(new ThreadPoolExecutor(
(int) Math.ceil(Runtime.getRuntime().availableProcessors() / 2F),
Runtime.getRuntime().availableProcessors(),
private final static EventExecutor executor = new EventExecutor(new TimeLimitThreadPoolExecutor(
0,
Math.max(Runtime.getRuntime().availableProcessors(), 4),
Math.max(Math.max(Runtime.getRuntime().availableProcessors() * 2, 4), 32),
30L,
TimeUnit.SECONDS,
new LinkedBlockingQueue<>(1536),
@ -127,6 +125,29 @@ public class BotEventHandler implements EventHandler {
BotCommandProcess.initialize();
}
/**
* 投递消息事件
* @param event 事件对象
*/
@NotAccepted
public static void executeMessageEvent(MessageEvent event) {
String debuggerName = SettingProperties.getProperty(0, "debug.debugger");
if(!event.getMessage().startsWith(ADMIN_COMMAND_PREFIX) &&
!Strings.isNullOrEmpty(debuggerName)) {
try {
MessageEventExecutionDebugger debugger = MessageEventExecutionDebugger.valueOf(debuggerName.toUpperCase());
debugger.debugger.accept(executor, event, SettingProperties.getProperties(SettingProperties.GLOBAL),
MessageEventExecutionDebugger.getDebuggerLogger(debugger));
} catch(IllegalArgumentException e) {
log.warn("未找到指定调试器: '{}'", debuggerName);
} catch (Exception e) {
log.error("事件调试处理时发生异常", e);
}
} else {
BotEventHandler.executor.executor(event);
}
}
/**
* 以事件形式处理消息事件
* @param event 消息事件对象
@ -141,7 +162,7 @@ public class BotEventHandler implements EventHandler {
Pattern pattern = Pattern.compile("/\\s*(\".+?\"|[^:\\s])+((\\s*:\\s*(\".+?\"|[^\\s])+)|)|(\".+?\"|[^\"\\s])+");
Matcher matcher = pattern.matcher(Strings.nullToEmpty(msg));
ArrayList<String> argsList = new ArrayList<>();
List<String> argsList = new ArrayList<>();
while (matcher.find()) {
String arg = matcher.group();
int startIndex = 0;
@ -163,13 +184,20 @@ public class BotEventHandler implements EventHandler {
String[] args = new String[argsList.size()];
argsList.toArray(args);
log.debug("传入参数: {}", Arrays.toString(args));
argsList.add("-$fromGroup");
argsList.add(String.valueOf(event.getFromGroup()));
argsList.add("-$fromQQ");
argsList.add(String.valueOf(event.getFromQQ()));
args = Arrays.copyOf(args, args.length + 4);
argsList.toArray(args);
log.info("正在处理命令...");
long time = System.currentTimeMillis();
Object result;
try {
if(msg.toLowerCase().startsWith(ADMIN_COMMAND_PREFIX)) {
if(!String.valueOf(event.getFromQQ()).equals(BotCommandProcess.globalProp.getProperty("admin.adminId"))) {
if(!String.valueOf(event.getFromQQ())
.equals(SettingProperties.getProperty(0, "admin.adminId"))) {
result = "你没有执行该命令的权限!";
} else {
result = adminRunner.run(args.length <= 1 ? new String[0] : Arrays.copyOfRange(args, 1, args.length));
@ -182,10 +210,15 @@ public class BotEventHandler implements EventHandler {
} catch(ParameterNoFoundException e) {
result = "命令缺少参数: " + e.getParameterName();
} catch(DeveloperRunnerException e) {
log.error("执行命令时发生异常", e);
result = "命令执行时发生错误,无法完成!";
if (!(e.getCause() instanceof InterruptedException)) {
log.error("执行命令时发生异常", e);
result = "色图姬在执行命令时遇到了一个错误!";
} else {
log.error("命令执行超时, 终止执行.");
result = "色图姬发现这个命令的处理时间太久了!所以打断了这个命令。";
}
}
log.info("命令处理完成.(耗时: {}ms)", System.currentTimeMillis() - time);
long processTime = System.currentTimeMillis() - time;
if(Objects.requireNonNull(result) instanceof String) {
try {
event.sendMessage((String) result);
@ -193,7 +226,10 @@ public class BotEventHandler implements EventHandler {
log.error("发送消息时发生异常", e);
}
}
log.info("命令反馈完成.(耗时: {}ms)", System.currentTimeMillis() - time);
long totalTime = System.currentTimeMillis() - time;
log.info("命令反馈完成.(事件耗时: {}ms, P: {}%({}ms), R: {}%({}ms))", totalTime,
String.format("%.3f", ((double) processTime / (double)totalTime) * 100F), processTime,
String.format("%.3f", ((double) (totalTime - processTime) / (double)totalTime) * 100F), totalTime - processTime);
}
/**

View File

@ -56,7 +56,7 @@ public abstract class MessageEvent implements EventObject, MessageSender {
@Override
public String toString() {
return "MessageEvent{" +
return this.getClass().getSimpleName() + "{" +
"fromGroup=" + getFromGroup() +
", fromQQ=" + getFromQQ() +
", message='" + getMessage() + '\'' +

View File

@ -5,6 +5,28 @@ package net.lamgc.cgj.bot.event;
*/
public class VirtualLoadMessageEvent extends MessageEvent {
/**
* 将任意消息事件转换为假负载消息事件.
* <p>转换之后, 除了fromGroup, fromQQ, message外其他信息不会保留</p>
* @param event 待转换的消息事件
* @param inheritImpl 是否继承除 sendMessage 外的其他 MessageEvent 实现
* @return 转换后的消息事件
*/
public static VirtualLoadMessageEvent toVirtualLoadMessageEvent(MessageEvent event, boolean inheritImpl) {
if(event instanceof VirtualLoadMessageEvent) {
return (VirtualLoadMessageEvent) event;
} else if(!inheritImpl) {
return new VirtualLoadMessageEvent(event.getFromGroup(), event.getFromQQ(), event.getMessage());
} else {
return new VirtualLoadMessageEvent(event.getFromGroup(), event.getFromQQ(), event.getMessage()) {
@Override
public String getImageUrl(String image) {
return event.getImageUrl(image);
}
};
}
}
public VirtualLoadMessageEvent(long fromGroup, long fromQQ, String message) {
super(fromGroup, fromQQ, message);
}

View File

@ -1,4 +1,4 @@
package net.lamgc.cgj.bot;
package net.lamgc.cgj.bot.framework.coolq;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import net.lz1998.cq.CQGlobal;

View File

@ -1,7 +1,7 @@
package net.lamgc.cgj.bot;
package net.lamgc.cgj.bot.framework.coolq;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.event.SpringCQMessageEvent;
import net.lamgc.cgj.bot.framework.coolq.message.SpringCQMessageEvent;
import net.lamgc.utils.event.EventHandler;
import net.lz1998.cq.event.message.CQDiscussMessageEvent;
import net.lz1998.cq.event.message.CQGroupMessageEvent;
@ -18,25 +18,22 @@ public class CQPluginMain extends CQPlugin implements EventHandler {
public CQPluginMain() {
// TODO(LamGC, 2020.04.21): SpringCQ无法适配MessageSenderBuilder
BotEventHandler.preLoad();
LoggerFactory.getLogger(this.toString())
LoggerFactory.getLogger(CQPluginMain.class.getName())
.info("BotEventHandler.COMMAND_PREFIX = {}", BotEventHandler.COMMAND_PREFIX);
}
@Override
public int onPrivateMessage(CoolQ cq, CQPrivateMessageEvent event) {
//log.info("私聊消息到达: 发送者[{}], 消息内容: {}", event.getSender().getUserId(), event.getMessage());
return processMessage(cq, event);
}
@Override
public int onGroupMessage(CoolQ cq, CQGroupMessageEvent event) {
//log.info("群消息到达: 群[{}], 发送者[{}], 消息内容: {}", event.getGroupId(), event.getSender().getUserId(), event.getMessage());
return processMessage(cq, event);
}
@Override
public int onDiscussMessage(CoolQ cq, CQDiscussMessageEvent event) {
//log.info("讨论组消息到达: 群[{}], 发送者[{}], 消息内容: {}", event.getDiscussId(), event.getSender().getUserId(), event.getMessage());
return processMessage(cq, event);
}
@ -44,7 +41,7 @@ public class CQPluginMain extends CQPlugin implements EventHandler {
if(!BotEventHandler.match(event.getMessage())) {
return MESSAGE_IGNORE;
}
BotEventHandler.executor.executor(new SpringCQMessageEvent(cq, event));
BotEventHandler.executeMessageEvent(new SpringCQMessageEvent(cq, event));
return MESSAGE_BLOCK;
}

View File

@ -1,9 +1,9 @@
package net.lamgc.cgj.bot.event;
package net.lamgc.cgj.bot.framework.coolq.message;
import net.lamgc.cgj.bot.BotCode;
import net.lamgc.cgj.bot.event.MessageEvent;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lamgc.cgj.bot.message.SpringCQMessageSender;
import net.lz1998.cq.event.message.CQDiscussMessageEvent;
import net.lz1998.cq.event.message.CQGroupMessageEvent;
import net.lz1998.cq.event.message.CQMessageEvent;

View File

@ -1,5 +1,7 @@
package net.lamgc.cgj.bot.message;
package net.lamgc.cgj.bot.framework.coolq.message;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lz1998.cq.robot.CoolQ;
public class SpringCQMessageSender implements MessageSender {

View File

@ -1,5 +1,8 @@
package net.lamgc.cgj.bot.message;
package net.lamgc.cgj.bot.framework.coolq.message;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lz1998.cq.robot.CoolQ;
public class SpringCQMessageSenderFactory implements MessageSenderFactory {

View File

@ -1,14 +1,14 @@
package net.lamgc.cgj.bot;
package net.lamgc.cgj.bot.framework.mirai;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.event.MiraiMessageEvent;
import net.lamgc.cgj.bot.framework.mirai.message.MiraiMessageEvent;
import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import net.lamgc.cgj.bot.message.MiraiMessageSenderFactory;
import net.lamgc.cgj.bot.framework.mirai.message.MiraiMessageSenderFactory;
import net.mamoe.mirai.Bot;
import net.mamoe.mirai.BotFactoryJvm;
import net.mamoe.mirai.japt.Events;
import net.mamoe.mirai.message.FriendMessage;
import net.mamoe.mirai.message.GroupMessage;
import net.mamoe.mirai.qqandroid.QQAndroid;
import net.mamoe.mirai.utils.BotConfiguration;
import org.apache.commons.net.util.Base64;
import org.slf4j.Logger;
@ -19,7 +19,7 @@ import java.util.Properties;
public class MiraiMain implements Closeable {
private final Logger log = LoggerFactory.getLogger(this.toString());
private final Logger log = LoggerFactory.getLogger(MiraiMain.class.getName());
private Bot bot;
@ -34,7 +34,7 @@ public class MiraiMain implements Closeable {
return;
}
File botPropFile = new File("./bot.properties");
File botPropFile = new File(System.getProperty("cgj.botDataDir"), "./bot.properties");
try (Reader reader = new BufferedReader(new FileReader(botPropFile))) {
botProperties.load(reader);
} catch (IOException e) {
@ -42,9 +42,9 @@ public class MiraiMain implements Closeable {
return;
}
bot = QQAndroid.INSTANCE.newBot(Long.parseLong(botProperties.getProperty("bot.qq", "0")), Base64.decodeBase64(botProperties.getProperty("bot.password", "")), new BotConfiguration());
Events.subscribeAlways(GroupMessage.class, (msg) -> BotEventHandler.executor.executor(new MiraiMessageEvent(msg)));
Events.subscribeAlways(FriendMessage.class, (msg) -> BotEventHandler.executor.executor(new MiraiMessageEvent(msg)));
bot = BotFactoryJvm.newBot(Long.parseLong(botProperties.getProperty("bot.qq", "0")), Base64.decodeBase64(botProperties.getProperty("bot.password", "")), new BotConfiguration());
Events.subscribeAlways(GroupMessage.class, (msg) -> BotEventHandler.executeMessageEvent(new MiraiMessageEvent(msg)));
Events.subscribeAlways(FriendMessage.class, (msg) -> BotEventHandler.executeMessageEvent(new MiraiMessageEvent(msg)));
bot.login();
MessageSenderBuilder.setCurrentMessageSenderFactory(new MiraiMessageSenderFactory(bot));
BotEventHandler.preLoad();

View File

@ -1,8 +1,8 @@
package net.lamgc.cgj.bot.event;
package net.lamgc.cgj.bot.framework.mirai.message;
import net.lamgc.cgj.bot.event.MessageEvent;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lamgc.cgj.bot.message.MiraiMessageSender;
import net.mamoe.mirai.message.ContactMessage;
import net.mamoe.mirai.message.GroupMessage;
import net.mamoe.mirai.message.data.MessageUtils;
@ -16,7 +16,7 @@ public class MiraiMessageEvent extends MessageEvent {
public MiraiMessageEvent(ContactMessage message) {
super(message instanceof GroupMessage ? ((GroupMessage) message).getGroup().getId() : 0,
message.getSender().getId(), message.getMessage().toString());
message.getSender().getId(), message.getMessage().contentToString());
this.messageObject = Objects.requireNonNull(message);
if(message instanceof GroupMessage) {
messageSender = new MiraiMessageSender(((GroupMessage) message).getGroup(), MessageSource.Group);

View File

@ -1,4 +1,4 @@
package net.lamgc.cgj.bot.message;
package net.lamgc.cgj.bot.framework.mirai.message;
import com.google.common.base.Strings;
import net.lamgc.cgj.bot.BotCode;
@ -7,11 +7,13 @@ import net.lamgc.cgj.bot.cache.HotDataCacheStore;
import net.lamgc.cgj.bot.cache.LocalHashCacheStore;
import net.lamgc.cgj.bot.cache.StringRedisCacheStore;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.Bot;
import net.mamoe.mirai.contact.Contact;
import net.mamoe.mirai.message.data.CombinedMessage;
import net.mamoe.mirai.message.data.Image;
import net.mamoe.mirai.message.data.Message;
import net.mamoe.mirai.message.data.MessageChain;
import net.mamoe.mirai.message.data.MessageUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@ -27,7 +29,7 @@ public class MiraiMessageSender implements MessageSender {
private final Contact member;
private final MessageSource source;
private final static Logger log = LoggerFactory.getLogger("MiraiMessageSender");
private final static Logger log = LoggerFactory.getLogger(MiraiMessageSender.class.getName());
private final static CacheStore<String> imageIdCache = new HotDataCacheStore<>(
new StringRedisCacheStore(BotEventHandler.redisServer, "mirai.imageId"),
new LocalHashCacheStore<>(),
@ -78,7 +80,7 @@ public class MiraiMessageSender implements MessageSender {
.replaceAll("&38", "&")
.split("\\|");
CombinedMessage chain = MessageUtils.newChain().plus("");
MessageChain messages = MessageUtils.newChain().plus("");
int codeIndex = 0;
for(String text : texts) {
if(text.equals("{BotCode}")) {
@ -89,25 +91,32 @@ public class MiraiMessageSender implements MessageSender {
log.warn("解析待发送消息内的BotCode时发生异常, 请检查错误格式BotCode的来源并尽快排错!", e);
continue;
}
chain = chain.plus(processBotCode(code));
messages = messages.plus(processBotCode(code));
} else {
chain = chain.plus(text);
messages = messages.plus(text);
}
}
return chain;
return messages;
}
private Message processBotCode(BotCode code) {
switch(code.getFunctionName().toLowerCase()) {
case "image":
Image img;
if(code.containsParameter("id")) {
return MessageUtils.newImage(code.getParameter("id"));
img = MessageUtils.newImage(code.getParameter("id"));
} else if(code.containsParameter("absolutePath")) {
return uploadImage(code);
img = uploadImage(code);
} else {
return MessageUtils.newChain("(参数不存在)");
}
if(Strings.nullToEmpty(code.getParameter("flashImage"))
.equalsIgnoreCase("true")) {
return MessageUtils.flash(img);
} else {
return img;
}
default:
log.warn("解析到不支持的BotCode: {}", code);
return MessageUtils.newChain("(不支持的BotCode)");
@ -138,7 +147,7 @@ public class MiraiMessageSender implements MessageSender {
log.debug("imageName [{}] 缓存失效或强制更新, 正在更新缓存...", imageName);
image = uploadImage0(new File(absolutePath));
String cacheExpireAt;
long expireTime = 0;
long expireTime = 864000000; // 10d
if(!Strings.isNullOrEmpty(cacheExpireAt = code.getParameter("cacheExpireAt"))) {
try {
expireTime = Integer.parseInt(cacheExpireAt);
@ -147,7 +156,7 @@ public class MiraiMessageSender implements MessageSender {
}
}
imageIdCache.update(imageName, image.getImageId(), expireTime);
log.info("imageName [{}] 缓存更新完成.(有效时间: {})", imageName, expireTime);
log.debug("imageName [{}] 缓存更新完成.(有效时间: {})", imageName, expireTime);
} else {
log.debug("ImageName: [{}] 缓存命中.", imageName);
}

View File

@ -1,5 +1,8 @@
package net.lamgc.cgj.bot.message;
package net.lamgc.cgj.bot.framework.mirai.message;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.Bot;
public class MiraiMessageSenderFactory implements MessageSenderFactory {

View File

@ -0,0 +1,33 @@
package net.lamgc.cgj.pixiv;
public interface AdultContentDetector {
/**
* 检查某一作品的成人内容判断指数
* @param illustId 作品Id
* @param isUgoira 是否为动图
* @param pageIndex 指定页数, 设为0或负数则视为单页面作品
* @return 返回成人作品判断指数(0 ~ 1), 需按照情况设置阀值.
*/
double detect(int illustId, boolean isUgoira, int pageIndex) throws Exception;
/**
* 检查某一作品是否为成人内容
* @param illustId 作品Id
* @param isUgoira 是否为动图
* @param pageIndex 指定页数, 设为0或负数则视为单页面作品
* @return 如果为true则为成人作品, 该方法将由检测器决定如何为成人作品.
*/
boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex) throws Exception;
/**
* 检查某一作品是否为成人内容
* @param illustId 作品Id
* @param isUgoira 是否为动图
* @param pageIndex 指定页数, 设为0或负数则视为单页面作品
* @param threshold 指数阀值, 当等于或大于该阀值时返回true.
* @return 如果为true则为成人作品, 该方法将由 threshold 参数决定是否为成人作品.
*/
boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex, double threshold) throws Exception;
}

View File

@ -1,69 +0,0 @@
package net.lamgc.cgj.pixiv;
/**
* 插图(集)的信息
*/
public class IllustInfo{
/**
* 插图(集)ID
*/
public final int illustID;
/**
* 第几页
*/
public final int page;
/**
* 插图标题
*/
public final String title;
/**
* 插图说明
*/
public final String description;
/**
* 插图标签
*/
public final String[] tags;
/**
* 插图图片长度
*/
public final int width;
/**
* 插图图片高度
*/
public final int height;
/**
* 作者名
*/
public final String authorName;
/**
* 作者用户ID
*/
public final int authorUserID;
public IllustInfo(int illustID, String title, String description, String[] tags, int width, int height, String authorName, int authorUserID){
this(illustID, 0, title, description, tags, width, height, authorName, authorUserID);
}
public IllustInfo(int illustID, int p, String title, String description, String[] tags, int width, int height, String authorName, int authorUserID){
this.illustID = illustID;
this.page = p;
this.title = title;
this.description = description;
this.tags = tags;
this.width = width;
this.height = height;
this.authorName = authorName;
this.authorUserID = authorUserID;
}
}

View File

@ -0,0 +1,65 @@
package net.lamgc.cgj.pixiv;
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
import java.io.IOException;
/**
* 使用ModerateContent服务开发的检测器.<br/>
* ModerateContent: www.moderatecontent.com
*/
public class ModerateContentDetector implements AdultContentDetector {
private final static HttpClient httpClient = HttpClientBuilder.create().build();
private final static Gson gson = new Gson();
private final String requestUrl;
private final static String API_URL = "https://www.moderatecontent.com/api/v2?key={key}&url=https://pixiv.cat/";
/**
* 创建一个使用ModerateContent鉴黄服务的检测器
* @param apiKey API密钥
*/
public ModerateContentDetector(String apiKey) {
requestUrl = API_URL.replace("{key}", apiKey);
}
private JsonObject accessInterface(int illustId, boolean isUgoira, int pageIndex) throws IOException {
HttpResponse response;
if(pageIndex <= 0) {
response = httpClient.execute(new HttpGet(requestUrl + illustId + (isUgoira ? ".gif" : ".jpg")));
} else {
response = httpClient.execute(new HttpGet(requestUrl + illustId + "-" + pageIndex + (isUgoira ? ".gif" : ".jpg")));
}
if(response.getStatusLine().getStatusCode() != 200) {
throw new IOException("Http response error: " + response.getStatusLine());
}
JsonObject result = gson.fromJson(EntityUtils.toString(response.getEntity()), JsonObject.class);
if (result.get("error_code").getAsInt() != 0) {
throw new IOException("Interface result error: " + (result.has("error") ? result.get("error").getAsString() : "(error message is empty)"));
}
return result;
}
@Override
public double detect(int illustId, boolean isUgoira, int pageIndex) throws IOException {
return accessInterface(illustId, isUgoira, pageIndex).getAsJsonObject("predictions").get("adult").getAsDouble();
}
@Override
public boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex) throws IOException {
return accessInterface(illustId, isUgoira, pageIndex).get("rating_index").getAsInt() == 3;
}
@Override
public boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex, double threshold) throws IOException {
return detect(illustId, isUgoira, pageIndex) >= threshold;
}
}

View File

@ -1,267 +0,0 @@
package net.lamgc.cgj.pixiv;
import com.google.gson.JsonArray;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.conn.ConnectionPoolTimeoutException;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.*;
public class Pixiv {
/**
* illust Link
*/
public final static String ATTR_LINK = "link";
/**
* illust Id
*/
public final static String ATTR_ILLUST_ID = "illustId";
/**
* illust Title
*/
public final static String ATTR_TITLE = "title";
/**
* illust Author Name
*/
public final static String ATTR_AUTHOR_NAME = "authorName";
/**
* illust Author UserID
*/
public final static String ATTR_AUTHOR_ID = "authorId";
private final HttpClient httpClient;
public Pixiv(HttpClient client){
this.httpClient = client;
}
/**
* 使用帐号密码登录Pixiv
* @param PixivID Pixiv账户登录名
* @param Password Pixiv帐号密码
* @throws IOException 当登录连接出现异常时抛出
* @deprecated {@link PixivSession#Login(String, String)} 已经废除, 故本方法不可用
*/
public Pixiv(String PixivID, String Password) throws IOException {
this(PixivID, Password, null);
}
/**
* 使用帐号密码登录Pixiv
* @param PixivID Pixiv账户登录名
* @param Password Pixiv帐号密码
* @param proxy 代理设置
* @throws IOException 当登录连接出现异常时抛出
* @deprecated {@link PixivSession#Login(String, String)} 已经废除, 故本方法不可用
*/
public Pixiv(String PixivID, String Password, HttpHost proxy) throws IOException {
PixivSession pixivSession = new PixivSession(proxy, null);
if(pixivSession.Login(PixivID, Password)){
System.out.println("P站登录成功!");
}else{
System.out.println("P站登录失败!错误信息: " + pixivSession.getErrMsg());
throw new RuntimeException(pixivSession.getErrMsg());
}
//httpClient = pixivSession.getHttpClient();
httpClient = HttpClientBuilder.create()
.setDefaultCookieStore(pixivSession.getCookieStore())
.build();
}
/**
* 获取首页推荐列表
* @return 首页推荐列表, 一个Map对应一个推荐项, 使用<code>ATTR_</code>开头常量访问即可
* @throws IOException
*/
public List<Map<String, String>> getRecommend() throws IOException {
HttpGet getRequest = new HttpGet(PixivURL.PIXIV_INDEX_URL);
HttpResponse response = httpClient.execute(getRequest);
String pageAsXML = EntityUtils.toString(response.getEntity(),"utf-8");
//获取推荐图列表(li)
//System.out.println(pageAsXML);
Document document = Jsoup.parse(pageAsXML);
List<String> links = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-thumbnail-link").eachAttr("href");
List<String> illustId = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-thumbnail-link").eachAttr("data-gtm-recommend-illust-id");
List<String> title = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-title>h1").eachAttr("title");
List<String> authorName = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-user-name").eachText();
List<String> authorId = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-user-name").eachAttr("data-user_id");
List<Map<String, String>> recommendList = new ArrayList<>();
for(int i = 0; i < links.size(); i++){
//System.out.println(links.get(i));
Map<String, String> info = new HashMap<>();
info.put(ATTR_LINK, PixivURL.PIXIV_INDEX_URL + links.get(i));
info.put(ATTR_ILLUST_ID, illustId.get(i));
info.put(ATTR_TITLE, title.get(i));
info.put(ATTR_AUTHOR_NAME, authorName.get(i));
info.put(ATTR_AUTHOR_ID, authorId.get(i));
recommendList.add(info);
}
return recommendList;
}
public String[] getAllDownloadLink(int illustID) throws IOException {
HttpGet illustPage = new HttpGet(PixivURL.PIXIV_ILLUST_API_URL.replaceAll("\\{illustId}", String.valueOf(illustID)));
HttpResponse response = httpClient.execute(illustPage);
String pageAsXML = EntityUtils.toString(response.getEntity(),"utf-8");
//System.out.println(pageAsXML);
JsonObject resultObj = (JsonObject) new JsonParser().parse(pageAsXML);
if(!resultObj.get("error").getAsBoolean()){
JsonArray bodyArray = resultObj.get("body").getAsJsonArray();
int length = bodyArray.size();
String[] result = new String[length];
for(int i = 0; i < length; i++){
JsonObject childObj = bodyArray.get(i).getAsJsonObject();
result[i] = childObj.get("urls").getAsJsonObject().get("original").getAsString();
}
return result;
}else{
return null;
}
}
/**
* 下载P站图片
* @param illustID 插图ID
* @return 成功返回图片输入流,失败或为多图则返回null
*/
public InputStream[] downloadIllustImage(int illustID) throws IOException {
String[] links = getAllDownloadLink(illustID);
List<InputStream> inputStreamList = new ArrayList<>();
int count = 1;
boolean retry = false;
for(int i = 0; i < links.length; i++){
try {
long sleepTime = (new Random().nextInt(4) + 2) * 1000;
System.out.println("nextTime: " + (float)(sleepTime / 1000));
Thread.sleep(sleepTime);
} catch (InterruptedException ignored) {}
String link = links[i];
System.out.print("page:" + count++ + "/" + links.length + " ...");
HttpGet imgGet = new HttpGet(link);
//关键!如果不加上Referer的话,会返回403
imgGet.setHeader("Referer", PixivURL.PIXIV_ILLUST_MEDIUM_URL.replaceAll("\\{illustId}", String.valueOf(illustID)));
RequestConfig config = RequestConfig.custom()
.setConnectTimeout(20 * 1000)
.setConnectionRequestTimeout(20 * 1000)
.setSocketTimeout(30 * 1000)
.build();
imgGet.setConfig(config);
HttpResponse response;
try {
response = httpClient.execute(imgGet);
}catch(ConnectionPoolTimeoutException e){
if(retry){
retry = false;
System.out.println("获取失败,跳过...");
continue;
}
System.out.println("连接超时,重新获取...");
retry = true;
i--;
continue;
}
retry = false;
ByteArrayOutputStream cacheOutputStream = new ByteArrayOutputStream((int)response.getEntity().getContentLength());
InputStream content = response.getEntity().getContent();
int readLength;
byte[] cache = new byte[4096];
while((readLength = content.read(cache)) != -1){
cacheOutputStream.write(cache, 0, readLength);
}
byte[] data = cacheOutputStream.toByteArray();
//System.out.println("读到数据: " + data.length);
inputStreamList.add(new ByteArrayInputStream(data));
System.out.println("done!length: " + response.getEntity().getContentLength() + ")");
}
return inputStreamList.toArray(new InputStream[0]);
}
/**
* 下载P站图片
* @param illustID 插图ID
* @return 成功返回图片输入流,失败或为多图则返回null
*/
public InputStream downloadIllustImages(int illustID){
throw new UnsupportedOperationException();
}
/**
* 通过解析插图详情页获取
* - 插图标题
* - 插图作者(及其UserId)
* - 插图上传时间
* - 插图标签(原始标签)
* ...
* @return 成功返回IllustInfo对象,失败返回null
*/
public IllustInfo[] getIllustInfo(int[] illustIDs) throws IOException {
//获取Api
HttpGet apiRequest = new HttpGet(PixivURL.getPixivIllustInfoAPI(illustIDs));
HttpResponse response = httpClient.execute(apiRequest);
String resultText = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
System.out.println(resultText);
JsonObject resultObj = ((JsonObject) new JsonParser().parse(resultText));
if(resultObj.get("error").getAsBoolean()){
System.err.println("获取失败!");
return null;
}
List<IllustInfo> illustInfoList = new ArrayList<>();
JsonArray illustArray = resultObj.get("body").getAsJsonObject().get("illusts").getAsJsonArray();
illustArray.forEach(jsonElement -> {
JsonObject illustInfoObj = jsonElement.getAsJsonObject();
JsonArray tagsArray = illustInfoObj.get("tags").getAsJsonArray();
String[] tags = new String[tagsArray.size()];
for(int i = 0; i < tags.length; i++){
tags[i] = tagsArray.get(i).getAsString();
}
//TODO: 通过不需要作者id就能获取图片信息的api无法获取图片尺寸
IllustInfo illustInfo = new IllustInfo(
illustInfoObj.get("workId").getAsInt(),
illustInfoObj.get("title").getAsString(),
null,
tags,
-1,
-1,
illustInfoObj.get("userName").getAsString(),
illustInfoObj.get("userId").getAsInt()
);
});
return null;
}
/**
* 获取指定用户的所有插画
*/
public void getUserAllIllustTest() {
}
}

View File

@ -33,7 +33,7 @@ import java.util.function.BiConsumer;
public class PixivDownload {
private final static Logger log = LoggerFactory.getLogger("PixivDownload");
private final static Logger log = LoggerFactory.getLogger(PixivDownload.class.getName());
private final HttpClient httpClient;
@ -285,7 +285,8 @@ public class PixivDownload {
}
/**
* 获取排行榜
* 获取排行榜.
* <p>注意: 如果范围实际上没超出, 但返回排行榜不足, 会导致与实际请求的数量不符, 需要检查</p>
* @param contentType 排行榜类型
* @param mode 排行榜模式
* @param time 查询时间
@ -299,6 +300,7 @@ public class PixivDownload {
*/
public List<JsonObject> getRanking(PixivURL.RankingContentType contentType, PixivURL.RankingMode mode,
Date time, int rankStart, int range) throws IOException {
Objects.requireNonNull(time);
if(!Objects.requireNonNull(contentType).isSupportedMode(Objects.requireNonNull(mode))) {
throw new IllegalArgumentException("ContentType不支持指定的RankingMode: ContentType: " + contentType.name() + ", Mode: " + mode.name());
} else if(rankStart <= 0) {
@ -315,18 +317,24 @@ public class PixivDownload {
int count = 0;
Gson gson = new Gson();
ArrayList<JsonObject> results = new ArrayList<>(range);
for (int pageIndex = startPages; pageIndex <= endPages && count < range; pageIndex++) {
boolean canNext = true;
for (int pageIndex = startPages; canNext && pageIndex <= endPages && count < range; pageIndex++) {
HttpGet request = createHttpGetRequest(PixivURL.getRankingLink(contentType, mode, time, pageIndex, true));
log.debug("RequestUri: {}", request.getURI());
HttpResponse response = httpClient.execute(request);
String responseBody = EntityUtils.toString(response.getEntity());
log.debug("ResponseBody: {}", responseBody);
if(response.getStatusLine().getStatusCode() != 200) {
throw new IOException("Http Response Error: " + response.getStatusLine());
throw new IOException("Http Response Error: '" + response.getStatusLine() + "', ResponseBody: '" + responseBody + '\'');
}
JsonArray resultArray = gson.fromJson(EntityUtils.toString(response.getEntity()), JsonObject.class).getAsJsonArray("contents");
JsonObject resultObject = gson.fromJson(responseBody, JsonObject.class);
canNext = resultObject.get("next").getAsJsonPrimitive().isNumber();
JsonArray resultArray = resultObject.getAsJsonArray("contents");
for (int resultIndex = startIndex; resultIndex < resultArray.size() && count < range; resultIndex++, count++) {
results.add(resultArray.get(resultIndex).getAsJsonObject());
}
// 重置索引
startIndex = 0;
}
@ -430,13 +438,14 @@ public class PixivDownload {
THUMB_MINI
}
/**
* 获取帐号所有的收藏插画,并以输入流形式提供
* @return 获取所有链接的InputStream, 请注意关闭InputStream
* @throws IOException 当获取时发生异常则直接抛出
* @deprecated 该方法可能会导致已经打开的InputStream超时, 使图片获取失败,
* 请直接使用{@linkplain #getCollectionAsInputStream(PageQuality, BiConsumer)}
*/
@Deprecated
public Set<Map.Entry<String, InputStream>> getCollectionAsInputStream(PageQuality quality) throws IOException {
HashSet<Map.Entry<String, InputStream>> illustInputStreamSet = new HashSet<>();
getCollectionAsInputStream(quality, (link, inputStream) -> illustInputStreamSet.add(new AbstractMap.SimpleEntry<>(link, inputStream)));
@ -597,12 +606,29 @@ public class PixivDownload {
return null;
}
/**
* 通过预加载数据获取作品类型
* @param illustId 作品Id
* @param preLoadDataObject 预加载数据(IllustInfo也可以)
* @return 如果存在illustType属性, 则返回对应项, 如没有, 或数据内不存在指定作品id的数据, 返回null
*/
public static PixivIllustType getIllustTypeByPreLoadData(int illustId, JsonObject preLoadDataObject) {
JsonObject illustData;
JsonObject illustData = null;
if(preLoadDataObject.has("illust")) {
illustData = preLoadDataObject.getAsJsonObject("illust").getAsJsonObject(String.valueOf(illustId));
} else if(preLoadDataObject.has(String.valueOf(illustId))) {
illustData = preLoadDataObject.getAsJsonObject(String.valueOf(illustId));
} else if(preLoadDataObject.has("body")) { // 解析IllustInfo
for (JsonElement jsonElement : preLoadDataObject.getAsJsonObject("body").getAsJsonArray("illusts")) {
JsonObject illustInfo = jsonElement.getAsJsonObject();
if (illustInfo.get("illustId").getAsInt() == illustId) {
illustData = illustInfo;
break;
}
}
if(illustData == null) {
return null;
}
} else {
illustData = preLoadDataObject;
}

View File

@ -9,6 +9,13 @@ import java.util.Date;
import java.util.HashSet;
import java.util.Objects;
/**
* Pixiv搜索URL构造器
* <p>该构造器通过分析Pixiv搜索链接可用的参数而开发, 对搜索链接的构造有高度自定义能力.</p>
* @author LamGC
* @see PixivURL#PIXIV_SEARCH_CONTENT_URL
*/
@SuppressWarnings("ALL")
public class PixivSearchBuilder {
private final String content;

View File

@ -1,171 +0,0 @@
package net.lamgc.cgj.pixiv;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import org.apache.http.Header;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.CookieStore;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicHeader;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
public class PixivSession {
public static final String USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36";
/**
* 全登陆过程的关键,
* 保存会话用的cookieStore!
*/
private CookieStore cookieStore = new BasicCookieStore();
/**
* 可以直接使用的HttpClient对象
*/
private HttpClient httpClient;
/**
* 最后一次登录的错误信息
*/
private String errMsg;
public PixivSession(){
this(null);
}
public PixivSession(CookieStore cookieStore){
this(null, cookieStore);
}
/**
* 创建一个Pixiv登录会话
*/
public PixivSession(HttpHost proxy, CookieStore cookieStore) {
if(cookieStore != null){
this.cookieStore = cookieStore;
}
List<Header> defaultHeader = new ArrayList<>();
defaultHeader.add(new BasicHeader("accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"));
defaultHeader.add(new BasicHeader("user-agent", PixivSession.USER_AGENT));
defaultHeader.add(new BasicHeader("accept-encoding", "gzip, deflate, br"));
defaultHeader.add(new BasicHeader("accept-language", "zh-CN,zh;q=0.9"));
/*defaultHeader.add(new BasicHeader("sec-fetch-mode", "navigate"));
defaultHeader.add(new BasicHeader("sec-fetch-site", "same-origin"));
defaultHeader.add(new BasicHeader("upgrade-insecure-requests", "1"));*/
//创建一个Http访问器
httpClient = HttpClients.custom()
.setDefaultCookieStore(cookieStore)
.setDefaultHeaders(defaultHeader)
.setProxy(proxy)
.build();
}
/**
* 程序自行通过帐号密码登录Pixiv.
* @param PixivID Pixiv帐号
* @param Password Pixiv密码
* @return 登录成功返回true
* @throws IOException 当登录抛出异常时返回
* @deprecated Pixiv已经新增Google人机验证, 程序已无法自行登录Pixiv
*/
public boolean Login(String PixivID, String Password) throws IOException {
// 获取登录接口所需的PostKey
String post_key = getPostKey();
HttpPost postRequest = new HttpPost(PixivURL.PIXIV_LOGIN_URL); //https://accounts.pixiv.net/api/login?lang=zh
List<NameValuePair> params = new ArrayList<>();
params.add(new BasicNameValuePair("pixiv_id", PixivID));
params.add(new BasicNameValuePair("password", Password));
params.add(new BasicNameValuePair("post_key", post_key));
//Form编码表单,作为Post的数据
postRequest.setEntity(new UrlEncodedFormEntity(params, StandardCharsets.UTF_8));
//启动访问
HttpResponse response = httpClient.execute(postRequest);
//获取接口返回数据
String httpXML = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
System.out.println(httpXML);
JsonObject responseJson = (JsonObject) new JsonParser().parse(httpXML);
if(!responseJson.get("error").getAsBoolean() && !responseJson.get("body").getAsJsonObject().has("validation_errors")){
errMsg = null;
return true;
}else{
errMsg = responseJson.get("body").getAsJsonObject().get("validation_errors").toString();
//System.err.println("登录失败MSG: " + errMsg);
return false;
}
}
/**
* 登录前准备, 获取PostKey
* @return Post_Key
*/
private String getPostKey() throws IOException {
//创建请求,获取PostKey
HttpGet getRequest = new HttpGet(PixivURL.PIXIV_LOGIN_PAGE_URL);
//设置请求
//getRequest.setConfig(config);
getRequest.setHeader("User-Agent", USER_AGENT);
//启动访问
HttpResponse response = httpClient.execute(getRequest);
//获取网页内容
String pageAsXML = EntityUtils.toString(response.getEntity(),"utf-8");
//创建Http解析器
Document document = Jsoup.parse(pageAsXML);
//获取init-config内容
String init_config = document.getElementById("init-config").val();
//System.out.println(init_config);
//创建Json解析器解析init-config
JsonObject initConfigObj = (JsonObject) new JsonParser().parse(init_config);
//检查是否有postKey
if(!initConfigObj.has("pixivAccount.postKey")){
throw new RuntimeException("postKey获取失败!可能是Pixiv修改了登录过程!");
}
//获取postKey
return initConfigObj.get("pixivAccount.postKey").getAsString();
}
/**
* 获取CookieStore
* @return CookieStore
*/
public CookieStore getCookieStore(){
return cookieStore;
}
/**
* 获取可直接使用的HttpClient对象
* @return 已配置好的HttpClient对象
*/
public HttpClient getHttpClient(){
return this.httpClient;
}
public boolean hasError(){
return errMsg == null;
}
/**
* 获取错误信息
* @return 返回登录错误信息
* @deprecated {@link #Login(String, String)}已经废除, 故本接口废除
*/
public String getErrMsg(){
return errMsg;
}
}

View File

@ -1,16 +0,0 @@
package net.lamgc.cgj.pixiv;
public class PixivTag {
public static PixivTag create(String tagName) {
return null;
}
private PixivTag() {
}
}

View File

@ -9,6 +9,7 @@ import java.util.GregorianCalendar;
/**
* 目前已整理的一些Pixiv接口列表
*/
@SuppressWarnings("unused")
public class PixivURL {
@ -52,10 +53,10 @@ public class PixivURL {
/**
* P站用户插图列表获取API
* <p>所需数据在 body属性内的 illusts(属性名,属性值不重要), manga(多图) pickup(精选)</p>
* 需要替换的文本:
* {userId} - 用户ID
*/
//TODO: 所需数据在 body属性内的 illusts(属性名,属性值不重要), manga(多图) pickup(精选)
//{"error":false,"message":"","body":{"illusts":{"74369837":null,"70990542":null,"70608653":null,"69755191":null,"69729450":null,"69729416":null,"69503608":null,"69288766":null,"69083882":null,"69051458":null,"68484200":null,"68216927":null,"68216866":null,"68192333":null,"67915106":null,"67914932":null,"67854803":null,"67854745":null,"67854670":null,"67787211":null,"67772199":null,"67770637":null,"67754861":null,"67754804":null,"67754726":null,"67740486":null,"67740480":null,"67740450":null,"67740434":null,"67726337":null,"67499196":null,"67499163":null,"67499145":null,"67499111":null,"67499085":null,"67499038":null,"67498987":null,"67473178":null,"66271465":null,"63682753":null,"63682697":null,"59385148":null,"59383265":null,"59383240":null,"59383227":null,"59383173":null},"manga":[],"novels":[],"mangaSeries":[],"novelSeries":[],"pickup":[],"bookmarkCount":{"public":{"illust":1,"novel":0},"private":{"illust":0,"novel":0}}}}
public static final String PIXIV_USER_ILLUST_LIST_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/all";
@ -186,7 +187,7 @@ public class PixivURL {
public static String getRankingLink(RankingContentType contentType, RankingMode mode, Date time, int pageIndex, boolean json){
StringBuilder linkBuilder = new StringBuilder(PIXIV_RANKING_LINK);
linkBuilder.append("mode=").append(mode == null ? RankingMode.MODE_DAILY.modeParam : mode.modeParam);
if(contentType != null && !contentType.equals(RankingContentType.ALL)){
if(contentType != null && !contentType.equals(RankingContentType.TYPE_ALL)){
linkBuilder.append("&content=").append(contentType.typeName);
}
SimpleDateFormat format = new SimpleDateFormat("yyyyMMdd");
@ -278,7 +279,7 @@ public class PixivURL {
* 排名榜类型
*/
public enum RankingContentType{
ALL("", RankingMode.values()),
TYPE_ALL("", RankingMode.values()),
/**
* 插画
* 支持的时间类型: 每天, 每周, 每月, 新人
@ -336,7 +337,7 @@ public class PixivURL {
* @return 如果支持返回true
*/
public boolean isSupportedMode(RankingMode mode) {
return Arrays.binarySearch(supportedMode, mode) != -1;
return Arrays.binarySearch(supportedMode, mode) >= 0;
}
}

View File

@ -26,12 +26,11 @@ import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
/**
* Pixiv动图构建器.
* 可便捷的接收并处理动图.
* Pixiv动图构建器
*/
public final class PixivUgoiraBuilder {
private final Logger log = LoggerFactory.getLogger(PixivUgoiraBuilder.class.getSimpleName() + "@" + Integer.toHexString(this.hashCode()));
private final Logger log = LoggerFactory.getLogger(this.toString());
private final HttpClient httpClient;
private final JsonObject ugoiraMeta;
@ -135,7 +134,7 @@ public final class PixivUgoiraBuilder {
image.getRGB(0, 0, image.getWidth(), image.getHeight(), rgb, 0, image.getWidth());
log.trace("帧解析完成, 正在插入...");
encoder.addImage(Image.fromRgb(rgb, image.getWidth()), new ImageOptions().setDelay(frameInfo.get("delay").getAsLong(), TimeUnit.MILLISECONDS));
log.debug("帧 {} 插入完成.", frameFileName);
log.trace("帧 {} 插入完成.", frameFileName);
} catch (IOException e) {
log.error("解析帧图片数据时发生异常", e);
}

View File

@ -27,7 +27,7 @@ import java.util.List;
*/
public class PixivAccessProxyServer {
private final Logger log = LoggerFactory.getLogger(this.getClass().getSimpleName());
private final Logger log = LoggerFactory.getLogger(PixivAccessProxyServer.class.getName());
private final HttpProxyServer proxyServer;

View File

@ -1,31 +0,0 @@
package net.lamgc.cgj.util;
import org.apache.http.client.CookieStore;
import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.cookie.BasicClientCookie;
import java.util.Date;
public class CookieUtil {
/**
* 将{@link java.net.CookieStore}转换到{@link CookieStore}
* @param netCookieStore java.net.CookieStore
* @return org.apache.http.client.CookieStore
*/
public static CookieStore NAParse(java.net.CookieStore netCookieStore){
CookieStore apacheCookieStore = new BasicCookieStore();
netCookieStore.getCookies().forEach(netCookie -> {
BasicClientCookie aCookie = new BasicClientCookie(netCookie.getName(), netCookie.getValue());
aCookie.setComment(netCookie.getComment());
aCookie.setDomain(netCookie.getDomain());
aCookie.setExpiryDate(new Date(netCookie.getMaxAge()));
aCookie.setPath(netCookie.getPath());
aCookie.setSecure(netCookie.getSecure());
aCookie.setVersion(netCookie.getVersion());
apacheCookieStore.addCookie(aCookie);
});
return apacheCookieStore;
}
}

View File

@ -7,7 +7,7 @@ import net.lamgc.utils.base.runner.StringParameterParser;
public class PagesQualityParser implements StringParameterParser<PixivDownload.PageQuality> {
@Override
public PixivDownload.PageQuality parse(String strValue) throws Exception {
public PixivDownload.PageQuality parse(String strValue) {
return PixivDownload.PageQuality.valueOf(strValue.toUpperCase());
}

View File

@ -0,0 +1,151 @@
package net.lamgc.cgj.util;
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicLong;
/**
* 带有时间限制的线程池.
* 当线程超出了限制时间时, 将会对该线程发出中断.
*/
public class TimeLimitThreadPoolExecutor extends ThreadPoolExecutor {
/**
* 执行时间限制, 单位毫秒.
* 默认30s.
*/
private final AtomicLong executeTimeLimit = new AtomicLong();
/**
* 检查间隔时间.
* 默认100ms.
*/
private final AtomicLong timeoutCheckInterval = new AtomicLong(100);
private final Map<Thread, AtomicLong> workerThreadMap = new Hashtable<>();
private final Thread timeoutCheckThread = createTimeoutCheckThread();
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
setInitialTime(0, executeLimitTime);
timeoutCheckThread.start();
}
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory);
setInitialTime(0, executeLimitTime);
timeoutCheckThread.start();
}
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, RejectedExecutionHandler handler) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, handler);
setInitialTime(0, executeLimitTime);
timeoutCheckThread.start();
}
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory, RejectedExecutionHandler handler) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler);
setInitialTime(0, executeLimitTime);
timeoutCheckThread.start();
}
private void setInitialTime(long checkInterval, long executeLimitTime) {
if(checkInterval > 0) {
timeoutCheckInterval.set(checkInterval);
}
if(executeLimitTime > 0) {
executeTimeLimit.set(executeLimitTime);
}
}
/**
* 设置执行时间.
* <p>注意: 该修改仅在线程池完全停止后才有效</p>
* @see #isTerminated()
* @param time 新的限制时间(ms)
*/
public void setExecuteTimeLimit(long time) {
if(time <= 0) {
throw new IllegalArgumentException("Time is not allowed to be set to 0 or less");
}
if(this.isTerminated()) {
executeTimeLimit.set(time);
}
}
/**
* 设置超时检查间隔.
* <p>该方法仅会在当前检查后生效.</p>
* @param time 新的检查间隔(ms)
*/
public void setTimeoutCheckInterval(long time) {
if(time <= 0) {
throw new IllegalArgumentException("Time is not allowed to be set to 0 or less");
}
timeoutCheckInterval.set(time);
}
/**
* 获取当前设置的执行时间限制.
* @return 执行时间限制(ms).
*/
public long getExecuteTimeLimit() {
return executeTimeLimit.get();
}
/**
* 获取当前设定的超时检查间隔
* @return 间隔时间(ms).
*/
public long getTimeoutCheckInterval() {
return timeoutCheckInterval.get();
}
private Thread createTimeoutCheckThread() {
Thread checkThread = new Thread(() -> {
if(executeTimeLimit.get() <= 0) {
return;
}
while (true) {
try {
long interval = this.timeoutCheckInterval.get();
Thread.sleep(interval);
// 检查是否存在超时的任务
workerThreadMap.forEach((thread, time) -> {
long currentTime = time.getAndAdd(interval);
if(currentTime > executeTimeLimit.get()) {
if(!thread.isInterrupted()) {
thread.interrupt();
}
}
});
} catch(InterruptedException ignored) {
break;
}
}
});
checkThread.setName("ThreadPool-" + Integer.toHexString(this.hashCode()) +"-TimeoutCheck");
return checkThread;
}
@Override
protected void beforeExecute(Thread t, Runnable r) {
workerThreadMap.put(t, new AtomicLong());
super.beforeExecute(t, r);
}
@Override
protected void afterExecute(Runnable r, Throwable t) {
workerThreadMap.remove(Thread.currentThread());
super.afterExecute(r, t);
}
@Override
protected void terminated() {
this.timeoutCheckThread.interrupt();
super.terminated();
}
}

View File

@ -0,0 +1,40 @@
<?xml version="1.0" encoding="UTF-8"?>
<configuration status="WARN">
<properties>
<property name="logStorePath">./logs</property>
<property name="charset">UTF-8</property>
<property name="pattern">[%-d{HH:mm:ss.SSS} %5level][%logger.%method():%-3L][%thread]: %msg%n</property>
<property name="logsDir">${sys:cgj.logsPath:-logs}</property>
</properties>
<Appenders>
<Console name="CONSOLE_STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="${pattern}" charset="${charset}"/>
<Filters>
<ThresholdFilter level="WARN" onMatch="DENY" onMismatch="ACCEPT"/>
</Filters>
</Console>
<Console name="CONSOLE_STDERR" target="SYSTEM_ERR">
<PatternLayout pattern="${pattern}" charset="${charset}"/>
<Filters>
<ThresholdFilter level="WARN" onMatch="ACCEPT" onMismatch="DENY"/>
</Filters>
</Console>
<RollingFile name="rollingFile" fileName="${logsDir}/latest.log" filePattern="${logsDir}/running.%-d{yyyy-MM-dd_HH-mm-ss}.log.gz">
<PatternLayout pattern="${pattern}" charset="${charset}"/>
<Policies>
<OnStartupTriggeringPolicy />
</Policies>
</RollingFile>
</Appenders>
<Loggers>
<Logger level="INFO" name="org.apache.http"/>
<Root level="TRACE">
<AppenderRef ref="CONSOLE_STDOUT"/>
<AppenderRef ref="CONSOLE_STDERR"/>
<AppenderRef ref="rollingFile"/>
</Root>
</Loggers>
</configuration>

View File

@ -4,13 +4,14 @@
<property name="logStorePath">./logs</property>
<property name="charset">UTF-8</property>
<property name="pattern">[%-d{HH:mm:ss.SSS} %5level][%logger.%method():%-3L][%thread]: %msg%n</property>
<property name="logsDir">${sys:cgj.logsPath:-logs}</property>
</properties>
<Appenders>
<Console name="CONSOLE_STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="${pattern}" charset="${charset}"/>
<Filters>
<ThresholdFilter level="WARN" onMatch="DENY" onMismatch="ACCEPT"/>
<LevelRangeFilter minLevel="INFO" maxLevel="INFO" />
</Filters>
</Console>
<Console name="CONSOLE_STDERR" target="SYSTEM_ERR">
@ -20,7 +21,7 @@
</Filters>
</Console>
<RollingFile name="rollingFile" fileName="logs/latest.log" filePattern="logs/running.%-d{yyyy-MM-dd_HH-mm-ss}.log.gz">
<RollingFile name="rollingFile" fileName="${logsDir}/latest.log" filePattern="${logsDir}/running.%-d{yyyy-MM-dd_HH-mm-ss}.log.gz">
<PatternLayout pattern="${pattern}" charset="${charset}"/>
<Policies>
<OnStartupTriggeringPolicy />
@ -30,7 +31,7 @@
<Loggers>
<Logger level="INFO" name="org.apache.http"/>
<Root level="INFO">
<Root level="TRACE">
<AppenderRef ref="CONSOLE_STDOUT"/>
<AppenderRef ref="CONSOLE_STDERR"/>
<AppenderRef ref="rollingFile"/>

View File

@ -0,0 +1,22 @@
package net.lamgc.cgj.pixiv;
import org.junit.Ignore;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@Ignore
public class ModerateContentDetectorTest {
private final static AdultContentDetector acd = new ModerateContentDetector("d91b6c3fa2bba9ee8f9e68827ba0d937");
private final static Logger log = LoggerFactory.getLogger(ModerateContentDetector.class);
@Test
public void checkTest() throws Exception {
log.info("Detect: {}, isAdult: {}",
acd.detect(80840411, false, 0),
acd.isAdultContent(80840411, false, 0));
}
}

View File

@ -1,8 +1,15 @@
package net.lamgc.cgj.pixiv;
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import org.apache.commons.io.IOUtils;
import org.apache.http.HttpHost;
import org.apache.http.client.CookieStore;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Ignore;
@ -24,7 +31,7 @@ public class PixivDownloadTest {
private static CookieStore cookieStore;
private final static Logger log = LoggerFactory.getLogger("PixivDownloadTest");
private final static Logger log = LoggerFactory.getLogger(PixivDownloadTest.class.getName());
private static HttpHost proxy = new HttpHost("127.0.0.1", 1001);
@ -173,22 +180,8 @@ public class PixivDownloadTest {
log.info("正在调用方法...");
try {
pixivDownload.getRankingAsInputStream(null, null, queryDate, 5, 50, PixivDownload.PageQuality.ORIGINAL, (rank, link, rankInfo, inputStream) -> {
/*try {
ZipEntry entry = new ZipEntry("Rank" + rank + "-" + link.substring(link.lastIndexOf("/") + 1));
entry.setComment(rankInfo.toString());
log.info("正在写入: " + entry.getName());
zos.putNextEntry(entry);
IOUtils.copy(inputStream, zos);
zos.flush();
log.info("已成功写入 {}", entry.getName());
inputStream.close();
} catch (IOException e) {
log.error("写入文件项时发生异常", e);
}*/
log.info("空操作");
});
pixivDownload.getRankingAsInputStream(null, null, queryDate, 5, 50,
PixivDownload.PageQuality.ORIGINAL, (rank, link, rankInfo, inputStream) -> log.info("空操作"));
} finally {
zos.finish();
zos.flush();
@ -202,4 +195,17 @@ public class PixivDownloadTest {
log.info(new PixivDownload(cookieStore, proxy).getIllustPreLoadDataById(64076261).toString());
}
@Test
public void illustInfoTest() throws IOException {
CloseableHttpClient httpClient = HttpClientBuilder.create().setProxy(new HttpHost("127.0.0.1", 1001)).build();
HttpGet request = new HttpGet(PixivURL.getPixivIllustInfoAPI(80880547));
CloseableHttpResponse response = httpClient.execute(request);
String body = EntityUtils.toString(response.getEntity());
JsonObject illustInfoResult = new Gson().fromJson(body, JsonObject.class);
log.info("IllustInfoJsonResult: {}", illustInfoResult);
PixivDownload.PixivIllustType illustType = PixivDownload.PixivIllustType.getIllustTypeByPreLoadData(80880547, illustInfoResult);
log.info("IllustType: {}", illustType);
Assert.assertEquals(PixivDownload.PixivIllustType.UGOIRA, illustType);
}
}

View File

@ -0,0 +1,30 @@
package net.lamgc.cgj.util;
import org.junit.Assert;
import org.junit.Test;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.TimeUnit;
public class TimeLimitThreadPoolExecutorTest {
@Test
public void timeoutTest() throws InterruptedException {
TimeLimitThreadPoolExecutor executor = new TimeLimitThreadPoolExecutor(1000, 1, 1, 30, TimeUnit.SECONDS, new ArrayBlockingQueue<>(50));
System.out.println(executor.isTerminated());
System.out.println(executor.isShutdown());
executor.setTimeoutCheckInterval(150);
System.out.println("当前设定: ETL: " + executor.getExecuteTimeLimit() + "ms, TCI: " + executor.getTimeoutCheckInterval() + "ms");
executor.execute(() -> {
try {
Thread.sleep(5 * 1000);
} catch (InterruptedException e) {
System.out.println("线程 " + Thread.currentThread().getName() + " 被中断");
}
});
executor.shutdown();
Assert.assertTrue(executor.awaitTermination(5 * 1000, TimeUnit.MILLISECONDS));
}
}