Compare commits

...

105 Commits

Author SHA1 Message Date
273dbd45b0 [Version] 更新版本(2.5.2-20200604.2-SNAPSHOT -> 2.5.2-20200604.3-SNAPSHOT); 2020-06-04 16:16:53 +08:00
b39a82b936 [Fix] BotGlobal 修复BotGlobal初始化失败的问题; 2020-06-04 16:15:27 +08:00
4589624b9b [Version] 更新版本(2.5.2-20200604.1-SNAPSHOT -> 2.5.2-20200604.2-SNAPSHOT); 2020-06-04 15:58:56 +08:00
033fd0188b [Change] 调整Logger创建方式(getLogger(String) -> getLogger(Class)); 2020-06-04 15:58:10 +08:00
4318869846 [Change] 将'Proxy', 'CookieStore'转移到BotGlobal类; 2020-06-04 15:54:52 +08:00
749b89e668 [Version] 更新版本(2.5.2-20200522.1-SNAPSHOT -> 2.5.2-20200604.1-SNAPSHOT); 2020-06-04 11:26:26 +08:00
9ce18469e6 [Add] 增加控制台框架以进行本地测试, 或非聊天机器人使用; 2020-06-04 11:25:08 +08:00
e803d2161b [Add] PropertiesUtils 增加一个工具类; 2020-06-04 11:24:40 +08:00
3fbf80f233 Merge remote-tracking branch 'origin/master' 2020-06-04 11:20:22 +08:00
8b8ef7e744 [Change] Issue #12 调整框架启动机器人应用的方式, 增加一个用于机器人应用内部共享数据的类;
[Fix] BotEventHandler, ImageCacheStore 增加ShutdownHook用于关闭线程池, 解决线程池阻塞关闭过程的问题;
[Change] BotEventHandler 调整'match(String)'方法;
[Change] BotAdminCommandProcess 调整'savePushList()'方法对文件创建失败的行为;
2020-06-04 11:19:18 +08:00
7fbdf28ec8 [Update] Bug_Report.md 增加默认标签; 2020-06-04 10:50:37 +08:00
b29482927c [Add] Bug_Report.md 添加Bug反馈模板; 2020-06-04 10:47:28 +08:00
69da2b02ac [Fix #11] 修复在图片缓存失效的情况下, 'getImageToBotCode'依然会尝试从缓存获取图片File对象导致NPE;
[Change] BotCommandProcess 调整字符串拼接形式, 统一错误提示语的格式;
2020-06-04 09:29:53 +08:00
eb2de09859 [Change] RandomIntervalSendTimer 将随机间隔发送器的Timer设为daemon thread; 2020-06-03 19:37:45 +08:00
ef70ac77cb [Update] HotDataCacheStore, Cleanable 更新Javadoc; 2020-06-03 17:03:54 +08:00
b4e9fdab7d [Change] TimeLimitThreadPoolExecutor 整理代码, 清除无用参数;
[Update] Pixiv排行榜接口.md 更新文档内容, 补充返回数据信息;
2020-06-03 17:02:46 +08:00
5c5827123d [Update] net.lamgc:java-utils 更新依赖项版本(1.2.0_20200514.1-SNAPSHOT -> 1.2.0_20200517.1-SNAPSHOT), 以修复作品无效检测不生效的问题; 2020-06-03 16:51:26 +08:00
2bfb78304c [Change] HotDataCacheStore, RandomIntervalSendTimer 支持浮动时间参数为0; 2020-06-03 14:50:45 +08:00
637ea46b9a [Fix] MiraiMessageEvent 修复群消息事件错误设置私聊Sender的问题; 2020-06-01 21:02:46 +08:00
f5140a7a1e [Change] SpringCQMessageSenderFactory 尝试通过ThreadLocal获取当前处理的CoolQ对象, 目前EventExecutor暂不支持, 仅预留;
[Change] CQPluginMain 调整'processMessage'方法修饰符, 补充Javadoc;
2020-06-01 20:15:59 +08:00
0fafddc715 [Change] AdultContentDetector, PixivURL 整理代码, 补充Javadoc; 2020-06-01 16:12:33 +08:00
4c03a0f7d9 [Change] HotDataCacheStore 调整'supportedList'的条件, HotDataCacheStore将支持使用ListCacheStore;
[Change] AutoSender 将AutoSender的messageSender增加final属性;
2020-06-01 16:01:17 +08:00
ee02072b2d [Deprecated] PixivURL 将旧版Search接口设为弃用; 2020-06-01 15:58:54 +08:00
3f6c87da3c [Add] MiraiMain 增加对临时消息的支持;
[Change] MiraiMessageEvent 调整消息对象转换的方式, 通过静态方法转换Mirai的MessageEvent;
2020-06-01 15:55:31 +08:00
d45cd8aab5 [Update] 更新文档, 增加排行榜接口文档, 待补充返回数据; 2020-05-28 16:20:16 +08:00
598d6ef205 [Fix] README.md 修复格式错误;
[Update] README.md 补充参数细节;
2020-05-27 19:30:04 +08:00
2190ef1b59 [Change] README.md 调整Readme内容以减少对用户选择带来的影响; 2020-05-27 19:27:29 +08:00
fcc522c057 Merge branch 'master' of github.com:LamGC/ContentGrabbingJi 2020-05-26 14:45:01 +08:00
5a81604d94 [Change] BotCommandProcess 调整'色图'命令处理方法的命名; 2020-05-26 14:31:12 +08:00
376306e404 [Update] README.md 更新文档说明, 提供详细命令说明和注意事项; 2020-05-26 10:37:40 +08:00
0fe69253ed [Change] .gitignore 将Idea项目文件由指定文件名改为匹配后缀名; 2020-05-26 10:28:46 +08:00
fe8bf16d3a [Clean] RandomRankingArtworksSender 清理无用导入;
[Change] AutoCleanTimer 统一LoggerName名称;
2020-05-25 20:23:14 +08:00
99c66babec [Add] BotEventHandler 增加对'Sender.sendMessage(String)'返回值的处理; 2020-05-25 09:45:09 +08:00
cb8b01fd74 Merge branch 'master' of github.com:LamGC/ContentGrabbingJi 2020-05-25 09:25:49 +08:00
1f48b3ffdc [Change] MessageSender, MessageEvent 允许'senderMessage(String)'抛出异常;
[Change] RandomRankingArtworksSender 调整异常捕获日志输出;
[Change] MiraiMessageEvent, MiraiMessageSender, SpringCQMessageEvent 适配更改;
2020-05-25 09:21:46 +08:00
68feed8d3c Merge remote-tracking branch 'origin/master' 2020-05-22 23:09:39 +08:00
d89711b0b9 [Version] 更新版本(2.5.2-20200520.1-SNAPSHOT -> 2.5.2-20200522.1-SNAPSHOT); 2020-05-22 20:42:29 +08:00
a3376e96ee [Add] BotCommandProcess, BotEventHandler, PixivDownload 对作品Id不存在的情况做出反应(而不是作为内部异常反馈); 2020-05-22 20:39:49 +08:00
33d18cef6b Merge remote-tracking branch 'origin/master' 2020-05-22 20:34:32 +08:00
7aa00ff98b [Fix] BotCommandProcess 修复ImageStore不允许使用软链接的问题; 2020-05-22 20:34:15 +08:00
e956e36584 [Fix] RankingUpdateTimer 修复首次执行时间计算错误的问题; 2020-05-22 20:23:46 +08:00
8e27221457 [Change] BufferMessageEvent 调整成员变量修饰符, 补充新的构造方法'<init>(int, int, String)'; 2020-05-22 08:49:48 +08:00
a5fca68ef5 [Change] MessageEventExcutionDebugger 在配置文件无法转换的情况下不再忽略'NumberFormatException'异常; 2020-05-20 15:31:43 +08:00
8edb728fe9 [Update] README.md 补充Readme内容 2020-05-20 14:34:55 +08:00
cc05af8a24 Create README.md 2020-05-20 11:00:25 +08:00
60fa641962 [Version] 更新版本(2.5.2-20200517.1-SNAPSHOT -> 2.5.2-20200520.1-SNAPSHOT); 2020-05-20 10:36:30 +08:00
3ab373cc70 [Update] .gitignore 补充VSCode忽略项; 2020-05-20 10:34:55 +08:00
461cd246d8 [Add] BotCommandProcess 新增'色图'功能以随机从排行榜中获取一张图片;
[Add] BufferMessageEvent 增加用于获取Sender发送内容的Event实现;
[Change] BotEventHandler Handler现在允许命令处理方法不返回消息(返回'null');
2020-05-20 09:55:50 +08:00
f040f130d7 [Update] Main, AutoArtworksSender, ImageCacheStore, PixivAccessProxyServer, PagesQualityParser 整理代码问题, 删除无用类; 2020-05-20 09:05:50 +08:00
1f402fbbac [Change] LocalHashCacheStore 实现'Cleanable'接口, 增加'<init>(int, int, boolean)'构造函数以允许自动清理
[Update] LocalHashCacheStore 补充Javadoc内容;
[Update] .gitignore 补充并整理忽略项;
2020-05-20 08:56:27 +08:00
f7f3c3beaf [Fix] BotCommandProcess 修复help命令中帮助信息的错误; 2020-05-17 19:38:17 +08:00
04c1753c22 [Version] 更新版本(2.5.1 -> 2.5.2-20200517.1-SNAPSHOT); 2020-05-17 17:45:06 +08:00
d993e9d719 [Add] MiraiMessageSender 增加对默认表情的支持; 2020-05-12 15:25:29 +08:00
60e91987d1 Merge branch 'mirai' 2020-05-12 15:14:58 +08:00
65392fc2fe [Fix] net.mamoe:mirai-core, net.mamoe:mirai-core-qqandroid 更新Mirai-Core版本号以修复VerifyError问题(1.0-RC2 -> 1.0-RC2-1);
[Fix] MiraiMessageSender 调整日志输出, 以修复日志打印处理后消息顺序混乱的问题;
2020-05-12 15:14:28 +08:00
d38934a0f4 [Change] RankingUpdateTimer 调整时间补齐的阀值(12:00 -> 11:30);
[Add] documents/interfaces/* 整理部分Pixiv接口的文档;
2020-05-12 15:03:08 +08:00
7943357d96 [Fix] MiraiMain 修复未登录时异常退出, 抛出NPE的问题; 2020-05-11 12:16:35 +08:00
a170ff040b [Update] net.mamoe:mirai-core, net.mamoe:mirai-core-qqandroid 更新依赖项版本(0.39.4 -> 1.0-RC2);
[Update] MiraiMain, MiraiMessageEvent 适配mirai新版本;
[Change] MiraiMessageSenderFactory 增加参数检查并优化过程;
[Change] MiraiMain 显性指定Mirai所用协议为'BotConfiguration.MiraiProtocol.ANDROID_PAD';
2020-05-11 11:36:37 +08:00
cb2ebfdb73 [Change] MessageEvent 对消息内容删除首尾空格以提高用户体验性; 2020-05-09 23:13:53 +08:00
66b22c543a [Fix] SettingProperties 修复了配置文件无法创建, 导致无法保存配置的问题;
[Fix] Main 修复了潜在的文件创建失败的问题;
2020-05-09 22:54:32 +08:00
6bace4b048 [Version] 更新版本(2.5.0 -> 2.5.1); 2020-05-09 22:35:46 +08:00
597aac4e95 [Fix] BotCommandProcess 修复部分指令无法使用群组配置的问题; 2020-05-09 22:35:21 +08:00
181d60285b [Version] 更新版本(2.5.0-20200509.1-SNAPSHOT -> 2.5.0); 2020-05-09 22:20:07 +08:00
c06b8717c3 [Add] CacheStoreUtils 增加用于缓存库的工具类;
[Update] BotCommandProcess 补充注释, 优化"info"命令输出内容的格式;
2020-05-09 22:19:29 +08:00
b4a28b6735 [Version] 更新版本(2.5.0-20200506.1-SNAPSHOT -> 2.5.0-20200509.1-SNAPSHOT); 2020-05-09 18:16:54 +08:00
40057c3683 [Fix] Issue #1 修复了ImageCache对重复缓存请求的策略(忽略并返回)导致的后续处理发生NPE的问题;
[Update] ImageCache 更新了ImageCache, 将该部分单独分离到 ImageCacheStore;
[Update] BotCommandProcess 适配新的ImageCache, 调整内容检查的时机以减少不必要的API访问;
2020-05-09 18:16:16 +08:00
4beb4d78fb [Fix] BotCommandProcess 修复Search命令格式不正确的问题; 2020-05-09 12:52:15 +08:00
d9edaa681f [Add] BotAdminCommandProcess 为 addPushGroup 增加参数检查; 2020-05-09 10:10:48 +08:00
7b52abde60 [Change] BotCommandProcess 增加Search命令的反馈内容(可能必须要QQ支持长文本才能使用); 2020-05-08 23:31:24 +08:00
2cb4fe1dbc [Fix] SettingProperties 修复无法设置全局配置项的问题; 2020-05-08 12:23:43 +08:00
5830327dad [Add] BotEventHandler 增加禁言记录, 防止应用因过多的在禁言状态反馈导致被封号;
[Update] MiraiMain 适配禁言记录功能;
2020-05-07 19:46:54 +08:00
cbd10ff281 [Fix] Issue #4 修复因Mirai API变动导致应用无法从消息获取ImageId的问题; 2020-05-07 18:24:09 +08:00
84b8006f89 [Change] RankingUpdateTimer 将定时更新时间调整为"+8 11:30", 并增加日期指定; 2020-05-07 15:21:05 +08:00
d32d891ad7 [Update] BotCommandProcess 补充Javadoc内容;
[Add] BotCommandProcess 在Ranking命令增加force参数以允许强制查询预计不在排行榜范围的日期;
2020-05-07 11:29:31 +08:00
aabd35ce71 [Add] AutoCleanTimer, Cleanable 增加缓存库自动清理;
[Update] HotDataCacheStore 增加对自动清理的支持;
[Change] RedisPoolCacheStore 将Redis缓存库抽象类设为default(原public);
[Change] MiraiMessageSender, BotCommandProcess 适配HotDataCacheStore的修改;
[Change] BotCommandProcess 调整ranking命令的输出格式;
2020-05-07 10:26:11 +08:00
a36eb713d7 [Version] 更新版本(2.5.0-20200505.1-SNAPSHOT -> 2.5.0-20200506.1-SNAPSHOT); 2020-05-06 11:09:18 +08:00
f452c1c128 Merge branch 'add-MultiProperties' 2020-05-06 11:08:16 +08:00
cb42aadb15 [Fix] SettingProperties 修复在初次设置群组配置时未创建该群组所属Properties导致NPE的问题;
[Update] BotAdminCommandProcess 更新日志内容及命令返回信息;
[Change] BotCommandProcess 调整配置项key格式;
2020-05-06 10:55:32 +08:00
b969bb29b2 Merge branch 'master' into add-MultiProperties 2020-05-06 10:18:00 +08:00
0d74007a98 [Change] search.txt 整理接口相关文档的存储目录; 2020-05-06 09:42:34 +08:00
edade24883 [Change] Issue #2 调整Logger Name以统一Logger Name格式; 2020-05-06 09:34:28 +08:00
34f57404ca [Update] 重构Setting Properties, 使其支持分群配置; 2020-05-06 00:57:48 +08:00
99b6e14ff7 [Change] RandomIntervalSendTimer 调整日志输出内容; 2020-05-05 17:09:45 +08:00
c8fe2c3fdd [Version] 更新版本(2.5.0-20200504.2-SNAPSHOT -> 2.5.0-20200505.1-SNAPSHOT);
[Update] net.lamgc:java-utils 更新依赖版本(1.1.0 -> 1.2.0_20200505.1-SNAPSHOT);
2020-05-05 01:29:11 +08:00
49a33d4078 [Add] MessageEventExecutionDebugger 添加对消息处理的调试器Enum;
[Add] BotEventHandler 添加对 MessageEventExecutionDebugger 的支持;
[Add] VirtualLoadMessageEvent 增加 toVirtualLoadMessageEvent(MessageEvent) 方法;
2020-05-05 01:27:10 +08:00
cd1d2316ee [Add] BotEventHandler 添加executeMessageEvent(MessageEvent)方法;
[Change] BotEventHandler 将BotEventHandler.executor设为private;
[Change] MiraiMain, CQPluginMain, RankingUpdateTimer 适配BotEventHandler的调整
2020-05-04 23:06:07 +08:00
cf08353ed9 [Change] BotEventHandler 调整事件处理线程池的线程数(min: 4, Max: 32); 2020-05-04 22:52:47 +08:00
bef6a684b9 [Change] BotEventHandler 调整事件处理线程池的线程数; 2020-05-04 22:46:06 +08:00
04960889b4 [Version] 更新版本(2.5.0-20200504.01-SNAPSHOT -> 2.5.0-20200504.2-SNAPSHOT);
[Update] net.lamgc:java-utils 更新依赖版本(1.1.0_5-SNAPSHOT -> 1.1.0);
[Update] net.lamgc:PixivLoginProxyServer 更新依赖版本(1.1.0 -> 1.1.1);
2020-05-04 19:44:11 +08:00
0dc8fc78b4 [Version] 更新版本(2.4.0 -> 2.5.0-20200504.01-SNAPSHOT); 2020-05-04 17:07:47 +08:00
40c5284be2 [Change] Dockerfile.sample 调整构建指令顺序以充分利用构建缓存;
[Fix] Main 修复SystemProperties设置null抛出NPE的问题;
2020-05-04 17:07:39 +08:00
46cb47c7d1 [Add] LICENSE 添加开源许可证(LGPLv3); 2020-05-04 16:55:16 +08:00
fe213deecb [Change] Main 调整参数的接收形式;
[Fix] Dockerfile.sample 修复Java应用无法接收stop命令发送信号的问题;
[Change] RankingUpdateTimer 调整更新投递形式, 将同步更新调整为异步更新;
[Change] BotEventHandler 实装TimeLimitThreadPoolExecutor;
2020-05-04 02:07:46 +08:00
f279d99fda [Add] TimeLimitThreadPoolExecutor 增加一个带有执行时间限制的线程池及对应单元测试类; 2020-05-04 02:04:59 +08:00
a09aef5be2 [Delete] 删除已完成或已经没有存在意义的TODO; 2020-05-02 10:40:11 +08:00
766411fa09 [Fix] Dockerfile.sample 修复应用无法接收容器退出信号的问题(现在应用将可以接受信号主动退出, 即使ExitCode不为0); 2020-05-02 00:56:05 +08:00
84b544cea9 [Change] Main 调整日志输出级别, 补充日志输出内容;
[Change] Dockerfile.sample 调整镜像内应用运行路径;
2020-05-01 22:28:15 +08:00
2f492f5b03 [Add] log4j2.xml, log4j2-test.xml 添加允许指定日志存储目录的功能(sys:cgj.logsPath); 2020-05-01 13:19:08 +08:00
15af939c3f [Fix] PixivDownload 修复因排行榜总量的不确定导致排行榜获取异常的问题;
[Change] BotEventHandler 调整事件处理完成后的日志输出形式;
2020-04-30 19:08:34 +08:00
a28cb142b4 [Change] 调整数据存储的路径设置及存储目录参数名; 2020-04-30 17:51:57 +08:00
e8fda3214b [Version] 更新版本(2.3.0 -> 2.4.0); 2020-04-30 02:24:21 +08:00
fba6d3532c [Delete] 删除不再使用的类和方法;
[Update] 优化Javadoc并设置编译注解, 优化代码;
[Change] 调整部分日志的输出级别;
[Change] RedisPoolCacheStore clear方法将根据返回信息确定是否清空成功;
[Change] Dockerfile.sample 调整镜像构建步骤;
2020-04-30 02:23:31 +08:00
0075446412 [Fix] MiraiMessageSender 修复因BotCode-Image中flashImage参数不存在导致NPE的问题; 2020-04-30 01:54:27 +08:00
2388cd419e [Add] Dockerfile.sample 增加一个用于构建CGJ-BotMode镜像的Dockerfile模板; 2020-04-30 00:45:41 +08:00
7f83b16118 [Add] Main 添加日志输出内容; 2020-04-29 23:44:38 +08:00
66 changed files with 2578 additions and 1252 deletions

26
.github/ISSUE_TEMPLATE/Bug_Report.md vendored Normal file
View File

@ -0,0 +1,26 @@
---
name: Bug Report
about: Use this template to feedback bugs.
title: ''
labels: bug
assignees: ''
---
## Environmental information ##
OS(e.g: Windows 10 1909):
Java(e.g: Oracle Jdk 8.242):
Issue version(versionTag or commitId):
## Problem description ##
// Describe the problem in as much detail as possible here
## Expected behavior ##
// What will this function do under normal circumstances?
## Actual behavior ##
// But what does this feature actually look like?
## Recurrence steps ##
// What can we do to recreate this situation?
1.

14
.gitignore vendored
View File

@ -1,6 +1,16 @@
# Ignore test date folder
/pluginData/
/logs/
/.idea/
/CGJ_2.iml
/cookies.store
/target/
# Ignore Idea files
/.idea/
*.iml
# Ignore Visual Studio Code files
.classpath
.factorypath
.project
/.settings/
/.vscode/

10
Dockerfile.sample Normal file
View File

@ -0,0 +1,10 @@
FROM openjdk:8u252-jre
ENV jarFileName=ContentGrabbingJi-exec.jar
ENV CGJ_REDIS_URI="127.0.0.1:6379"
ENV CGJ_PROXY=""
RUN mkdir /data/
COPY ${jarFileName} /CGJ.jar
CMD ["java", "-Dcgj.logsPath=/data/logs", "-jar", "/CGJ.jar", "botMode", "-botDataDir=/data"]

165
LICENSE Normal file
View File

@ -0,0 +1,165 @@
GNU LESSER GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
This version of the GNU Lesser General Public License incorporates
the terms and conditions of version 3 of the GNU General Public
License, supplemented by the additional permissions listed below.
0. Additional Definitions.
As used herein, "this License" refers to version 3 of the GNU Lesser
General Public License, and the "GNU GPL" refers to version 3 of the GNU
General Public License.
"The Library" refers to a covered work governed by this License,
other than an Application or a Combined Work as defined below.
An "Application" is any work that makes use of an interface provided
by the Library, but which is not otherwise based on the Library.
Defining a subclass of a class defined by the Library is deemed a mode
of using an interface provided by the Library.
A "Combined Work" is a work produced by combining or linking an
Application with the Library. The particular version of the Library
with which the Combined Work was made is also called the "Linked
Version".
The "Minimal Corresponding Source" for a Combined Work means the
Corresponding Source for the Combined Work, excluding any source code
for portions of the Combined Work that, considered in isolation, are
based on the Application, and not on the Linked Version.
The "Corresponding Application Code" for a Combined Work means the
object code and/or source code for the Application, including any data
and utility programs needed for reproducing the Combined Work from the
Application, but excluding the System Libraries of the Combined Work.
1. Exception to Section 3 of the GNU GPL.
You may convey a covered work under sections 3 and 4 of this License
without being bound by section 3 of the GNU GPL.
2. Conveying Modified Versions.
If you modify a copy of the Library, and, in your modifications, a
facility refers to a function or data to be supplied by an Application
that uses the facility (other than as an argument passed when the
facility is invoked), then you may convey a copy of the modified
version:
a) under this License, provided that you make a good faith effort to
ensure that, in the event an Application does not supply the
function or data, the facility still operates, and performs
whatever part of its purpose remains meaningful, or
b) under the GNU GPL, with none of the additional permissions of
this License applicable to that copy.
3. Object Code Incorporating Material from Library Header Files.
The object code form of an Application may incorporate material from
a header file that is part of the Library. You may convey such object
code under terms of your choice, provided that, if the incorporated
material is not limited to numerical parameters, data structure
layouts and accessors, or small macros, inline functions and templates
(ten or fewer lines in length), you do both of the following:
a) Give prominent notice with each copy of the object code that the
Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the object code with a copy of the GNU GPL and this license
document.
4. Combined Works.
You may convey a Combined Work under terms of your choice that,
taken together, effectively do not restrict modification of the
portions of the Library contained in the Combined Work and reverse
engineering for debugging such modifications, if you also do each of
the following:
a) Give prominent notice with each copy of the Combined Work that
the Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the Combined Work with a copy of the GNU GPL and this license
document.
c) For a Combined Work that displays copyright notices during
execution, include the copyright notice for the Library among
these notices, as well as a reference directing the user to the
copies of the GNU GPL and this license document.
d) Do one of the following:
0) Convey the Minimal Corresponding Source under the terms of this
License, and the Corresponding Application Code in a form
suitable for, and under terms that permit, the user to
recombine or relink the Application with a modified version of
the Linked Version to produce a modified Combined Work, in the
manner specified by section 6 of the GNU GPL for conveying
Corresponding Source.
1) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (a) uses at run time
a copy of the Library already present on the user's computer
system, and (b) will operate properly with a modified version
of the Library that is interface-compatible with the Linked
Version.
e) Provide Installation Information, but only if you would otherwise
be required to provide such information under section 6 of the
GNU GPL, and only to the extent that such information is
necessary to install and execute a modified version of the
Combined Work produced by recombining or relinking the
Application with a modified version of the Linked Version. (If
you use option 4d0, the Installation Information must accompany
the Minimal Corresponding Source and Corresponding Application
Code. If you use option 4d1, you must provide the Installation
Information in the manner specified by section 6 of the GNU GPL
for conveying Corresponding Source.)
5. Combined Libraries.
You may place library facilities that are a work based on the
Library side by side in a single library together with other library
facilities that are not Applications and are not covered by this
License, and convey such a combined library under terms of your
choice, if you do both of the following:
a) Accompany the combined library with a copy of the same work based
on the Library, uncombined with any other library facilities,
conveyed under the terms of this License.
b) Give prominent notice with the combined library that part of it
is a work based on the Library, and explaining where to find the
accompanying uncombined form of the same work.
6. Revised Versions of the GNU Lesser General Public License.
The Free Software Foundation may publish revised and/or new versions
of the GNU Lesser General Public License from time to time. Such new
versions will be similar in spirit to the present version, but may
differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the
Library as you received it specifies that a certain numbered version
of the GNU Lesser General Public License "or any later version"
applies to it, you have the option of following the terms and
conditions either of that published version or of any later version
published by the Free Software Foundation. If the Library as you
received it does not specify a version number of the GNU Lesser
General Public License, you may choose any version of the GNU Lesser
General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide
whether future versions of the GNU Lesser General Public License shall
apply, that proxy's public statement of acceptance of any version is
permanent authorization for you to choose that version for the
Library.

39
README.md Normal file
View File

@ -0,0 +1,39 @@
# ContentGrabbingJi
Pixiv爬虫一只同时也是一个机器人/插件!
## 支持的机器人平台 ##
- [Mirai](https://github.com/mamoe/mirai)
- [CoolQ](https://cqp.cc)(基于[`SpringCQ`](https://github.com/lz1998/spring-cq), 不支持多账号使用, 需要使用[`CQHttp`](https://cqhttp.cc/)插件)
## Usage ##
> 注意: 运行色图姬前, 你需要准备一个Pixiv账号的会话Cookie存储文件, 否则色图姬将无法运行.
> 详见[PixivLoginProxyServer](https://github.com/LamGC/PixivLoginProxyServer)项目的[Readme](https://github.com/LamGC/PixivLoginProxyServer/blob/master/README.md).
### Arguments ###
> ENV参数名为环境变量名, 用于给Docker容器提供设置方式.
- 通用参数
- `-proxy` / `ENV: CGJ_PROXY`: (**可选**) 设置代理
- 格式: `协议://地址:端口`
- 示例: `socks5://127.0.0.1:1080`
- 机器人参数
- `-botDataDir` / `ENV: CGJ_BOT_DATA_DIR`: (**可选**) 设置`botMode`运行模式下机器人数据存储目录
- 格式: `路径`
- 示例: `./data`
- 默认: `./`
- `-redisAddress` / `ENV: CGJ_REDIS_URI`: (**必填, 计划支持可选**) Redis服务器地址
- 格式: `地址:端口`
- 示例: `127.0.0.1:6379`
> 例如以BotMode启动应用: `java -jar CGJ.jar botMode -proxy "socks5://127.0.0.1:1080 -redisAddress 127.0.0.1:6379`
### Commands ###
- `pluginMode`: CoolQ插件模式(依赖[酷Q机器人](https://cqp.cc/), 支持与CoolQ其他插件共存, 性能耗损大)
- `botMode`: Mirai独立模式(机器人独立运行, 不支持与其他插件共存, 性能耗损小)
- `collectionDownload`: 收藏下载, 以原图画质下载Cookie所属账号的所有收藏作品
- `getRecommends`: 将访问主页获得的推荐作品全部以原图画质下载
- `getRankingIllust`: 以原图画质下载指定排行榜类型的全部作品
- `search`: 搜索指定内容并获取相关作品信息(不下载)
> 注意: 除去机器人模式外, 其他功能后续可能会出现改动.

View File

@ -0,0 +1,149 @@
## Pixiv 排行榜获取接口 ##
### 接口地址 ###
```
GET https://www.pixiv.net/ranking.php
```
- 需要登录: `是`
### 参数 ###
> 提示: 该接口参数较为复杂,请结合表格查看
- `date`: 排行榜时间与Mode有关 (格式: yyyy-MM-dd)
- `mode`: 排行榜模式
- `daily`: 每天
- `weekly`: 每周
- `monthly`: 每月
- `rookie`: 新人
- `original`: 原创
- `male`: 男性偏好
- `female`: 女性偏好
- `daily_r18`: 每天 - 仅成人内容
- `weekly_r18`: 每周 - 仅成人内容
- `male_r18`: 男性偏好 - 仅成人内容
- `female_r18`: 女性偏好 - 仅成人内容
- `content`: 排行榜内容类型
- `all`: 全部内容 (实际使用请直接省略`content`参数)
- `illust`: 插画
- `ugoira`: 动图
- `manga`: 漫画
- `p`: 排行榜页数 (如超出范围,则返回错误信息)
- `format`: 格式
- `json`: 以Json返回数据
- (留空): 返回完整的排行榜网页
#### 参数关系表 ####
`mode`参数与`content`参数有一些支持关系,并不是所有的`mode`参数都能被所有的`content`参数支持,故附下表。
参数 |all|illust|ugoira|manga
:-: |:-:| :-: | :-: | :-:
daily |`√`|`√`|`√`|`√`
weekly |`√`|`√`|`√`|`√`
monthly |`√`|`√`|×|`√`
rookie |`√`|`√`|×|`√`
original |`√`|×|×|×
male |`√`|×|×|×
female |`√`|×|×|×
daily_r18 |`√`|`√`|`√`|`√`
weekly_r18|`√`|`√`|`√`|`√`
male_r18 |`√`|×|×|×
female_r18|`√`|×|×|×
### 返回数据 ###
#### 数据示例 ####
```json
{
"contents":[
{
"title":"【伊アオ】髪結い。",
"date":"2020年05月31日 14:26",
"tags":[
"鬼滅の刃",
"伊アオ",
"嘴平伊之助",
"神崎アオイ",
"鬼滅の刃1000users入り"
],
"url":"https:\/\/i.pximg.net\/c\/240x480\/img-master\/img\/2020\/05\/31\/14\/26\/41\/81987309_p0_master1200.jpg",
"illust_type":"0",
"illust_book_style":"0",
"illust_page_count":"1",
"user_name":"シロウ",
"profile_img":"https:\/\/i.pximg.net\/user-profile\/img\/2020\/05\/01\/02\/18\/18\/18450100_ac34872504959f8cc26f086248066b39_50.png",
"illust_content_type":{
"sexual":0,
"lo":false,
"grotesque":false,
"violent":false,
"homosexual":false,
"drug":false,
"thoughts":false,
"antisocial":false,
"religion":false,
"original":false,
"furry":false,
"bl":false,
"yuri":false
},
"illust_series":false,
"illust_id":81987309,
"width":600,
"height":2226,
"user_id":174995,
"rank":51,
"yes_rank":83,
"rating_count":707,
"view_count":19759,
"illust_upload_timestamp":1590902801,
"attr":"",
"is_bookmarked":false,
"bookmarkable":true
}, // ....
],
"mode":"daily",
"content":"all",
"page":2,
"prev":1,
"next":3,
"date":"20200601",
"prev_date":"20200531",
"next_date":false,
"rank_total":500
}
```
#### 参数详解 ####
- `contents`: (`Object[]`) 排行榜数组, 最多50行排行榜信息
- `illust_id`: (`number`) 作品Id
- `title`: (`string`) 作品标题
- `attr`: (`string`) 不明?
- `tags`: (`string[]`) 原始标签数组
- `url`: (`string`) 预览画质的原始尺寸图下载链接(存在防盗链)
- `illust_type`: (`string` -> `number`) 作品类型
- `illust_book_style`: (`string` -> `number`) 不明?
- `illust_page_count`: (`string` -> `number`) 作品页数
- `user_name`: (`string`) 画师用户名
- `user_id`: (`number`) 画师用户Id
- `profile_img`: (`string`) 画师用户头像
- `illust_content_type`: (`Object`) 作品内容信息
- 待补充
- `illust_series`: (`boolean`) 不明?
- `width`: (`number`) 作品宽度(建议以原图为准)
- `height`: (`number`) 作品高度(建议以原图为准)
- `rank`: (`number`) 本期排行榜排名
- `yes_rank`: (`number`) 上期同排行榜排名
- `rating_count`:
- `view_count`: (`number`) 浏览量
- `illust_upload_timestamp`: (`number`) 作品上传时间戳(10位)
- `is_bookmarked`: (`boolean`) 不明?
- `bookmarkable`: (`boolean`) 不明?
- `mode`: (`string`) 请求的排行榜模式字段
- `content`: (`string`) 请求的内容类型
- `page`: (`number`) 当前排行榜页数
- `prev`: (`string` / `boolean`) 上一页排行榜页数, 如果该请求的页数为首页, 则为`false`
- `next`: (`string` / `boolean`) 下一页排行榜页数, 如果该请求的页数为页尾, 则为`false`
- `date`: (`string`) 排行榜日期(格式:`yyyyMMdd`)
- `prev_date`: (`string` / `boolean`) 如果存在上一期排行榜, 则该属性为上期排行榜日期字符串, 否则为`false`
- `next_date`: (`string` / `boolean`) 如果存在下一期排行榜, 则该属性为下期排行榜日期字符串, 否则为`false`
- `rank_total`: (`number`) 该排行榜的总排行数

View File

@ -0,0 +1,18 @@
## Pixiv接口标准返回格式 ##
Pixiv大部分接口在返回数据时都会遵循以下格式
```json
{
"error": false,
"message": "",
"body": {
}
}
```
大部分是如此(部分接口比较特别, 不是这个格式)
属性|类型|说明
---|---|---
error|Boolean|如果接口返回错误信息, 该属性为`true`
message|String|如果`error`属性为`true`, 则该属性不为空字符串
body|Object/Array|如果`error`不为`true`, 该属性有数据, 否则属性可能不存在

View File

@ -0,0 +1,58 @@
## Pixiv内容搜索接口 ##
### 说明 ###
> 注意: 本接口可能会影响Pixiv对账号的行为判断猜测不一定会
该接口用于在Pixiv搜索内容。
### 接口地址 ###
```
GET https://www.pixiv.net/ajax/search/{Type}/{Content}?{Param...}
```
- 需要登录: `是`
### Url参数 ###
- `Type`: 内容类型
- illustrations(插画)
- top(推荐)
- manga(漫画)
- novels(小说)
- `Content`: 搜索内容
### 参数 ###
#### 必填 ####
- `word`: 与搜索内容一致 (经测试似乎可以省略)
- `s_mode`: 匹配模式
- `s_tag`: 标签,部分一致
- `s_tag_full`: 标签,完全一致
- `s_tc`: 标签和说明文字
- `type`: 作品类型
- `all`: 插画、漫画和动图
- `illust_and_ugoira`: 插画和动图
- `illust`: 仅插画
- `manga`: 仅漫画
- `ugoira`: 仅动图
- `p`: 指定页数 (当指定页数超出范围后,`body.illust.data`为空数组)
- `order`: 排序设置
- `date`: 按时间从旧到新
- `date_d`: 按时间从新到旧
- `(Unknown)`: 未知, 猜测是会员功能的热门搜索
- `mode`: 内容分级设置
- `all`: 全部内容
- `safe`: 排除成人内容
- `r18`: 仅成人内容
#### 选填 ####
- `wlt`: 作品最低宽度(px)
- `wgt`: 作品最高宽度(px)
- `hlt`: 作品最低高度(px)
- `hgt`: 作品最高高度(px)
- `ratio`: 作品横宽比过滤 (初步测试表明,该参数无法指定横宽比,可能暂不支持该功能)
- `0.5`: 仅横图
- `-0.5`: 仅纵图
- `0`: 仅正方形图
- `tool`: 限定作品绘制工具
- `scd`: 过滤作品发布时间 - 开始时间(yyyy-MM-dd)
- `scd`: 过滤作品发布时间 - 结束时间(yyyy-MM-dd)
- `(Unknown)`: 最小收藏数 (该参数为会员限定功能,后续补充)

View File

@ -0,0 +1,129 @@
## Pixiv首页数据接口 ##
### 说明 ###
> 注意: 该接口涉及用户账户隐私, 不要尝试对该接口返回数据做不安全云端存储, 或未经用户允许的发送出去.
该接口用于获取Pixiv推荐给账号的首页作品信息每次调用都会有不同结果。
### 接口地址 ###
```
GET https://www.pixiv.net/ajax/top/{type}?mode={mode}&lang={lang}
```
### 参数 ###
- `type`: 首页类型
- `illust`: 插画
- `manga`: 漫画
- `novel`: 小说
- `mode`: 内容类型
- `all`: 不限类型
- `r18`: 成人内容
- `lang`: 语言(只写几个)
- `zh`: 中文
> 是否需要登录: `是`
> 是否为Pixiv接口标准返回格式: `是`
> 是否需要Referer请求头: `未知`
请求Url示例:
```
GET https://www.pixiv.net/ajax/top/illust?mode=all&lang=zh
```
响应示例:
```
(内容过长, 略)
```
返回内容(Json):
- `page`: 网页相关内容
- `tags`: (`Object[]`) 热门标签
- `tag`: (`String`) 标签名
- `count`: (`Number`) 作品数量?
- `lev`: (`Number`) 不明?
- `ids`: (`Number[]`) 作品数组
- `follow`: (`Number[]`) 已关注作者的作品推荐
- `mypixiv`: (`?[]`) 不明
- `recommend`: (`String[]` -> `Number[]`) 推荐作品的Id?
- `recommendByTag`: (`Object[]`) 标签的推荐作品
- `tag`: (`String`) 标签名
- `ids`: ((`String[]` -> `Number[]`) 作品Id数组
- `ranking`: (`Object`) 排行榜前100名数据
- `date`: (`String`) 排行榜日期(`yyyyMMdd`)
- `items`: (`Object[]`) 排行榜简略数据
- `rank`: (`String` -> `Number`) 排行榜名次
- `id`: (`String` -> `Number`) 作品Id
- `pixivision`: (`Object[]`) Pixiv的推荐文章
- `id`: (`String` -> `Number`) 文章Id
- `title`: (`String`) 文章标题
- `thumbnailUrl`: (`String`) 文章封面图的Url地址
- `abtestId`: (`String` -> `Number`) pixivision文章地址的参数`utm_content`的值
- `recommendUser`: (`Object[]`) 推荐用户及其作品
- `id`: (`Number`) 用户Id
- `illustIds`: (`String[]` -> `Number[]`) 插画作品Id
- `novelIds`: (`String[]` -> `Number[]`) 小说作品Id
- `contestOngoing`: (`Object[]`) 进行中的比赛活动(没找到更多信息了, 先这么说着)
- (待完善)
- `contestResult`: (`Object[]`) 比赛结果信息
- `slug`: (`String`) 比赛代号?
- `type`: (`String`) 比赛作品类型?
- `name`: (`String`) 比赛名
- `url`: (`String`) 结果公布链接
- `iconUrl`: (`String`) 图标链接(不明意义的图标)
- `editorRecommend`: (`Object[]`) 编辑推荐(小说), 后续可能会删除, 这个应该是配合活动出的数据
- (待完善)
- `boothFollowItemIds`: (`String[]` -> `Number[]`) 已关注用户的最新商品
- `sketchLiveFollowIds`: (`?[]`) 不明?
- `sketchLivePopularIds`: (`String[]` -> `Number[]`) 不明?
- `myFavoriteTags`: (`?[]`) 关注的标签
- `newPost`: (`String[]` -> `Number[]`) 不明?
- `thumbnails`: (`Object`) 已关注用户的作品
- `illust`: (`Object[]`) 插画作品
- `illustId`: (`String` -> `Number`) 作品Id(或者准确来讲是 插画Id?)
- `illustTitle`: (`String`) 插画标题
- `id`: (`String` -> `Number`) 作品Id
- `title`: (`String`) 作品标题
- `illustType`: (`Number`) 作品类型
- `xRestrict`: (`Number`) 不明?
- `restrict`: (`Number`) 不明?
- `sl`: (`Number`) 不明?
- `url`: (`String`) 作品在主页的封面图Url
- `description`: (`String`) 作品说明?
- `tags`: (`String[]`) 标签原始名数组(不带翻译的原始名称)
- `userId`: (`String` -> `Number`) 用户Id
- `userName`: (`String`) 用户名
- `width`: (`Number`) 作品宽度
- `height`: (`Number`) 作品高度
- `pageCount`: (`Number`) 作品页数
- `isBookmarkable`: (`Boolean`) 不明?
- `bookmarkData`: (`?`) 不明?
- `alt`: (`String`) 简略信息
- `isAdContainer`: (`Boolean`) 广告标识?
- `titleCaptionTranslation`: (`Object`) 不明?
- (待完善)
- `urls`: (`Object`) 其他封面图尺寸的链接
- (略)
- `seriesId`: (`?`) 不明?
- `seriesTitle`: (`?`) 不明?
- `profileImageUrl`: (`String`) 用户头像图链接
- `users`: (`Object[]`) 用户信?
- `userId`: (`String` -> `Number`) 用户Id
- `name`: (`String`) 用户名
- `image`: (`String`) 用户头像图链接
- `imageBig`: (`String`) 用户大尺寸头像图链接
- `premium`: (`Boolean`) 是否为Pixiv高级会员
- `isFollowed`: (`Boolean`) 是否关注
- `isMypixiv`: (`Boolean`) 不明?
- `isBlocking`: (`Boolean`) 是否为黑名单?
- `background`: (`?`) 不明?
- `partial`: (`Number`) 不明?
- `tagTranslation`: 标签翻译名
- `key=${标签原始名}`: (`Object`) 标签翻译信息
- `${语言代码}`: (`String`) 对应语言代码的翻译, 不一定有
- `boothItems`: (`Object[]`) 商品信息
- (待完善)
- `sketchLives`: (`Object[]`) 不明?
- (待完善)
- `zoneConfig`: (`Object`) 不明?
- (待完善)

View File

@ -6,7 +6,7 @@
<groupId>net.lamgc</groupId>
<artifactId>ContentGrabbingJi</artifactId>
<version>2.3.0</version>
<version>2.5.2-20200604.3-SNAPSHOT</version>
<repositories>
<repository>
@ -19,7 +19,7 @@
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<maven.compiler.encoding>UTF-8</maven.compiler.encoding>
<mirai.CoreVersion>0.39.4</mirai.CoreVersion>
<mirai.CoreVersion>1.0-RC2-1</mirai.CoreVersion>
<mirai.JaptVersion>1.1.1</mirai.JaptVersion>
<kotlin.version>1.3.71</kotlin.version>
<ktor.version>1.3.2</ktor.version>
@ -86,7 +86,7 @@
<dependency>
<groupId>net.lamgc</groupId>
<artifactId>java-utils</artifactId>
<version>1.1.0_5-SNAPSHOT</version>
<version>1.2.0_20200517.1-SNAPSHOT</version>
</dependency>
<dependency>
@ -172,7 +172,7 @@
<dependency>
<groupId>net.lamgc</groupId>
<artifactId>PixivLoginProxyServer</artifactId>
<version>1.1.0</version>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>com.squareup</groupId>

View File

@ -7,12 +7,15 @@ import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.boot.ApplicationBoot;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.framework.cli.ConsoleMain;
import net.lamgc.cgj.bot.framework.coolq.CQConfig;
import net.lamgc.cgj.bot.framework.mirai.MiraiMain;
import net.lamgc.cgj.pixiv.*;
import net.lamgc.cgj.proxy.PixivAccessProxyServer;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivSearchBuilder;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.plps.PixivLoginProxyServer;
import net.lamgc.utils.base.ArgumentsProperties;
import net.lamgc.utils.base.runner.Argument;
import net.lamgc.utils.base.runner.ArgumentsRunner;
import net.lamgc.utils.base.runner.Command;
@ -20,9 +23,7 @@ import org.apache.commons.io.IOUtils;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.client.CookieStore;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.message.BasicHeader;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@ -30,7 +31,6 @@ import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import java.io.*;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import java.text.ParseException;
import java.text.SimpleDateFormat;
@ -43,77 +43,63 @@ import java.util.zip.ZipOutputStream;
@SpringBootApplication
public class Main {
private final static Logger log = LoggerFactory.getLogger("Main");
private final static Logger log = LoggerFactory.getLogger(Main.class);
private final static File storeDir = new File("store/");
public static CookieStore cookieStore;
private static CookieStore cookieStore;
public static HttpHost proxy;
private static HttpHost proxy;
public static void main(String[] args) throws IOException, ClassNotFoundException {
ArgumentsProperties argsProp = new ArgumentsProperties(args);
if(argsProp.containsKey("proxy")) {
URL proxyUrl = new URL(argsProp.getValue("proxy"));
proxy = new HttpHost(proxyUrl.getHost(), proxyUrl.getPort());
log.info("已启用Http协议代理{}", proxy.toHostString());
} else {
proxy = null;
}
log.trace("ContentGrabbingJi 正在启动...");
log.debug("Args: {}, LogsPath: {}", Arrays.toString(args), System.getProperty("cgj.logsPath"));
log.debug("运行目录: {}", System.getProperty("user.dir"));
if(!storeDir.exists() && !storeDir.mkdirs()) {
log.error("创建文件夹失败!");
}
ApplicationBoot.initialApplication(args);
log.debug("botDataDir: {}", System.getProperty("cgj.botDataDir"));
// TODO: 需要修改参数名了, 大概改成类似于 workerDir这样的吧
if(argsProp.containsKey("cqRootDir")) {
log.info("cqRootDir: {}", argsProp.getValue("cqRootDir"));
System.setProperty("cgj.cqRootDir", argsProp.getValue("cqRootDir"));
} else {
log.warn("未设置cqRootDir, 当前运行目录将作为酷Q机器人所在目录.");
System.setProperty("cgj.cqRootDir", "./");
}
if(argsProp.containsKey("redisAddr")) {
log.info("redisAddress: {}", argsProp.getValue("redisAddr"));
System.setProperty("cgj.redisAddress", argsProp.getValue("redisAddr"));
} else {
log.info("未设置RedisAddress, 将使用默认值连接Redis服务器(127.0.0.1:6379)");
System.setProperty("cgj.redisAddress", "127.0.0.1");
}
File cookieStoreFile = new File("cookies.store");
proxy = BotGlobal.getGlobal().getProxy();
File cookieStoreFile = new File(BotGlobal.getGlobal().getDataStoreDir(), "cookies.store");
if(!cookieStoreFile.exists()) {
log.error("未找到cookies.store文件, 是否启动PixivLoginProxyServer? (yes/no)");
Scanner scanner = new Scanner(System.in);
if(scanner.nextLine().equalsIgnoreCase("yes")) {
startPixivLoginProxyServer();
} else {
System.exit(1);
return;
log.warn("未找到cookies.store文件, 是否启动PixivLoginProxyServer? (yes/no)");
try(Scanner scanner = new Scanner(System.in)) {
if(scanner.nextLine().trim().equalsIgnoreCase("yes")) {
startPixivLoginProxyServer();
} else {
System.exit(1);
return;
}
}
}
ObjectInputStream ois = new ObjectInputStream(new FileInputStream(cookieStoreFile));
cookieStore = (CookieStore) ois.readObject();
BotGlobal.getGlobal().setCookieStore(cookieStore);
ois.close();
log.info("已载入CookieStore");
log.debug("传入参数: {}", Arrays.toString(args));
ArgumentsRunner.run(Main.class, args);
System.exit(0);
}
@Command
public static void botMode(@Argument(name = "args", force = false) String argsStr) {
new MiraiMain().init();
MiraiMain main = new MiraiMain();
main.init();
main.close();
}
@Command
public static void consoleMode() {
ConsoleMain.start();
}
@Command
public static void pluginMode(@Argument(name = "args", force = false) String argsStr) {
if(!System.getProperty("cgj.cqRootDir").endsWith("\\") && !System.getProperty("cgj.cqRootDir").endsWith("/")) {
System.setProperty("cgj.cqRootDir", System.getProperty("cgj.cqRootDir") + "/");
}
log.info("酷Q机器人根目录: {}", System.getProperty("cgj.cqRootDir"));
log.info("酷Q机器人根目录: {}", BotGlobal.getGlobal().getDataStoreDir().getPath());
CQConfig.init();
Pattern pattern = Pattern.compile("/\\s*(\".+?\"|[^:\\s])+((\\s*:\\s*(\".+?\"|[^\\s])+)|)|(\".+?\"|[^\"\\s])+");
Matcher matcher = pattern.matcher(Strings.nullToEmpty(argsStr));
@ -129,7 +115,7 @@ public class Main {
@Command
public static void collectionDownload() throws IOException {
PixivDownload pixivDownload = new PixivDownload(Objects.requireNonNull(cookieStore), proxy);
File outputFile = new File(storeDir, "collection.zip");
File outputFile = new File(getStoreDir(), "collection.zip");
if(!outputFile.exists() && !outputFile.createNewFile()) {
log.error("文件创建失败: " + outputFile.getAbsolutePath());
}
@ -156,10 +142,10 @@ public class Main {
PixivDownload pixivDownload = new PixivDownload(Objects.requireNonNull(cookieStore), proxy);
String date = new SimpleDateFormat("yyyyMMdd").format(new Date());
int id = 1;
File outputFile = new File(storeDir, "recommends-" + date + "-" + id + ".zip");
File outputFile = new File(getStoreDir(), "recommends-" + date + "-" + id + ".zip");
while(outputFile.exists()) {
id++;
outputFile = new File(storeDir, "recommends-" + date + "-" + id + ".zip");
outputFile = new File(getStoreDir(), "recommends-" + date + "-" + id + ".zip");
}
if(!outputFile.createNewFile()) {
@ -223,10 +209,10 @@ public class Main {
}
int id = 1;
File outputFile = new File(storeDir, "ranking" + rankingMode.modeParam + "-" + date + "-" + id + ".zip");
File outputFile = new File(getStoreDir(), "ranking" + rankingMode.modeParam + "-" + date + "-" + id + ".zip");
while(outputFile.exists()) {
id++;
outputFile = new File(storeDir, "ranking" + rankingMode.modeParam + "-" + date + "-" + id + ".zip");
outputFile = new File(getStoreDir(), "ranking" + rankingMode.modeParam + "-" + date + "-" + id + ".zip");
}
if(!outputFile.createNewFile()) {
@ -353,90 +339,20 @@ public class Main {
builder,
PixivURL.getPixivRefererLink(illustId)
);
/*log.info("正在下载...");
List<String> list = PixivDownload.getIllustAllPageDownload(
HttpClientBuilder.create()
.setProxy(proxy)
.build(),
illustId, PixivDownload.PageQuality.ORIGINAL);*/
}
}
/*
if(searchBuilder.getSearchArea().equals(PixivSearchBuilder.SearchArea.TOP)) {
} else {
JsonArray illustsArray = resultBody
.getAsJsonObject(searchBuilder.getSearchArea().jsonKey).getAsJsonArray("data");
log.info("已找到与 {} 相关插图信息:", content);
int count = 1;
for (JsonElement jsonElement : illustsArray) {
JsonObject illustObj = jsonElement.getAsJsonObject();
//TODO: 防止数据内混入无效内容, 需要检查对象是否有illustId
if(!illustObj.has("illustId")) {
continue;
}
int illustId = illustObj.get("illustId").getAsInt();
StringBuilder builder = new StringBuilder("[");
illustObj.get("tags").getAsJsonArray().forEach(el -> builder.append(el.getAsString()).append(", "));
builder.replace(builder.length() - 2, builder.length(), "]");
log.info("({} / {})\n\t作品id: {}, \n\t作者名(作者id): {} ({}), \n\t作品标题: {}, \n\t作品Tags: {}, \n\t作品链接: {}",
count++,
illustsArray.size(),
illustId,
illustObj.get("userName").getAsString(),
illustObj.get("userId").getAsInt(),
illustObj.get("illustTitle").getAsString(),
builder,
PixivURL.getPixivRefererLink(illustId)
);
}
}
*/
}
@Command(defaultCommand = true)
public static void testRun() {
/*loadCookieStoreFromFile();
if(cookieStore == null){
startPixivLoginProxyServer();
}*/
//accessPixivToFile();
//startPixivAccessProxyServer();
//saveCookieStoreToFile();
log.info("这里啥都没有哟w");
}
private static void loadCookieStoreFromFile() throws IOException {
log.info("正在加载CookieStore...");
File storeFile = new File("./cookies.store");
if(!storeFile.exists()){
log.info("未找到CookieStore, 跳过加载.");
return;
}
ObjectInputStream stream = new ObjectInputStream(new FileInputStream(storeFile));
Object result;
try {
result = stream.readObject();
} catch (ClassNotFoundException e) {
log.error("加载出错", e);
return;
}
cookieStore = (CookieStore) result;
cookieStore.getCookies().forEach(cookie -> log.debug(cookie.getName() + ": " + cookie.getValue() + ", isExpired: " + cookie.isExpired(new Date())));
log.info("CookieStore加载完成.");
}
private static void saveCookieStoreToFile() throws IOException {
log.info("正在保存CookieStore...");
File outputFile = new File("./cookies.store");
if(!outputFile.exists() && !outputFile.delete() && !outputFile.createNewFile()){
File outputFile = new File(BotGlobal.getGlobal().getDataStoreDir(), "cookies.store");
if(!outputFile.exists() && !outputFile.createNewFile()){
log.error("保存CookieStore失败.");
return;
}
@ -461,96 +377,34 @@ public class Main {
proxyServerStartThread.setName("LoginProxyServerThread");
proxyServerStartThread.start();
//System.console().readLine();
Scanner scanner = new Scanner(System.in);
log.info("登录完成后, 使用\"done\"命令结束登录过程.");
while(true) {
if (scanner.nextLine().equalsIgnoreCase("done")) {
log.info("关闭PLPS服务器...");
proxyServer.close();
cookieStore = proxyServer.getCookieStore();
try {
log.info("正在保存CookieStore...");
saveCookieStoreToFile();
log.info("CookieStore保存完成.");
} catch (IOException e) {
log.error("CookieStore保存时发生异常, 本次CookieStore仅可在本次运行使用.", e);
try(Scanner scanner = new Scanner(System.in)) {
while(true) {
if (scanner.nextLine().equalsIgnoreCase("done")) {
log.info("关闭PLPS服务器...");
proxyServer.close();
cookieStore = proxyServer.getCookieStore();
try {
log.info("正在保存CookieStore...");
saveCookieStoreToFile();
log.info("CookieStore保存完成.");
} catch (IOException e) {
log.error("CookieStore保存时发生异常, 本次CookieStore仅可在本次运行使用.", e);
}
break;
} else {
log.warn("要结束登录过程, 请使用\"done\"命令.");
}
break;
} else {
log.warn("要结束登录过程, 请使用\"done\"命令.");
}
}
}
private static void accessPixivToFile() throws IOException {
File cookieStoreFile = new File("./cookie.txt");
if (!cookieStoreFile.exists() && !cookieStoreFile.createNewFile()) {
log.info("Cookie文件存储失败");
private static File getStoreDir() {
if(!storeDir.exists() && !storeDir.mkdirs()) {
log.error("创建文件夹失败!");
}
/*log.info("正在写出Cookie, Cookie count: " + cookieStore.getCookies().size());
FileWriter cookieWriter = new FileWriter(cookieStoreFile);
cookieStore.getCookies().forEach(cookie -> {
try {
StringBuilder sb = new StringBuilder().append(cookie.getName()).append(" = ").append(cookie.getValue());
log.info("正在导出Cookie: " + sb.toString());
cookieWriter.append(sb.toString()).append("\n").flush();
} catch (IOException e) {
e.printStackTrace();
}
});
log.info("Cookie写出完成");*/
log.info("尝试通过捕获的Cookie访问Pixiv...");
HttpClient httpClient = new PixivSession(new HttpHost("127.0.0.1", 1080), cookieStore).getHttpClient();
HttpGet request = new HttpGet("https://www.pixiv.net");
request.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:73.0) Gecko/20100101 Firefox/73.0");
request.addHeader(new BasicHeader("accept-encoding", "gzip, deflate, br"));
request.addHeader(new BasicHeader("accept-language", "zh-CN,zh;q=0.9"));
StringBuilder cookieBuilder = new StringBuilder();
cookieStore.getCookies().forEach(cookie -> {
if(cookie.isExpired(new Date())){
return;
}
cookieBuilder.append(cookie.getName()).append("=").append(cookie.getValue()).append("; ");
});
request.addHeader("cookie", cookieBuilder.toString());
HttpResponse response = httpClient.execute(request);
log.info("正在写入文件...");
File outFile = new File("./pixiv.html");
if (outFile.createNewFile() && !outFile.exists()) {
log.info("文件创建失败!");
}else {
new FileWriter(outFile).append(EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8)).flush();
}
Pixiv pixiv = new Pixiv(httpClient);
pixiv.getRecommend().forEach(illustMap -> {
StringBuilder builder = new StringBuilder();
illustMap.forEach((key, value) -> builder.append(key).append(": ").append(value).append("\n"));
try {
builder.append("download Link: ").append(Arrays.toString(pixiv.getAllDownloadLink(Integer.parseInt(illustMap.get(Pixiv.ATTR_ILLUST_ID)))));
} catch (IOException e) {
log.error("获取下载链接时出错!", e);
}
log.info(builder.append("\n").toString());
});
return storeDir;
}
private static void startPixivAccessProxyServer(){
log.info("正在启动访问代理服务器, 将浏览器相关缓存清空后, 使用浏览器进行访问以尝试Cookie正确性.");
final PixivAccessProxyServer accessProxyServer = new PixivAccessProxyServer(cookieStore, new ProxyConfig(ProxyType.SOCKS5, "127.0.0.1", 1080));
Thread accessProxyServerThread = new Thread(() -> {
log.info("正在启动PAPS...");
accessProxyServer.start(1007);
log.info("PAPS已关闭.");
});
accessProxyServerThread.setName("AccessProxyServerThread");
accessProxyServerThread.start();
new Scanner(System.in).nextLine();
log.info("关闭PAPS服务器...");
accessProxyServer.close();
}
}

View File

@ -1,57 +0,0 @@
package net.lamgc.cgj.bot;
import net.lz1998.cq.robot.CoolQ;
import org.apache.http.client.methods.HttpGet;
import java.util.Random;
import java.util.Timer;
import java.util.TimerTask;
public class AutoArtworksSender {
private final CoolQ CQ;
private final ReceiveType receiveType;
private final long targetReceiveId;
private Timer timer = new Timer();
private TimerTask task = new TimerTask() {
@Override
public void run() {
HttpGet request = new HttpGet();
// https://api.imjad.cn/pixiv/v2/?type=tags
}
};
public AutoArtworksSender(CoolQ cq, ReceiveType receiveType, long receiveId) {
this.CQ = cq;
this.receiveType = receiveType;
this.targetReceiveId = receiveId;
}
public void reset(long time) {
if(time <= 0) {
timer.schedule(task, new Random().nextInt(10 * 60 * 60 * 1000) + 7200000L); //2H ~ 12H
} else {
timer.schedule(task, time);
}
}
public void sendMessage(String message, boolean auto_escape) {
switch (receiveType) {
case GROUP:
CQ.sendGroupMsg(targetReceiveId, message, auto_escape);
break;
case Discuss:
CQ.sendDiscussMsg(targetReceiveId, message, auto_escape);
break;
case PRIVATE:
CQ.sendPrivateMsg(targetReceiveId, message, auto_escape);
break;
}
}
public enum ReceiveType {
PRIVATE, GROUP, Discuss
}
}

View File

@ -9,7 +9,7 @@ import java.util.Objects;
*/
public abstract class AutoSender {
private MessageSender messageSender;
private final MessageSender messageSender;
/**
* 构造一个自动发送器

View File

@ -5,6 +5,7 @@ import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonObject;
import com.google.gson.reflect.TypeToken;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lamgc.cgj.pixiv.PixivDownload;
@ -20,11 +21,9 @@ import java.util.*;
public class BotAdminCommandProcess {
private final static Logger log = LoggerFactory.getLogger(BotAdminCommandProcess.class.getSimpleName());
private final static Logger log = LoggerFactory.getLogger(BotAdminCommandProcess.class);
private final static File globalPropFile = new File("global.properties");
private final static File pushListFile = new File("pushList.json");
private final static File pushListFile = new File(BotGlobal.getGlobal().getDataStoreDir(), "pushList.json");
private final static Hashtable<Long, JsonObject> pushInfoMap = new Hashtable<>();
@ -39,59 +38,47 @@ public class BotAdminCommandProcess {
}
@Command
public static String setGlobalProperty(@Argument(name = "key") String key, @Argument(name = "value") String value, @Argument(name = "save", force = false) boolean saveNow) {
String lastValue = BotCommandProcess.globalProp.getProperty(key);
BotCommandProcess.globalProp.setProperty(key, Strings.nullToEmpty(value));
if(saveNow) {
saveGlobalProperties();
public static String setProperty(
@Argument(name = "group", force = false) long groupId,
@Argument(name = "key") String key,
@Argument(name = "value") String value
) {
if(Strings.isNullOrEmpty(key)) {
return "未选择配置项key.";
}
return "全局配置项 " + key + " 现已设置为: " + value + " (设置前的值: " + lastValue + ")";
String lastValue = SettingProperties.setProperty(groupId, key, value.equals("null") ? null : value);
return (groupId <= 0 ? "已更改全局配置 " : "已更改群组 " + groupId + " 配置 ") +
key + " 的值: '" + value + "' (原配置值: '" + lastValue + "')";
}
@Command
public static String getGlobalProperty(@Argument(name = "key") String key) {
return "全局配置项 " + key + " 当前值: " + BotCommandProcess.globalProp.getProperty(key, "(Empty)");
public static String getProperty(
@Argument(name = "group", force = false) long groupId,
@Argument(name = "key") String key
) {
if(Strings.isNullOrEmpty(key)) {
return "未选择配置项key.";
}
return (groupId <= 0 ? "全局配置 " : "群组 " + groupId + " 配置 ") +
key + " 设定值: '" + SettingProperties.getProperty(groupId, key, "(empty)") + "'";
}
@Command
public static String saveGlobalProperties() {
log.info("正在保存全局配置文件...");
try {
if(!globalPropFile.exists()) {
if(!globalPropFile.createNewFile()) {
log.error("全局配置项文件保存失败!({})", "文件创建失败");
return "全局配置项文件保存失败!";
}
}
BotCommandProcess.globalProp.store(new FileOutputStream(globalPropFile), "");
log.info("全局配置文件保存成功!");
return "保存全局配置文件 - 操作已完成.";
} catch (IOException e) {
log.error("全局配置项文件保存失败!", e);
return "全局配置项文件保存失败!";
}
public static String saveProperties() {
log.info("正在保存配置文件...");
SettingProperties.saveProperties();
log.info("配置文件保存操作已完成.");
return "保存配置 - 操作已完成.";
}
@Command
public static String loadGlobalProperties(@Argument(name = "reload", force = false) boolean reload) {
Properties cache = new Properties();
if(!globalPropFile.exists()) {
return "未找到全局配置文件, 无法重载";
}
try(Reader reader = new BufferedReader(new FileReader(globalPropFile))) {
cache.load(reader);
} catch (IOException e) {
log.error("重载全局配置文件时发生异常", e);
return "加载全局配置文件时发生错误!";
}
public static String loadProperties(@Argument(name = "reload", force = false) boolean reload) {
if(reload) {
BotCommandProcess.globalProp.clear();
SettingProperties.clearProperties();
}
BotCommandProcess.globalProp.putAll(cache);
return "全局配置文件重载完成.";
SettingProperties.loadProperties();
return "操作已完成.";
}
@Command
@ -125,6 +112,30 @@ public class BotAdminCommandProcess {
@Argument(name = "type", force = false, defaultValue = "ILLUST") String rankingContentType,
@Argument(name = "original", force = false, defaultValue = "false") boolean original
) {
if(minTime <= 0 || floatTime <= 0) {
return "时间不能为0或负数";
} else if(rankingStart <= 0 || rankingStop - rankingStart <= 0) {
return "排行榜范围选取错误!";
}
PixivURL.RankingContentType type;
PixivURL.RankingMode mode;
try {
type = PixivURL.RankingContentType.valueOf("TYPE_" + rankingContentType.toUpperCase());
} catch(IllegalArgumentException e) {
return "无效的排行榜类型参数!";
}
try {
mode = PixivURL.RankingMode.valueOf("MODE_" + rankingMode.toUpperCase());
} catch(IllegalArgumentException e) {
return "无效的排行榜模式参数!";
}
if(!type.isSupportedMode(mode)) {
return "不兼容的排行榜模式与类型!";
}
long group = groupId <= 0 ? fromGroup : groupId;
JsonObject setting = new JsonObject();
setting.addProperty(RANKING_SETTING_TIME_MIN, minTime);
@ -141,7 +152,8 @@ public class BotAdminCommandProcess {
removePushGroup(fromGroup, groupId);
}
log.info("正在增加Timer...(Setting: {})", setting);
log.info("群组 {} 新推送配置: {}", group, setting);
log.info("正在增加Timer...");
pushInfoMap.put(group, setting);
addPushTimer(group, setting);
return "已在 " + group + " 开启定时推送功能。";
@ -171,8 +183,8 @@ public class BotAdminCommandProcess {
@Command
public static String savePushList() {
try {
if(!pushListFile.exists()) {
pushListFile.createNewFile();
if(!pushListFile.exists() && !pushListFile.createNewFile()) {
throw new IOException("文件夹创建失败!(Path: " + pushListFile.getPath() + ")");
}
} catch (IOException e) {
log.error("PushList.json文件创建失败", e);

View File

@ -2,20 +2,21 @@ package net.lamgc.cgj.bot;
import com.google.common.base.Strings;
import com.google.common.base.Throwables;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import com.google.gson.*;
import io.netty.handler.codec.http.HttpHeaderNames;
import net.lamgc.cgj.Main;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.cache.*;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.event.BufferMessageEvent;
import net.lamgc.cgj.bot.sort.PreLoadDataComparator;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivSearchBuilder;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.cgj.pixiv.PixivDownload.PageQuality;
import net.lamgc.cgj.pixiv.PixivURL.RankingContentType;
import net.lamgc.cgj.pixiv.PixivURL.RankingMode;
import net.lamgc.cgj.util.URLs;
import net.lamgc.utils.base.runner.Argument;
import net.lamgc.utils.base.runner.Command;
import net.lamgc.utils.event.EventExecutor;
import net.lz1998.cq.utils.CQCode;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
@ -25,75 +26,73 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
@SuppressWarnings({"SynchronizationOnLocalVariableOrMethodParameter", "SameParameterValue"})
public class BotCommandProcess {
private final static PixivDownload pixivDownload = new PixivDownload(Main.cookieStore, Main.proxy);
private final static PixivDownload pixivDownload =
new PixivDownload(BotGlobal.getGlobal().getCookieStore(), BotGlobal.getGlobal().getProxy());
private final static Logger log = LoggerFactory.getLogger(BotCommandProcess.class.getSimpleName());
private final static Logger log = LoggerFactory.getLogger(BotCommandProcess.class);
private final static File imageStoreDir = new File(System.getProperty("cgj.cqRootDir"), "data/image/cgj/");
public final static Properties globalProp = new Properties();
private final static File imageStoreDir = new File(BotGlobal.getGlobal().getDataStoreDir(), "data/image/cgj/");
private final static Gson gson = new GsonBuilder()
.serializeNulls()
.create();
/* -------------------- 缓存 -------------------- */
private final static Hashtable<String, File> imageCache = new Hashtable<>();
private final static CacheStore<JsonElement> illustInfoCache = new JsonRedisCacheStore(BotEventHandler.redisServer, "illustInfo", gson);
private final static CacheStore<JsonElement> illustPreLoadDataCache = new HotDataCacheStore<>(
new JsonRedisCacheStore(BotEventHandler.redisServer, "illustPreLoadData", gson),
new LocalHashCacheStore<>(), 3600000, 900000);
private final static CacheStore<JsonElement> searchBodyCache = new JsonRedisCacheStore(BotEventHandler.redisServer, "searchBody", gson);
private final static CacheStore<List<JsonObject>> rankingCache = new JsonObjectRedisListCacheStore(BotEventHandler.redisServer, "ranking", gson);
private final static CacheStore<List<String>> pagesCache = new StringListRedisCacheStore(BotEventHandler.redisServer, "imagePages");
public final static CacheStore<JsonElement> reportStore = new JsonRedisCacheStore(BotEventHandler.redisServer, "report", gson);
/**
* 图片异步缓存执行器
* 作品信息缓存 - 不过期
*/
private final static EventExecutor imageCacheExecutor = new EventExecutor(new ThreadPoolExecutor(
Runtime.getRuntime().availableProcessors() >= 2 ? 2 : 1,
(int) Math.ceil(Runtime.getRuntime().availableProcessors() / 2F),
5L, TimeUnit.SECONDS,
new LinkedBlockingQueue<>(128),
new ThreadFactoryBuilder()
.setNameFormat("imageCacheThread-%d")
.build(),
new ThreadPoolExecutor.DiscardOldestPolicy()
));
private final static CacheStore<JsonElement> illustInfoCache =
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(), "illustInfo", gson);
/**
* 作品信息预加载数据 - 有效期 2 小时, 本地缓存有效期1 ± 0.25
*/
private final static CacheStore<JsonElement> illustPreLoadDataCache =
CacheStoreUtils.hashLocalHotDataStore(
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(), "illustPreLoadData", gson),
3600000, 900000);
/**
* 搜索内容缓存, 有效期 2 小时
*/
private final static CacheStore<JsonElement> searchBodyCache =
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(), "searchBody", gson);
/**
* 排行榜缓存, 不过期
*/
private final static CacheStore<List<JsonObject>> rankingCache =
new JsonObjectRedisListCacheStore(BotGlobal.getGlobal().getRedisServer(), "ranking", gson);
/**
* 作品页面下载链接缓存 - 不过期
*/
private final static CacheStore<List<String>> pagesCache =
new StringListRedisCacheStore(BotGlobal.getGlobal().getRedisServer(), "imagePages");
/**
* 作品报告存储 - 不过期
*/
public final static CacheStore<JsonElement> reportStore =
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(), "report", gson);
private final static RankingUpdateTimer updateTimer = new RankingUpdateTimer();
public static void initialize() {
log.info("正在初始化...");
File globalPropFile = new File("./global.properties");
if(globalPropFile.exists() && globalPropFile.isFile()) {
log.info("正在加载全局配置文件...");
try {
globalProp.load(new FileInputStream(globalPropFile));
log.info("全局配置文件加载完成.");
} catch (IOException e) {
log.error("加载全局配置文件时发生异常", e);
}
} else {
log.info("未找到全局配置文件,跳过加载.");
}
try {
imageCacheExecutor.addHandler(new ImageCacheHandler());
} catch (IllegalAccessException e) {
e.printStackTrace();
}
SettingProperties.loadProperties();
updateTimer.schedule(null);
log.info("初始化完成.");
@ -111,74 +110,85 @@ public class BotCommandProcess {
@Command(defaultCommand = true)
public static String help() {
StringBuilder helpStrBuilder = new StringBuilder();
helpStrBuilder.append("CGJ Bot使用指南").append("\n");
helpStrBuilder.append("使用方法:.cgj <命令> [参数...]").append("\n");
helpStrBuilder.append("例如查询作品信息功能:").append("\n");
helpStrBuilder.append(".cgj artwork -id 80846159").append("\n");
helpStrBuilder.append("目前可用的命令:").append("\n");
helpStrBuilder.append("\t").append("ranking - 获取今天或指定日期排行榜的前10名作品").append("\n");
helpStrBuilder.append("\t\t").append("-date - 指定查询日期(年-月-日)").append("\n");
helpStrBuilder.append("\t\t").append("-type - 排行榜类型(illust/插画, ugoira/动图, manga/漫画)").append("\n");
helpStrBuilder.append("\t").append("search - 搜索指定关键词并显示前10个作品").append("\n");
helpStrBuilder.append("\t\t").append("-content - 搜索内容").append("\n");
helpStrBuilder.append("\t").append("link - 获取作品的Pixiv页面").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品id").append("\n");
helpStrBuilder.append("\t").append("info - 获取Pixiv作品信息").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品id").append("\n");
helpStrBuilder.append("\t").append("image - 获取指定作品的图片").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品id").append("\n");
helpStrBuilder.append("\t\t").append("-quality - 图片质量(original/原图 regular/预览图)").append("\n");
helpStrBuilder.append("\t\t").append("-page - 页数").append("\n");
helpStrBuilder.append("\t").append("report - 报告不当作品").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品Id").append("\n");
helpStrBuilder.append("\t\t").append("-msg - 报告原因").append("\n");
return helpStrBuilder.toString();
return "CGJ Bot使用指南" + "\n" +
"使用方法:.cgj <命令> [参数...]" + "\n" +
"例如查询作品信息功能:" + "\n" +
".cgj info -id 80846159" + "\n" +
"目前可用的命令:" + "\n" +
"\t" + "ranking - 获取今天或指定日期排行榜的前10名作品" + "\n" +
"\t\t" + "-date - 指定查询日期(年-月-日)" + "\n" +
"\t\t" + "-type - 排行榜类型(illust/插画, ugoira/动图, manga/漫画)" + "\n" +
"\t" + "search - 搜索指定关键词并显示前10个作品" + "\n" +
"\t\t" + "-content - 搜索内容" + "\n" +
"\t" + "link - 获取作品的Pixiv页面" + "\n" +
"\t\t" + "-id - 作品id" + "\n" +
"\t" + "info - 获取Pixiv作品信息" + "\n" +
"\t\t" + "-id - 作品id" + "\n" +
"\t" + "image - 获取指定作品的图片" + "\n" +
"\t\t" + "-id - 作品id" + "\n" +
"\t\t" + "-quality - 图片质量(original/原图 regular/预览图)" + "\n" +
"\t\t" + "-page - 页数" + "\n" +
"\t" + "report - 报告不当作品" + "\n" +
"\t\t" + "-id - 作品Id" + "\n" +
"\t\t" + "-msg - 报告原因" + "\n";
}
/**
* 作品信息查询
* @param fromGroup 来源群(系统提供)
* @param illustId 作品Id
* @return 返回作品信息
*/
@Command(commandName = "info")
public static String artworkInfo(@Argument(name = "id") int illustId) {
public static String artworkInfo(@Argument(name = "$fromGroup") long fromGroup, @Argument(name = "id") int illustId) {
if(illustId <= 0) {
return "错误的作品id";
return "这个作品Id是错误的!";
}
try {
if(isNoSafe(illustId, globalProp, false) || isReported(illustId)) {
if(isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false) || isReported(illustId)) {
return "阅览禁止:该作品已被封印!!";
}
JsonObject illustPreLoadData = getIllustPreLoadData(illustId, false);
StringBuilder builder = new StringBuilder("---------------- 作品信息 ----------------\n");
builder.append("作品Id: ").append(illustId).append("\n");
builder.append("作品标题:").append(illustPreLoadData.get("illustTitle").getAsString()).append("\n");
builder.append("作者(作者Id)").append(illustPreLoadData.get("userName").getAsString())
.append("(").append(illustPreLoadData.get("userId").getAsInt()).append(")\n");
builder.append("点赞数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.LIKE.attrName).getAsInt()).append("\n");
builder.append("收藏数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.BOOKMARK.attrName).getAsInt()).append("\n");
builder.append("围观数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.VIEW.attrName).getAsInt()).append("\n");
builder.append("评论数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.COMMENT.attrName).getAsInt()).append("\n");
builder.append("页数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.PAGE.attrName).getAsInt()).append("\n");
builder.append("---------------- 作品图片 ----------------\n");
builder.append(getImageById(illustId, PixivDownload.PageQuality.REGULAR, 1)).append("\n");
builder.append("使用 \".cgj image -id ")
.append(illustId)
.append("\" 获取原图。\n如有不当作品使用\".cgj report -id ")
.append(illustId).append("\"向色图姬反馈。");
return builder.toString();
// 在 Java 6 开始, 编译器会将用'+'进行的字符串拼接将自动转换成StringBuilder拼接
return "色图姬帮你了解了这个作品的信息!\n" + "---------------- 作品信息 ----------------" +
"\n作品Id: " + illustId +
"\n作品标题" + illustPreLoadData.get("illustTitle").getAsString() +
"\n作者(作者Id)" + illustPreLoadData.get("userName").getAsString() +
"(" + illustPreLoadData.get("userId").getAsInt() + ")" +
"\n点赞数:" + illustPreLoadData.get(PreLoadDataComparator.Attribute.LIKE.attrName).getAsInt() +
"\n收藏数" + illustPreLoadData.get(PreLoadDataComparator.Attribute.BOOKMARK.attrName).getAsInt() +
"\n围观数:" + illustPreLoadData.get(PreLoadDataComparator.Attribute.VIEW.attrName).getAsInt() +
"\n评论数" + illustPreLoadData.get(PreLoadDataComparator.Attribute.COMMENT.attrName).getAsInt() +
"\n页数" + illustPreLoadData.get(PreLoadDataComparator.Attribute.PAGE.attrName).getAsInt() + "" +
"\n作品链接" + artworksLink(fromGroup, illustId) + "\n" +
"---------------- 作品图片 ----------------\n" +
getImageById(fromGroup, illustId, PageQuality.REGULAR, 1) + "\n" +
"使用 \".cgj image -id " +
illustId +
"\" 获取原图。\n如有不当作品可使用\".cgj report -id " +
illustId + "\"向色图姬反馈。";
} catch (IOException e) {
e.printStackTrace();
}
return "尚未支持";
}
/**
* 排行榜命令
* @param fromGroup 来源群(系统提供)
* @param queryTime 查询时间, 格式: 年-月-日
* @param force 是否强制查询, 当主动提供的时间不在查询范围时, 是否强制查询, 仅系统可用
* @param contentMode 内容模式
* @param contentType 排行榜类型
* @return 返回排行榜信息
*/
@Command
public static String ranking(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(force = false, name = "date") Date queryTime,
@Argument(force = false, name = "force") boolean force,
@Argument(force = false, name = "mode", defaultValue = "DAILY") String contentMode,
@Argument(force = false, name = "type", defaultValue = "ILLUST") String contentType
) {
@ -195,7 +205,7 @@ public class BotCommandProcess {
}
queryDate = gregorianCalendar.getTime();
} else {
if(new Date().before(queryDate)) {
if(new Date().before(queryDate) && !force) {
log.warn("查询的日期过早, 无法查询排行榜.");
return "查询日期过早, 暂未更新指定日期的排行榜!";
}
@ -229,9 +239,10 @@ public class BotCommandProcess {
try {
int index = 0;
int itemLimit = 10;
String itemLimitPropertyKey = "ranking.ItemCountLimit";
String itemLimitPropertyKey = "ranking.itemCountLimit";
try {
itemLimit = Integer.parseInt(globalProp.getProperty(itemLimitPropertyKey, "10"));
itemLimit = Integer.parseInt(SettingProperties
.getProperty(fromGroup, itemLimitPropertyKey, "10"));
} catch(NumberFormatException e) {
log.warn("配置项 {} 的参数值格式有误!", itemLimitPropertyKey);
}
@ -239,12 +250,12 @@ public class BotCommandProcess {
int imageLimit = 3;
String imageLimitPropertyKey = "ranking.imageCountLimit";
try {
imageLimit = Integer.parseInt(globalProp.getProperty(imageLimitPropertyKey, "3"));
imageLimit = Integer.parseInt(
SettingProperties.getProperty(fromGroup, imageLimitPropertyKey, "3"));
} catch(NumberFormatException e) {
log.warn("配置项 {} 的参数值格式有误!", imageLimitPropertyKey);
}
//TODO(LamGC, 2020.4.11): 将JsonRedisCacheStore更改为使用Redis的List集合, 以提高性能
List<JsonObject> rankingInfoList = getRankingInfoByCache(type, mode, queryDate, 1, Math.max(0, itemLimit), false);
if(rankingInfoList.isEmpty()) {
return "无法查询排行榜,可能排行榜尚未更新。";
@ -261,7 +272,7 @@ public class BotCommandProcess {
resultBuilder.append(rank).append(". (id: ").append(illustId).append(") ").append(title)
.append("(Author: ").append(authorName).append(",").append(authorId).append(") ").append(pagesCount).append("p.\n");
if (index <= imageLimit) {
resultBuilder.append(getImageById(illustId, PixivDownload.PageQuality.REGULAR, 1)).append("\n");
resultBuilder.append(getImageById(fromGroup, illustId, PixivDownload.PageQuality.REGULAR, 1)).append("\n");
}
}
} catch (IOException e) {
@ -271,14 +282,34 @@ public class BotCommandProcess {
return resultBuilder.append("如查询当前时间获取到昨天时间,则今日排名榜未更新。\n如有不当作品可使用\".cgj report -id 作品id\"向色图姬反馈。").toString();
}
/**
* 查询指定作者的作品(尚未完成)
* @return 返回作者信息和部分作品
*/
@Command(commandName = "userArt")
public static String userArtworks() {
return "功能未完成";
}
/**
* 随机获取一副作品
*/
@Command(commandName = "random")
public static String randomImage() {
BufferMessageEvent event = new BufferMessageEvent();
RandomRankingArtworksSender artworksSender =
new RandomRankingArtworksSender(event, 1, 200,
RankingMode.MODE_MALE,
RankingContentType.TYPE_ALL,
PageQuality.ORIGINAL);
artworksSender.send();
return event.getBufferMessage();
}
/**
* 搜索命令
* @param fromGroup 来源群(系统提供)
* @param content 搜索内容
* @param type 搜索类型
* @param area 搜索区域
@ -290,13 +321,15 @@ public class BotCommandProcess {
* @throws IOException 当搜索发生异常时抛出
*/
@Command
public static String search(@Argument(name = "content") String content,
@Argument(name = "type", force = false) String type,
@Argument(name = "area", force = false) String area,
@Argument(name = "in", force = false) String includeKeywords,
@Argument(name = "ex", force = false) String excludeKeywords,
@Argument(name = "contentOption", force = false) String contentOption,
@Argument(name = "page", force = false, defaultValue = "1") int pagesIndex
public static String search(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "content") String content,
@Argument(name = "type", force = false) String type,
@Argument(name = "area", force = false) String area,
@Argument(name = "in", force = false) String includeKeywords,
@Argument(name = "ex", force = false) String excludeKeywords,
@Argument(name = "contentOption", force = false) String contentOption,
@Argument(name = "page", force = false, defaultValue = "1") int pagesIndex
) throws IOException {
log.info("正在执行搜索...");
PixivSearchBuilder searchBuilder = new PixivSearchBuilder(Strings.isNullOrEmpty(content) ? "" : content);
@ -360,7 +393,8 @@ public class BotCommandProcess {
}
long expire = 7200 * 1000;
String propValue = globalProp.getProperty("cache.searchBody.expire", "7200000");
String propValue = SettingProperties
.getProperty(SettingProperties.GLOBAL, "cache.searchBody.expire", "7200000");
try {
expire = Long.parseLong(propValue);
} catch (Exception e) {
@ -385,7 +419,8 @@ public class BotCommandProcess {
log.debug("正在处理信息...");
int limit = 8;
try {
limit = Integer.parseInt(globalProp.getProperty("search.ItemCountLimit", "8"));
limit = Integer.parseInt(SettingProperties.
getProperty(fromGroup, "search.itemCountLimit", "8"));
} catch (Exception e) {
log.warn("参数转换异常!将使用默认值(" + limit + ")", e);
}
@ -414,7 +449,7 @@ public class BotCommandProcess {
StringBuilder builder = new StringBuilder("[");
illustObj.get("tags").getAsJsonArray().forEach(el -> builder.append(el.getAsString()).append(", "));
builder.replace(builder.length() - 2, builder.length(), "]");
log.debug("{} ({} / {})\n\t作品id: {}, \n\t作者名(作者id): {} ({}), \n\t作品标题: {}, \n\t作品Tags: {}, \n\t页数: {}, \n\t作品链接: {}",
log.debug("{} ({} / {})\n\t作品id: {}, \n\t作者名(作者id): {} ({}), \n\t作品标题: {}, \n\t作品Tags: {}, \n\t页数: {}, \n\t作品链接: {}",
searchArea.name(),
count,
illustsList.size(),
@ -427,10 +462,8 @@ public class BotCommandProcess {
PixivURL.getPixivRefererLink(illustId)
);
//pageCount
String imageMsg = getImageById(illustId, PixivDownload.PageQuality.REGULAR, 1);
if (isNoSafe(illustId, globalProp, true)) {
String imageMsg = getImageById(fromGroup, illustId, PixivDownload.PageQuality.REGULAR, 1);
if (isNoSafe(illustId, SettingProperties.getProperties(fromGroup), true)) {
log.warn("作品Id {} 为R-18作品, 跳过.", illustId);
continue;
} else if(isReported(illustId)) {
@ -438,10 +471,20 @@ public class BotCommandProcess {
continue;
}
result.append(searchArea.name()).append(" (").append(count).append(" / ").append(limit).append(")\n\t作品id: ").append(illustId)
JsonObject illustPreLoadData = getIllustPreLoadData(illustId, false);
result.append(searchArea.name()).append(" (").append(count).append(" / ")
.append(limit).append(")\n\t作品id: ").append(illustId)
.append(", \n\t作者名: ").append(illustObj.get("userName").getAsString())
.append("\n\t作品标题: ").append(illustObj.get("illustTitle").getAsString())
.append("\n\t作品页数: ").append(illustObj.get("pageCount").getAsInt())
.append("\n\t作品页数: ").append(illustObj.get("pageCount").getAsInt()).append("")
.append("\n\t点赞数")
.append(illustPreLoadData.get(PreLoadDataComparator.Attribute.LIKE.attrName).getAsInt())
.append("\n\t收藏数")
.append(illustPreLoadData.get(PreLoadDataComparator.Attribute.BOOKMARK.attrName).getAsInt())
.append("\n\t围观数")
.append(illustPreLoadData.get(PreLoadDataComparator.Attribute.VIEW.attrName).getAsInt())
.append("\n\t评论数")
.append(illustPreLoadData.get(PreLoadDataComparator.Attribute.COMMENT.attrName).getAsInt())
.append("\n").append(imageMsg).append("\n");
count++;
}
@ -452,9 +495,22 @@ public class BotCommandProcess {
return Strings.nullToEmpty(result.toString()) + "预览图片并非原图,使用“.cgj image -id 作品id”获取原图\n如有不当作品可使用\".cgj report -id 作品id\"向色图姬反馈。";
}
/**
* 获取作品页面的下载链接
* @param illustId 作品Id
* @param quality 画质类型
* @return 返回作品所有页面在Pixiv的下载链接(有防盗链, 考虑要不要设置镜像站)
*/
@Command(commandName = "pages")
public static String getPagesList(@Argument(name = "id") int illustId, @Argument(name = "quality", force = false) PixivDownload.PageQuality quality) {
public static String getPagesList(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "id") int illustId,
@Argument(name = "quality", force = false) PixivDownload.PageQuality quality) {
try {
if(isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) {
log.warn("来源群 {} 查询的作品Id {} 为R18作品, 根据配置设定, 屏蔽该作品.", fromGroup, illustId);
return "该作品已被封印!";
}
List<String> pagesList = PixivDownload.getIllustAllPageDownload(pixivDownload.getHttpClient(), pixivDownload.getCookieStore(), illustId, quality);
StringBuilder builder = new StringBuilder("作品ID ").append(illustId).append(" 共有").append(pagesList.size()).append("页:").append("\n");
int index = 0;
@ -468,10 +524,16 @@ public class BotCommandProcess {
}
}
/**
* 获取作品链接
* @param fromGroup 来源群(系统提供)
* @param illustId 作品Id
* @return 返回作品在Pixiv的链接
*/
@Command(commandName = "link")
public static String artworksLink(@Argument(name = "id") int illustId) {
public static String artworksLink(@Argument(name = "$fromGroup") long fromGroup, @Argument(name = "id") int illustId) {
try {
if (isNoSafe(illustId, globalProp, false)) {
if (isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) {
log.warn("作品Id {} 已被屏蔽.", illustId);
return "由于相关设置,该作品已被屏蔽!";
} else if(isReported(illustId)) {
@ -487,16 +549,33 @@ public class BotCommandProcess {
/**
* 通过illustId获取作品图片
* @param fromGroup 来源群(系统提供)
* @param illustId 作品Id
* @param quality 图片质量
* @param pageIndex 指定页面索引, 从1开始
* @return 如果成功, 返回BotCode, 否则返回错误信息.
*/
@Command(commandName = "image")
public static String getImageById(@Argument(name = "id") int illustId,
@Argument(name = "quality", force = false) PixivDownload.PageQuality quality,
@Argument(name = "page", force = false, defaultValue = "1") int pageIndex) {
public static String getImageById(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "id") int illustId,
@Argument(name = "quality", force = false) PixivDownload.PageQuality quality,
@Argument(name = "page", force = false, defaultValue = "1") int pageIndex) {
log.debug("IllustId: {}, Quality: {}, PageIndex: {}", illustId, quality.name(), pageIndex);
try {
if (isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) {
log.warn("作品 {} 存在R-18内容且设置\"image.allowR18\"为false将屏蔽该作品不发送.", illustId);
return "(根据设置,该作品已被屏蔽!)";
} else if(isReported(illustId)) {
log.warn("作品Id {} 被报告, 正在等待审核, 跳过该作品.", illustId);
return "(该作品已被封印)";
}
} catch (IOException e) {
log.warn("作品信息无法获取!", e);
return "(发生网络异常,无法获取图片!)";
}
List<String> pagesList;
try {
pagesList = getIllustPages(illustId, quality, false);
@ -506,7 +585,7 @@ public class BotCommandProcess {
}
if(log.isDebugEnabled()) {
StringBuilder logBuilder = new StringBuilder("作品Id {} 所有页面下载链接: \n");
StringBuilder logBuilder = new StringBuilder("作品Id " + illustId + " 所有页面下载链接: \n");
AtomicInteger index = new AtomicInteger();
pagesList.forEach(item -> logBuilder.append(index.incrementAndGet()).append(". ").append(item).append("\n"));
log.debug(logBuilder.toString());
@ -517,19 +596,6 @@ public class BotCommandProcess {
return "指定的页数超出了范围(总共 " + pagesList.size() + " 页)";
}
try {
if (isNoSafe(illustId, globalProp, false)) {
log.warn("作品 {} 存在R-18内容且设置\"image.allowR18\"为false将屏蔽该作品不发送.", illustId);
return "(根据设置,该作品已被屏蔽!)";
} else if(isReported(illustId)) {
log.warn("作品Id {} 被报告, 正在等待审核, 跳过该作品.", illustId);
return "(该作品已被封印)";
}
} catch (IOException e) {
log.warn("作品信息无法获取!", e);
return "发生网络异常,无法获取图片!";
}
String downloadLink = pagesList.get(pageIndex - 1);
String fileName = URLs.getResourceName(Strings.nullToEmpty(downloadLink));
File imageFile = new File(getImageStoreDir(), downloadLink.substring(downloadLink.lastIndexOf("/") + 1));
@ -554,12 +620,17 @@ public class BotCommandProcess {
}
}
ImageCacheObject taskObject = new ImageCacheObject(imageCache, illustId, downloadLink, imageFile);
try {
imageCacheExecutor.executorSync(taskObject);
Throwable throwable = ImageCacheStore.executeCacheRequest(new ImageCacheObject(imageCache, illustId, downloadLink, imageFile));
if(throwable != null) {
throw throwable;
}
} catch (InterruptedException e) {
log.error("等待图片下载时发生中断", e);
return "图片获取失败!";
log.warn("图片缓存被中断", e);
return "(错误:图片获取超时)";
} catch (Throwable e) {
log.error("图片 {} 获取失败:\n{}", illustId + "p" + pageIndex, Throwables.getStackTraceAsString(e));
return "(错误: 图片获取出错)";
}
} else {
log.debug("图片 {} 缓存命中.", fileName);
@ -575,7 +646,7 @@ public class BotCommandProcess {
* @return 返回设定好参数的BotCode
*/
private static BotCode getImageToBotCode(File targetFile, boolean updateCache) {
String fileName = targetFile.getName();
String fileName = Objects.requireNonNull(targetFile, "targetFile is null").getName();
BotCode code = BotCode.parse(CQCode.image(getImageStoreDir().getName() + "/" + fileName));
code.addParameter("absolutePath", targetFile.getAbsolutePath());
code.addParameter("imageName", fileName.substring(0, fileName.lastIndexOf(".")));
@ -590,7 +661,7 @@ public class BotCommandProcess {
illustPreLoadDataCache.clear();
pagesCache.clear();
searchBodyCache.clear();
File imageStoreDir = new File(System.getProperty("cgj.cqRootDir") + "data/image/cgj/");
File imageStoreDir = new File(BotGlobal.getGlobal().getDataStoreDir(), "data/image/cgj/");
File[] listFiles = imageStoreDir.listFiles();
if (listFiles == null) {
log.debug("图片缓存目录为空或内部文件获取失败!");
@ -605,12 +676,18 @@ public class BotCommandProcess {
/**
* 举报某一作品
* @param fromGroup 来源群(系统提供)
* @param illustId 需要举报的作品id
* @param reason 举报原因
* @return 返回提示信息
*/
@Command
public static String report(@Argument(name = "$fromGroup") long fromGroup, @Argument(name = "$fromQQ") long fromQQ, @Argument(name = "id") int illustId, @Argument(name = "msg", force = false) String reason) {
public static String report(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "$fromQQ") long fromQQ,
@Argument(name = "id") int illustId,
@Argument(name = "msg", force = false) String reason
) {
log.warn("收到作品反馈(IllustId: {}, 原因: {})", illustId, reason);
JsonObject reportJson = new JsonObject();
reportJson.addProperty("illustId", illustId);
@ -631,13 +708,6 @@ public class BotCommandProcess {
return reportStore.exists(String.valueOf(illustId));
}
/*
下一目标:
添加定时发图
定时发图支持设置关注标签
标签....标签支持搜索吧
*/
/**
* 检查指定作品是否为r18
* @param illustId 作品Id
@ -645,17 +715,26 @@ public class BotCommandProcess {
* @param returnRaw 是否返回原始值
* @return 如果为true, 则不为全年龄
* @throws IOException 获取数据时发生异常时抛出
* @throws NoSuchElementException 当作品不存在时抛出
*/
public static boolean isNoSafe(int illustId, Properties settingProp, boolean returnRaw) throws IOException {
public static boolean isNoSafe(int illustId, Properties settingProp, boolean returnRaw) throws IOException, NoSuchElementException {
boolean rawValue = getIllustInfo(illustId, false).getAsJsonArray("tags").contains(new JsonPrimitive("R-18"));
return returnRaw || settingProp == null ? rawValue : rawValue && !settingProp.getProperty("image.allowR18", "false").equalsIgnoreCase("true");
}
private static JsonObject getIllustInfo(int illustId, boolean flushCache) throws IOException {
/**
* 获取作品信息
* @param illustId 作品Id
* @param flushCache 强制刷新缓存
* @return 返回作品信息
* @throws IOException 当Http请求发生异常时抛出
* @throws NoSuchElementException 当作品未找到时抛出
*/
private static JsonObject getIllustInfo(int illustId, boolean flushCache) throws IOException, NoSuchElementException {
String illustIdStr = buildSyncKey(Integer.toString(illustId));
JsonObject illustInfoObj = null;
if (!illustInfoCache.exists(illustIdStr) || flushCache) {
synchronized (illustIdStr) { // TODO: 这里要不做成HashMap存储key而避免使用常量池?
synchronized (illustIdStr) {
if (!illustInfoCache.exists(illustIdStr) || flushCache) {
illustInfoObj = pixivDownload.getIllustInfoByIllustId(illustId);
illustInfoCache.update(illustIdStr, illustInfoObj, null);
@ -684,13 +763,14 @@ public class BotCommandProcess {
if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) {
synchronized (illustIdStr) {
if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) {
log.info("IllustId {} 缓存失效, 正在更新...", illustId);
log.debug("IllustId {} 缓存失效, 正在更新...", illustId);
JsonObject preLoadDataObj = pixivDownload.getIllustPreLoadDataById(illustId)
.getAsJsonObject("illust")
.getAsJsonObject(Integer.toString(illustId));
long expire = 7200 * 1000;
String propValue = globalProp.getProperty("cache.illustPreLoadData.expire", "7200000");
String propValue = SettingProperties.
getProperty(SettingProperties.GLOBAL, "cache.illustPreLoadData.expire", "7200000");
log.debug("PreLoadData有效时间设定: {}", propValue);
try {
expire = Long.parseLong(propValue);
@ -700,7 +780,7 @@ public class BotCommandProcess {
result = preLoadDataObj;
illustPreLoadDataCache.update(illustIdStr, preLoadDataObj, expire);
log.info("作品Id {} preLoadData缓存已更新(有效时间: {})", illustId, expire);
log.debug("作品Id {} preLoadData缓存已更新(有效时间: {})", illustId, expire);
}
}
}
@ -731,11 +811,19 @@ public class BotCommandProcess {
}
return result;
}
/**
* 获取图片存储目录.
* <p>每次调用都会检查目录是否存在, 如不存在则会抛出异常</p>
* @return 返回File对象
* @throws RuntimeException 当目录创建失败时将包装{@link IOException}异常并抛出.
*/
private static File getImageStoreDir() {
if(!imageStoreDir.exists() && !imageStoreDir.mkdirs()) {
log.warn("酷Q图片缓存目录失效(Path: {} )", imageStoreDir.getAbsolutePath());
throw new RuntimeException(new IOException("文件夹创建失败!"));
if(!imageStoreDir.exists() && !Files.isSymbolicLink(imageStoreDir.toPath())) {
if(!imageStoreDir.mkdirs()) {
log.warn("酷Q图片缓存目录失效(Path: {} )", imageStoreDir.getAbsolutePath());
throw new RuntimeException(new IOException("文件夹创建失败!"));
}
}
return imageStoreDir;
}
@ -772,15 +860,16 @@ public class BotCommandProcess {
if(!rankingCache.exists(requestSign) || flushCache) {
synchronized(requestSign) {
if(!rankingCache.exists(requestSign) || flushCache) {
log.info("Ranking缓存失效, 正在更新...(RequestSign: {})", requestSign);
log.debug("Ranking缓存失效, 正在更新...(RequestSign: {})", requestSign);
List<JsonObject> rankingResult = pixivDownload.getRanking(contentType, mode, queryDate, 1, 500);
long expireTime = 0;
if(rankingResult.size() == 0) {
log.info("数据获取失败, 将设置浮动有效时间以准备下次更新.");
expireTime = 5400000 + expireTimeFloatRandom.nextInt(1800000);
log.warn("数据获取失败, 将设置浮动有效时间以准备下次更新. (ExpireTime: {}ms)", expireTime);
}
result = new ArrayList<>(rankingResult).subList(start - 1, start + range - 1);
rankingCache.update(requestSign, rankingResult,
rankingResult.size() == 0 ? 5400000 + expireTimeFloatRandom.nextInt(1800000) : 0);
log.info("Ranking缓存更新完成.(RequestSign: {})", requestSign);
rankingCache.update(requestSign, rankingResult, expireTime);
log.debug("Ranking缓存更新完成.(RequestSign: {})", requestSign);
}
}
}

View File

@ -0,0 +1,83 @@
package net.lamgc.cgj.bot;
import net.lamgc.cgj.bot.event.MessageEvent;
import net.lamgc.cgj.bot.event.VirtualLoadMessageEvent;
import net.lamgc.utils.event.EventExecutor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Properties;
/**
* 消息事件处理调试器.
* <p>当启用了消息事件处理调试后, 将会根据调试器代号调用指定调试器</p>
*/
@SuppressWarnings("unused")
public enum MessageEventExecutionDebugger {
/**
* PM - 压力测试
*/
PM ((executor, event, properties, log) -> {
MessageEvent virtualLoadEvent = VirtualLoadMessageEvent.toVirtualLoadMessageEvent(event, false);
int rotation = 5;
int number = 50;
int interval = 2500;
try {
rotation = Integer.parseInt(properties.getProperty("debug.pm.rotation", "5"));
} catch(NumberFormatException e) {
log.warn("配置项 {} 值无效, 将使用默认值.({})", "debug.pm.rotation", rotation);
}
try {
number = Integer.parseInt(properties.getProperty("debug.pm.number", "50"));
} catch(NumberFormatException e) {
log.warn("配置项 {} 值无效, 将使用默认值.({})", "debug.pm.number", number);
}
try {
interval = Integer.parseInt(properties.getProperty("debug.pm.interval", "2500"));
} catch(NumberFormatException e) {
log.warn("配置项 {} 值无效, 将使用默认值.({})", "debug.pm.interval", interval);
}
boolean interrupted = false;
Thread currentThread = Thread.currentThread();
for(int rotationCount = 0; rotationCount < rotation && !interrupted; rotationCount++) {
for(int sendCount = 0; sendCount < number; sendCount++) {
if(currentThread.isInterrupted()) {
interrupted = true;
break;
}
executor.executor(virtualLoadEvent);
}
try {
Thread.sleep(interval);
} catch (InterruptedException e) {
break;
}
}
});
public final MessageExecuteDebugger debugger;
MessageEventExecutionDebugger(MessageExecuteDebugger debugger) {
this.debugger = debugger;
}
public static Logger getDebuggerLogger(MessageEventExecutionDebugger debugger) {
return LoggerFactory.getLogger(MessageEventExecutionDebugger.class.getName() + "." + debugger.name());
}
@FunctionalInterface
public interface MessageExecuteDebugger {
/**
* 接收事件并根据指定需求处理
* @param executor 事件执行器
* @param event 消息事件对象
* @param properties 配置项, 调试器应按'debug.[debuggerName].'为前缀存储相应调试信息
* @throws Exception 当抛出异常则打断调试, 并输出至日志
*/
void accept(EventExecutor executor, MessageEvent event, Properties properties, Logger logger) throws Exception;
}
}

View File

@ -5,8 +5,6 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.lang.reflect.Field;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.concurrent.atomic.AtomicBoolean;
@ -15,8 +13,8 @@ import java.util.concurrent.atomic.AtomicBoolean;
*/
public class RandomIntervalSendTimer extends TimerTask {
private final static Timer timer = new Timer("Thread-RIST");
private final static Logger log = LoggerFactory.getLogger("RandomIntervalSendTimer");
private final static Timer timer = new Timer("Thread-RIST", true);
private final static Logger log = LoggerFactory.getLogger(RandomIntervalSendTimer.class);
private final static Map<Long, RandomIntervalSendTimer> timerMap = new HashMap<>();
private final long timerId;
@ -24,8 +22,10 @@ public class RandomIntervalSendTimer extends TimerTask {
private final AutoSender sender;
private final long time;
private final int floatTime;
private AtomicBoolean loop = new AtomicBoolean();
private final AtomicBoolean loop = new AtomicBoolean();
private final AtomicBoolean start = new AtomicBoolean();
private final String hashId = Integer.toHexString(this.hashCode());
/**
* 创建一个随机延迟发送器
@ -92,18 +92,16 @@ public class RandomIntervalSendTimer extends TimerTask {
start(this.loop.get());
}
private final static DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-ss HH:mm:ss");
/**
* 启动定时器
* @param loop 是否循环, 如果为true, 则任务完成后会自动调用start方法继续循环, 直到被调用{@code #}或总定时器被销毁;
*/
public void start(boolean loop) {
this.loop.set(loop);
long nextDelay = time + timeRandom.nextInt(floatTime);
long nextDelay = time + (floatTime <= 0 ? 0 : timeRandom.nextInt(floatTime));
Date nextDate = new Date();
nextDate.setTime(nextDate.getTime() + nextDelay);
log.info("定时器 {} 下一延迟: {}ms ({})", Integer.toHexString(this.hashCode()), nextDelay, dateFormat.format(nextDate));
log.info("定时器 {} 下一延迟: {}ms ({})", hashId, nextDelay, nextDate);
if(start.get()) {
try {
Field state = this.getClass().getSuperclass().getDeclaredField("state");

View File

@ -7,7 +7,6 @@ import net.lamgc.cgj.pixiv.PixivURL;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.util.Calendar;
import java.util.Date;
import java.util.List;
@ -39,7 +38,7 @@ public class RandomRankingArtworksSender extends AutoSender {
super(messageSender);
this.mode = mode;
this.contentType = contentType;
log = LoggerFactory.getLogger("RecommendArtworksSender@" + Integer.toHexString(this.hashCode()));
log = LoggerFactory.getLogger(this.toString());
this.rankingStart = rankingStart > 0 ? rankingStart : 1;
this.rankingStop = rankingStop > 0 ? rankingStop : 150;
if(this.rankingStart > this.rankingStop) {
@ -69,7 +68,7 @@ public class RandomRankingArtworksSender extends AutoSender {
selectRanking,
1, false);
log.info("RankingResult.size: {}", rankingList.size());
log.debug("RankingResult.size: {}", rankingList.size());
if(rankingList.size() != 1) {
log.error("排行榜选取失败!(获取到了多个结果)");
return;
@ -77,7 +76,7 @@ public class RandomRankingArtworksSender extends AutoSender {
JsonObject rankingInfo = rankingList.get(0);
int illustId = rankingInfo.get("illust_id").getAsInt();
if(BotCommandProcess.isNoSafe(illustId, BotCommandProcess.globalProp, false)) {
if(BotCommandProcess.isNoSafe(illustId, SettingProperties.getProperties(SettingProperties.GLOBAL), false)) {
log.warn("作品为r18作品, 取消本次发送.");
return;
} else if(BotCommandProcess.isReported(illustId)) {
@ -89,11 +88,11 @@ public class RandomRankingArtworksSender extends AutoSender {
message.append("#美图推送 - 今日排行榜 第 ").append(rankingInfo.get("rank").getAsInt()).append("\n");
message.append("标题:").append(rankingInfo.get("title").getAsString()).append("(").append(illustId).append(")\n");
message.append("作者:").append(rankingInfo.get("user_name").getAsString()).append("\n");
message.append(BotCommandProcess.getImageById(illustId, quality, 1));
message.append(BotCommandProcess.getImageById(0, illustId, quality, 1));
message.append("\n如有不当作品可使用\".cgj report -id ").append(illustId).append("\"向色图姬反馈。");
getMessageSender().sendMessage(message.toString());
} catch (IOException e) {
e.printStackTrace();
} catch (Exception e) {
log.error("发送随机作品时发生异常", e);
}
}
}

View File

@ -1,12 +1,12 @@
package net.lamgc.cgj.bot;
import com.google.common.base.Throwables;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.event.VirtualLoadMessageEvent;
import net.lamgc.cgj.pixiv.PixivURL;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.text.SimpleDateFormat;
import java.time.LocalDate;
import java.util.Calendar;
import java.util.Date;
@ -27,21 +27,24 @@ public class RankingUpdateTimer {
Calendar cal = Calendar.getInstance();
cal.setTime(firstRunDate == null ? new Date() : firstRunDate);
LocalDate currentLocalDate = LocalDate.now();
if(cal.get(Calendar.DAY_OF_YEAR) <= currentLocalDate.getDayOfYear() && cal.get(Calendar.HOUR_OF_DAY) >= 12) {
cal.set(Calendar.DAY_OF_YEAR, cal.get(Calendar.DAY_OF_YEAR) + 1);
if(cal.get(Calendar.DAY_OF_YEAR) < currentLocalDate.getDayOfYear() || (
cal.get(Calendar.DAY_OF_YEAR) == currentLocalDate.getDayOfYear() &&
(cal.get(Calendar.HOUR_OF_DAY) * 60 + cal.get(Calendar.MINUTE) >= 690))) {
cal.set(Calendar.DAY_OF_YEAR, currentLocalDate.getDayOfYear() + 1);
}
cal.set(Calendar.HOUR_OF_DAY, 12);
cal.set(Calendar.HOUR_OF_DAY, 11);
cal.set(Calendar.MINUTE, 30);
cal.set(Calendar.SECOND, 0);
cal.set(Calendar.MILLISECOND, 0);
log.warn("已设置排行榜定时更新, 首次运行时间: {}", cal.getTime());
long delay = cal.getTime().getTime() - System.currentTimeMillis();
log.warn("已设置排行榜定时更新, 首次运行时间: {} ({}min)", cal.getTime(), delay / 1000 / 60);
timer.schedule(new TimerTask() {
@Override
public void run() {
now(null);
}
}, cal.getTime(), 86400000); // 1 Day
}, delay, 86400000); // 1 Day
}
public void now(Date queryDate) {
@ -52,13 +55,14 @@ public class RankingUpdateTimer {
LocalDate currentLocalDate = LocalDate.now();
if(calendar.get(Calendar.DAY_OF_YEAR) == currentLocalDate.getDayOfYear() ||
calendar.get(Calendar.DAY_OF_YEAR) == currentLocalDate.getDayOfYear() - 1) {
if(calendar.get(Calendar.HOUR_OF_DAY) < 12) {
if(calendar.get(Calendar.HOUR_OF_DAY) < 11) {
calendar.add(Calendar.DAY_OF_YEAR, -2);
} else {
calendar.add(Calendar.DAY_OF_YEAR, -1);
}
}
String dateStr = new SimpleDateFormat("yyyy-MM-dd").format(calendar.getTime());
log.info("正在获取 {} 期排行榜数据...", calendar.getTime());
for (PixivURL.RankingMode rankingMode : PixivURL.RankingMode.values()) {
for (PixivURL.RankingContentType contentType : PixivURL.RankingContentType.values()) {
@ -66,15 +70,9 @@ public class RankingUpdateTimer {
log.debug("不支持的类型, 填空值跳过...(类型: {}.{})", rankingMode.name(), contentType.name());
}
log.info("当前排行榜类型: {}.{}, 正在更新...", rankingMode.name(), contentType.name());
try {
//BotCommandProcess.getRankingInfoByCache(contentType, rankingMode, calendar.getTime(), 1, 0, true);
BotEventHandler.executor.executorSync(
new VirtualLoadMessageEvent(0,0,
".cgj ranking -type=" + contentType.name() + " -mode=" + rankingMode.name()));
log.info("排行榜 {}.{} 更新完成.", rankingMode.name(), contentType.name());
} catch (InterruptedException e) {
log.error("排行榜 {}.{} 更新时发生异常. \n{}", rankingMode.name(), contentType.name(), Throwables.getStackTraceAsString(e));
}
BotEventHandler.executeMessageEvent(new VirtualLoadMessageEvent(0,0,
".cgj ranking -type=" + contentType.name() + " -mode=" + rankingMode.name() + " -force -date " + dateStr));
log.info("排行榜 {}.{} 负载指令已投递.", rankingMode.name(), contentType.name());
}
}
log.warn("定时任务更新完成.");

View File

@ -0,0 +1,242 @@
package net.lamgc.cgj.bot;
import com.google.common.base.Throwables;
import net.lamgc.cgj.bot.boot.BotGlobal;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.*;
import java.nio.charset.StandardCharsets;
import java.util.*;
public final class SettingProperties {
private final static Logger log = LoggerFactory.getLogger(SettingProperties.class);
private final static File globalPropFile = new File(getPropertiesDir(), "global.properties");
private final static Properties globalProp = new Properties();
private final static Map<Long, Properties> groupPropMap = new HashMap<>();
private final static Set<Long> changeList = Collections.synchronizedSet(new HashSet<>());
/**
* 全局配置项
*/
public final static long GLOBAL = 0;
/**
* 清空所有Properties.
*/
public static void clearProperties() {
groupPropMap.clear();
globalProp.clear();
}
/**
* 加载配置文件
*/
public static void loadProperties() {
loadGlobalProperties();
File[] files = getPropertiesDir()
.listFiles((dir, fileName) -> fileName.startsWith("group.") && fileName.endsWith(".properties"));
if(files == null) {
log.error("检索群组配置文件失败, 可能是被拒绝访问.");
return;
}
for (File file : files) {
String name = file.getName();
long groupId;
try {
groupId = Long.parseLong(name.substring(name.indexOf("group.") + 6, name.lastIndexOf(".properties")));
} catch (NumberFormatException e) {
log.error("非法的配置文件名: {}", name);
continue;
}
if(!groupPropMap.containsKey(groupId)) {
groupPropMap.put(groupId, new Properties(globalProp));
}
loadGroupProperties(groupId, groupPropMap.get(groupId));
}
}
/**
* 保存配置项
*/
public static void saveProperties() {
log.info("正在保存所有配置...");
saveGlobalProperties();
for (Long groupId : groupPropMap.keySet()) {
if(!changeList.contains(groupId)) {
log.debug("群组 {} 配置无改动, 忽略保存.", groupId);
return;
}
log.debug("正在保存群组 {} 配置文件...", groupId);
saveGroupProperties(groupId);
}
log.info("配置保存完成.");
}
/**
* 保存指定群组的配置文件
* @param groupId 要保存配置的群组Id
*/
private static void saveGroupProperties(long groupId) {
try {
saveGroupProperties(groupId, getGroupProperties(groupId));
} catch (IOException e) {
log.error("群组 {} 配置保存失败\n{}", groupId, Throwables.getStackTraceAsString(e));
}
}
private static void saveGroupProperties(Long groupId, Properties properties) throws IOException {
File groupPropFile = new File(getPropertiesDir(), "group." + groupId + ".properties");
if(!groupPropFile.exists() && !groupPropFile.createNewFile()) {
log.error("群组 {} 配置文件创建失败!", groupId);
return;
}
saveProperties(properties, new FileOutputStream(groupPropFile));
}
private static void loadGlobalProperties() {
if(globalPropFile.exists() && globalPropFile.isFile()) {
log.info("正在加载全局配置文件...");
try (Reader reader = new InputStreamReader(new FileInputStream(globalPropFile), StandardCharsets.UTF_8)) {
globalProp.load(reader);
log.info("全局配置文件加载完成.");
} catch (IOException e) {
log.error("加载全局配置文件时发生异常", e);
}
} else {
log.info("未找到全局配置文件,跳过加载.");
}
}
/**
* 保存全局配置项
*/
private static void saveGlobalProperties() {
try {
if(!globalPropFile.exists() && !globalPropFile.createNewFile()) {
log.error("创建全局配置文件失败.");
return;
}
saveProperties(globalProp, new FileOutputStream(globalPropFile));
} catch (IOException e) {
log.error("全局配置文件保存时发生异常", e);
}
}
private static void loadGroupProperties(long groupId, Properties properties) {
File propFile = new File(getPropertiesDir(), "group." + groupId + ".properties");
Properties groupProp = Objects.requireNonNull(properties);
if(!propFile.exists() || !propFile.isFile()) {
log.warn("群组 {} 配置文件不存在, 或不是一个文件.({})", groupId, propFile.getAbsolutePath());
return;
}
try (Reader reader = new InputStreamReader(new FileInputStream(propFile), StandardCharsets.UTF_8)) {
groupProp.load(reader);
} catch (IOException e) {
log.error("读取群组 {} 群配置文件时发生异常:\n{}", groupId, Throwables.getStackTraceAsString(e));
}
}
private static void saveProperties(Properties properties, OutputStream stream) throws IOException {
properties.store(new OutputStreamWriter(stream, StandardCharsets.UTF_8), null);
}
/**
* 获取配置文件目录
* @return 返回目录File对象.
*/
private static File getPropertiesDir() {
File propDir = new File(BotGlobal.getGlobal().getDataStoreDir(), "/setting/");
if(!propDir.exists() && !propDir.mkdirs()) {
log.warn("Setting文件夹创建失败!");
}
return propDir;
}
public static String getProperty(long groupId, String key) {
return getProperty(groupId, key, null);
}
public static String getProperty(long groupId, String key, String defaultValue) {
if(groupId <= 0) {
return globalProp.getProperty(key, defaultValue);
} else {
Properties properties = groupPropMap.get(groupId);
return properties == null ? defaultValue : properties.getProperty(key, defaultValue);
}
}
/**
* 设置配置项
* @param groupId 群组Id, 如为0或负数则为全局配置
* @param key 配置项key名
* @param value 欲设置的新值, 如为null则删除该配置项
* @return 返回上一次设定值
*/
public static String setProperty(long groupId, String key, String value) {
Objects.requireNonNull(key);
Properties targetProperties;
if(groupId <= 0) {
targetProperties = globalProp;
} else {
changeList.add(groupId);
targetProperties = getGroupProperties(groupId);
}
String lastValue = targetProperties.getProperty(key);
if(value != null) {
targetProperties.setProperty(key, value);
} else {
targetProperties.remove(key);
}
return lastValue;
}
/**
* 获取GlobalProperties
* @return 全局Properties
*/
private static Properties getGlobalProperties() {
return globalProp;
}
/**
* 获取群组Properties
* @param groupId 群组Id
* @return 如果存在, 返回Properties, 不存在返回null.
* @throws IllegalArgumentException 当群组Id 小于或等于0 时抛出.
*/
private static Properties getGroupProperties(long groupId) {
if (groupId <= 0) {
throw new IllegalArgumentException("Group number cannot be 0 or negative: " + groupId);
}
if(!groupPropMap.containsKey(groupId)) {
groupPropMap.put(groupId, new Properties(globalProp));
}
return groupPropMap.get(groupId);
}
/**
* 获取群组 Properties, 如果指定群组没有 Properties, 则使用GlobalProperties.
* @param groupId 指定群组Id
* @return 如果群组存在所属Properties, 则返回群组Properties, 否则返回GlobalProperties.
*/
public static Properties getProperties(long groupId) {
if(groupPropMap.containsKey(groupId)) {
return groupPropMap.get(groupId);
}
return getGlobalProperties();
}
}

View File

@ -0,0 +1,46 @@
package net.lamgc.cgj.bot.boot;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.util.PropertiesUtils;
import net.lamgc.utils.base.ArgumentsProperties;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public final class ApplicationBoot {
private final static Logger log = LoggerFactory.getLogger(ApplicationBoot.class);
private ApplicationBoot() {}
/**
* 初始化应用.
* <p>该方法不会初始化机器人, 仅初始化应用所需的配置信息.</p>
*/
public static void initialApplication(String[] args) {
ArgumentsProperties argsProp = new ArgumentsProperties(args);
if(!PropertiesUtils.getSettingToSysProp(argsProp, "proxy", null)) {
PropertiesUtils.getEnvSettingToSysProp("CGJ_PROXY", "proxy", null);
}
if(!PropertiesUtils.getSettingToSysProp(argsProp, "botDataDir", "./") &&
!PropertiesUtils.getEnvSettingToSysProp("CGJ_BOT_DATA_DIR", "botDataDir", "./")) {
log.warn("未设置botDataDir, 当前运行目录将作为酷Q机器人所在目录.");
}
if(!PropertiesUtils.getSettingToSysProp(argsProp, "redisAddress", "127.0.0.1") &&
!PropertiesUtils.getEnvSettingToSysProp("CGJ_REDIS_URI", "redisAddress", "127.0.0.1")) {
log.warn("未设置RedisAddress, 将使用默认值连接Redis服务器(127.0.0.1:6379)");
}
// 初始化 BotGlobal
//noinspection ResultOfMethodCallIgnored 这里仅仅是加载BotGlobal而已
BotGlobal.getGlobal();
}
/**
* 初始化机器人.
* <p>本方法由框架调用.</p>
*/
public static void initialBot() {
BotEventHandler.initial();
}
}

View File

@ -0,0 +1,91 @@
package net.lamgc.cgj.bot.boot;
import com.google.common.base.Strings;
import org.apache.http.HttpHost;
import org.apache.http.client.CookieStore;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.JedisPool;
import java.io.File;
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URL;
public final class BotGlobal {
private final static Logger log = LoggerFactory.getLogger(BotGlobal.class);
private final static BotGlobal instance = new BotGlobal();
public static BotGlobal getGlobal() {
if(instance == null) {
throw new IllegalStateException("");
}
return instance;
}
private final URI redisUri;
/**
* 所有缓存共用的JedisPool
*/
private final JedisPool redisServer;
private final File dataStoreDir;
private final HttpHost proxy;
private CookieStore cookieStore;
private BotGlobal() {
this.redisUri = URI.create("redis://" + System.getProperty("cgj.redisAddress"));
this.redisServer = new JedisPool(
getRedisUri().getHost(),
getRedisUri().getPort() == -1 ? 6379 : getRedisUri().getPort());
String dataStoreDirPath = System.getProperty("cgj.botDataDir");
this.dataStoreDir = new File((!dataStoreDirPath.endsWith("/") || !dataStoreDirPath.endsWith("\\")) ?
dataStoreDirPath + System.getProperty("file.separator") : dataStoreDirPath);
String proxyAddress = System.getProperty("cgj.proxy");
HttpHost temp = null;
if(!Strings.isNullOrEmpty(proxyAddress)) {
try {
URL proxyUrl = new URL(proxyAddress);
temp = new HttpHost(proxyUrl.getHost(), proxyUrl.getPort());
log.info("已启用代理:{}", temp.toHostString());
} catch (MalformedURLException e) {
log.error("Proxy地址解析失败, 代理将不会启用.", e);
}
}
this.proxy = temp;
}
public URI getRedisUri() {
return redisUri;
}
public File getDataStoreDir() {
if(!dataStoreDir.exists() && !dataStoreDir.mkdirs()) {
log.error("DataStoreDir 创建失败, 数据存储可能失效!");
}
return dataStoreDir;
}
public JedisPool getRedisServer() {
return redisServer;
}
public HttpHost getProxy() {
return proxy;
}
public CookieStore getCookieStore() {
return cookieStore;
}
public void setCookieStore(CookieStore cookieStore) {
this.cookieStore = cookieStore;
}
}

View File

@ -0,0 +1,52 @@
package net.lamgc.cgj.bot.cache;
import com.google.common.base.Throwables;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Set;
import java.util.Timer;
import java.util.TimerTask;
import java.util.concurrent.CopyOnWriteArraySet;
public class AutoCleanTimer extends TimerTask {
private final static Set<Cleanable> cleanSet = new CopyOnWriteArraySet<>();
private final static Timer cleanTimer = new Timer("Thread-AutoClean", true);
private final static Logger log = LoggerFactory.getLogger(AutoCleanTimer.class);
static {
cleanTimer.schedule(new AutoCleanTimer(), 100L);
}
/**
* 增加需要定时执行清理的缓存库
* @param store 已实现Cleanable的对象
*/
public static void add(Cleanable store) {
cleanSet.add(store);
}
/**
* 移除已添加的缓存库
* @param store 需要从AutoCleanTimer移除的对象
*/
public static void remove(Cleanable store) {
cleanSet.remove(store);
}
private AutoCleanTimer() {}
@Override
public void run() {
cleanSet.forEach(cleanable -> {
try {
cleanable.clean();
} catch (Exception e) {
log.error("{} 执行清理动作时发生异常:\n{}", cleanable.toString(), Throwables.getStackTraceAsString(e));
}
});
}
}

View File

@ -0,0 +1,35 @@
package net.lamgc.cgj.bot.cache;
public final class CacheStoreUtils {
private CacheStoreUtils() {}
/**
* 将 CacheStore 转换成 基于 {@link LocalHashCacheStore} 的 {@link HotDataCacheStore}
* <p>通过该方法转换, 会自动启用 自动清理</p>
* @param cacheStore 上游缓存库
* @param expireTime 热点缓存最小有效期
* @param floatRange 缓存浮动最大范围
* @param <T> 缓存库数据类型
* @return 返回 {@link HotDataCacheStore}
*/
public static <T> CacheStore<T> hashLocalHotDataStore(CacheStore<T> cacheStore, long expireTime, int floatRange) {
return hashLocalHotDataStore(cacheStore, expireTime, floatRange, true);
}
/**
* 将 CacheStore 转换成 基于 {@link LocalHashCacheStore} 的 {@link HotDataCacheStore}
* @param cacheStore 上游缓存库
* @param expireTime 热点缓存最小有效期
* @param floatRange 缓存浮动最大范围
* @param autoClean 是否启用自动清理
* @param <T> 缓存库数据类型
* @return 返回 {@link HotDataCacheStore}
*/
public static <T> CacheStore<T> hashLocalHotDataStore(CacheStore<T> cacheStore,
long expireTime, int floatRange, boolean autoClean) {
return new HotDataCacheStore<>(cacheStore, new LocalHashCacheStore<>(), expireTime, floatRange, autoClean);
}
}

View File

@ -0,0 +1,13 @@
package net.lamgc.cgj.bot.cache;
/**
* 可清理接口, 实现该接口代表该类具有清理动作.
*/
public interface Cleanable {
/**
* 该方法需要CacheStore完成对过期Entry的清除.
*/
void clean() throws Exception;
}

View File

@ -10,31 +10,38 @@ import java.util.*;
* @param <T> 存储类型
* @author LamGC
*/
public class HotDataCacheStore<T> implements CacheStore<T> {
public class HotDataCacheStore<T> implements CacheStore<T>, Cleanable {
private final CacheStore<T> parent;
private final CacheStore<T> current;
private final long expireTime;
private final int expireFloatRange;
private final Random random = new Random();
private final Logger log = LoggerFactory.getLogger(HotDataCacheStore.class.getSimpleName() + "@" + Integer.toHexString(this.hashCode()));
private final Logger log = LoggerFactory.getLogger(this.toString());
/**
* 构造热点缓存存储对象
* @param parent 上级缓存存储库
* @param current 热点缓存存储库, 最好使用本地缓存(例如 {@linkplain LocalHashCacheStore LocalHashCacheStore})
* @param expireTime 本地缓存库的缓存项过期时间, 单位毫秒;
* 该时间并不是所有缓存项的最终过期时间, 还需要根据expireFloatRange的设定随机设置, 公式:
* {@code expireTime + new Random().nextInt(expireFloatRange)}
* @param expireFloatRange 过期时间的浮动范围(单位毫秒), 用于防止短时间内大量缓存项失效导致的缓存雪崩
* 该时间并不是所有缓存项的最终过期时间, 还需要根据expireFloatRange的设定随机设置, 公式:
* {@code expireTime + new Random().nextInt(expireFloatRange)}
* @param expireFloatRange 过期时间的浮动范围(单位毫秒), 用于防止短时间内大量缓存项失效导致的缓存雪崩,
* 如设置为0或负数, 则不启用浮动范围.
* @param autoClean 是否交由{@link AutoCleanTimer}自动执行清理, 启用后, AutoCleanTimer会自动检查过期Key并进行删除.
*/
public HotDataCacheStore(CacheStore<T> parent, CacheStore<T> current, long expireTime, int expireFloatRange) {
public HotDataCacheStore(CacheStore<T> parent, CacheStore<T> current, long expireTime, int expireFloatRange, boolean autoClean) {
this.parent = parent;
this.current = current;
this.expireTime = expireTime;
this.expireFloatRange = expireFloatRange;
log.debug("HotDataCacheStore初始化完成. (Parent: {}, Current: {}, expireTime: {}, expireFloatRange: {})",
parent, current, expireTime, expireFloatRange);
if(autoClean) {
AutoCleanTimer.add(this);
}
log.debug("HotDataCacheStore初始化完成. " +
"(Parent: {}, Current: {}, expireTime: {}, expireFloatRange: {}, autoClean: {})",
parent, current, expireTime, expireFloatRange, autoClean);
}
@Override
@ -63,7 +70,8 @@ public class HotDataCacheStore<T> implements CacheStore<T> {
return null;
}
log.debug("Parent缓存命中, 正在更新Current缓存库...");
current.update(key, parentResult, expireTime + random.nextInt(expireFloatRange));
current.update(key, parentResult,
expireTime + (expireFloatRange <= 0 ? 0 : random.nextInt(expireFloatRange)));
log.debug("Current缓存库更新完成.");
result = parentResult;
} else {
@ -114,11 +122,27 @@ public class HotDataCacheStore<T> implements CacheStore<T> {
@Override
public boolean supportedPersistence() {
// 由于Current的缓存数据会更新到Parent上,
// 所以只要任意一边支持持久化, 那么该缓存库就支持持久化
return current.supportedPersistence() || parent.supportedPersistence();
}
@Override
public boolean supportedList() {
return false;
// 只有两边都支持List, 该缓存库才会支持持久化
return current.supportedList() && parent.supportedList();
}
/**
* 检查并清理已过期的Entry.
* <p>该方法仅清理Current缓存库, 不会对上游缓存库造成影响.</p>
*/
@Override
public void clean() {
for(String key : this.current.keys()) {
if(current.exists(key)) {
current.remove(key);
}
}
}
}

View File

@ -1,6 +1,7 @@
package net.lamgc.cgj.bot.cache;
import net.lamgc.cgj.Main;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.cache.exception.HttpRequestException;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.cgj.util.URLs;
import net.lamgc.utils.event.EventHandler;
@ -21,16 +22,18 @@ import java.util.Set;
public class ImageCacheHandler implements EventHandler {
private final static Logger log = LoggerFactory.getLogger("ImageCacheHandler");
private final static Logger log = LoggerFactory.getLogger(ImageCacheHandler.class);
private final static HttpClient httpClient = HttpClientBuilder.create().setProxy(Main.proxy).build();
private final static HttpClient httpClient = HttpClientBuilder.create()
.setProxy(BotGlobal.getGlobal().getProxy())
.build();
private final static Set<ImageCacheObject> cacheQueue = Collections.synchronizedSet(new HashSet<>());
@SuppressWarnings("unused")
public void getImageToCache(ImageCacheObject event) {
public void getImageToCache(ImageCacheObject event) throws Exception {
if(cacheQueue.contains(event)) {
log.info("图片 {} 已存在相同缓存任务, 跳过.", event.getStoreFile().getName());
log.warn("图片 {} 已存在相同缓存任务, 跳过.", event.getStoreFile().getName());
return;
} else {
cacheQueue.add(event);
@ -39,15 +42,15 @@ public class ImageCacheHandler implements EventHandler {
try {
log.info("图片 {} Event正在进行...({})", event.getStoreFile().getName(), Integer.toHexString(event.hashCode()));
File storeFile = event.getStoreFile();
log.info("正在缓存图片 {} (Path: {})", storeFile.getName(), storeFile.getAbsolutePath());
log.debug("正在缓存图片 {} (Path: {})", storeFile.getName(), storeFile.getAbsolutePath());
try {
if(!storeFile.exists() && !storeFile.createNewFile()) {
log.error("无法创建文件(Path: {})", storeFile.getAbsolutePath());
return;
throw new IOException("Failed to create file");
}
} catch (IOException e) {
log.error("无法创建文件(Path: {})", storeFile.getAbsolutePath());
e.printStackTrace();
throw e;
}
HttpGet request = new HttpGet(event.getDownloadLink());
@ -57,19 +60,20 @@ public class ImageCacheHandler implements EventHandler {
response = httpClient.execute(request);
} catch (IOException e) {
log.error("Http请求时发生异常", e);
return;
throw e;
}
if(response.getStatusLine().getStatusCode() != 200) {
log.warn("Http请求异常{}", response.getStatusLine());
return;
HttpRequestException requestException = new HttpRequestException(response);
log.warn("Http请求异常{}", requestException.getStatusLine());
throw requestException;
}
log.info("正在下载...(Content-Length: {}KB)", response.getEntity().getContentLength() / 1024);
log.debug("正在下载...(Content-Length: {}KB)", response.getEntity().getContentLength() / 1024);
try(FileOutputStream fos = new FileOutputStream(storeFile)) {
IOUtils.copy(response.getEntity().getContent(), fos);
} catch (IOException e) {
log.error("下载图片时发生异常", e);
return;
throw e;
}
event.getImageCache().put(URLs.getResourceName(event.getDownloadLink()), storeFile);
} finally {

View File

@ -0,0 +1,136 @@
package net.lamgc.cgj.bot.cache;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import net.lamgc.cgj.bot.cache.exception.HttpRequestException;
import java.util.Hashtable;
import java.util.Map;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicReference;
import java.util.concurrent.locks.ReentrantLock;
public final class ImageCacheStore {
private final static Logger log = LoggerFactory.getLogger(ImageCacheStore.class);
private final static Map<ImageCacheObject, Task> cacheMap = new Hashtable<>();
private final static ThreadPoolExecutor imageCacheExecutor = new ThreadPoolExecutor(
4, 6,
30L,
TimeUnit.SECONDS,
new LinkedBlockingQueue<>(100),
new ThreadFactoryBuilder()
.setNameFormat("ImageCacheThread-%d")
.build()
);
private final static ImageCacheHandler handler = new ImageCacheHandler();
static {
Thread shutdownThread = new Thread(imageCacheExecutor::shutdownNow);
shutdownThread.setName("Thread-ImageCacheShutdown");
Runtime.getRuntime().addShutdownHook(shutdownThread);
}
private ImageCacheStore() {}
/**
* 传递图片缓存任务, 并等待缓存完成.
* @param cacheObject 缓存任务组
*/
public static Throwable executeCacheRequest(ImageCacheObject cacheObject) throws InterruptedException {
Task task = getTaskState(cacheObject);
if(task.taskState.get() == TaskState.COMPLETE) {
return null;
}
boolean locked = false;
try {
if(task.taskState.get() == TaskState.COMPLETE) {
return null;
}
task.lock.lock();
locked = true;
// 双重检查
if(task.taskState.get() == TaskState.COMPLETE) {
return null;
}
// 置任务状态
task.taskState.set(TaskState.RUNNING);
Throwable throwable = null;
try {
throwable = imageCacheExecutor.submit(() -> {
try {
handler.getImageToCache(cacheObject);
} catch (Throwable e) {
return e;
}
return null;
}).get();
if(throwable == null) {
task.taskState.set(TaskState.COMPLETE);
} else {
task.taskState.set(TaskState.ERROR);
}
} catch (ExecutionException e) {
log.error("执行图片缓存任务时发生异常", e);
}
return throwable;
} finally {
if(locked) {
task.lock.unlock();
}
}
}
private static Task getTaskState(ImageCacheObject cacheObject) {
if(!cacheMap.containsKey(cacheObject)) {
cacheMap.put(cacheObject, new Task());
}
return cacheMap.get(cacheObject);
}
/**
* 获取错误信息
*/
public static String getErrorMessageFromThrowable(Throwable throwable, boolean onlyRequestException) {
if(throwable == null) {
return "";
} else if(!(throwable instanceof HttpRequestException)) {
if(onlyRequestException) {
return "";
}
return throwable.getMessage();
}
JsonObject result = new Gson()
.fromJson(((HttpRequestException) throwable).getContent(), JsonObject.class);
return result.get("msg").getAsString();
}
/**
* 任务状态
*/
private enum TaskState {
READY, RUNNING, COMPLETE, ERROR
}
private static class Task {
public final ReentrantLock lock = new ReentrantLock(true);
public final AtomicReference<TaskState> taskState = new AtomicReference<>(TaskState.READY);
}
}

View File

@ -12,19 +12,45 @@ import java.util.concurrent.atomic.AtomicReference;
* 基于Hashtable的本地缓存库
* @param <T> 缓存类型
*/
public class LocalHashCacheStore<T> implements CacheStore<T> {
public class LocalHashCacheStore<T> implements CacheStore<T>, Cleanable {
private final Hashtable<String, CacheObject<T>> cache;
/**
* 构造一个基于Hashtable的本地缓存库
* @see Hashtable
*/
public LocalHashCacheStore() {
this(0);
}
/**
* 构造一个基于Hashtable的本地缓存库
* @param initialCapacity 初始容量
* @see Hashtable
*/
public LocalHashCacheStore(int initialCapacity) {
this(initialCapacity, 0F);
}
/**
* 构造一个基于Hashtable的本地缓存库
* @param initialCapacity 初始容量
* @param loadFactor 重载因子
* @see Hashtable
*/
public LocalHashCacheStore(int initialCapacity, float loadFactor) {
this(initialCapacity, loadFactor, false);
}
/**
* 构造一个基于Hashtable的本地缓存库
* @param initialCapacity 初始容量
* @param loadFactor 重载因子
* @param autoClean 是否自动清理
* @see Hashtable
*/
public LocalHashCacheStore(int initialCapacity, float loadFactor, boolean autoClean) {
if(initialCapacity != 0) {
if(loadFactor <= 0F) {
cache = new Hashtable<>(initialCapacity);
@ -34,6 +60,10 @@ public class LocalHashCacheStore<T> implements CacheStore<T> {
} else {
cache = new Hashtable<>();
}
if(autoClean) {
AutoCleanTimer.add(this);
}
}
@Override
@ -118,6 +148,15 @@ public class LocalHashCacheStore<T> implements CacheStore<T> {
return false;
}
@Override
public void clean() throws Exception {
Date currentDate = new Date();
cache.forEach((key, value) -> {
if(value.isExpire(currentDate)) {
cache.remove(key);
}
});
}
public static class CacheObject<T> implements Comparable<CacheObject<T>> {

View File

@ -13,7 +13,7 @@ import java.util.Set;
import java.util.function.Consumer;
import java.util.function.Function;
public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
private final JedisPool jedisPool;
private final String keyPrefix;
@ -36,7 +36,7 @@ public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
if(jedisPool.isClosed()) {
throw new IllegalStateException("JedisPool is closed");
}
log = LoggerFactory.getLogger(this.getClass().getSimpleName() + "@" + Integer.toHexString(jedisPool.hashCode()));
log = LoggerFactory.getLogger(this.getClass().getName() + "@" + Integer.toHexString(jedisPool.hashCode()));
if(!Strings.isNullOrEmpty(keyPrefix)) {
this.keyPrefix = keyPrefix.endsWith(".") ? keyPrefix : keyPrefix + ".";
} else {
@ -82,9 +82,7 @@ public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
@Override
public boolean clear() {
try (Jedis jedis = jedisPool.getResource()) {
String result = jedis.flushDB();
log.info("flushDB返回结果: {}", result);
return true;
return jedis.flushDB().equalsIgnoreCase("ok");
}
}
@ -158,13 +156,4 @@ public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
return false;
}
/**
* 替换原本的分隔符(.)为(:).<br/>
* 即将启用
* @param key 要处理的键名
* @return 处理后的键名
*/
public static String replaceKey(String key) {
return key.replaceAll("\\.", ":");
}
}

View File

@ -0,0 +1,44 @@
package net.lamgc.cgj.bot.cache.exception;
import java.io.IOException;
import java.util.Objects;
import org.apache.http.HttpResponse;
import org.apache.http.StatusLine;
import org.apache.http.util.EntityUtils;
public class HttpRequestException extends IOException {
private static final long serialVersionUID = -2229221075943552798L;
private final StatusLine statusLine;
private final String content;
public HttpRequestException(HttpResponse response) throws IOException {
this(response.getStatusLine(), EntityUtils.toString(response.getEntity()));
}
public HttpRequestException(StatusLine statusLine, String content) {
super("Http Response Error: " + Objects.requireNonNull(statusLine, "statusLine is null") +
", Response Content: " + (content == null ? "null" : '\'' + content + '\''));
this.statusLine = statusLine;
this.content = content;
}
/**
* 获取Http状态行
*/
public StatusLine getStatusLine() {
return statusLine;
}
/**
* 获取Response内容
* @return 如果没有返回, 则返回null
*/
public String getContent() {
return content;
}
}

View File

@ -5,27 +5,26 @@ import com.google.common.base.Throwables;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import net.lamgc.cgj.bot.BotAdminCommandProcess;
import net.lamgc.cgj.bot.BotCommandProcess;
import net.lamgc.cgj.bot.MessageEventExecutionDebugger;
import net.lamgc.cgj.bot.SettingProperties;
import net.lamgc.cgj.util.DateParser;
import net.lamgc.cgj.util.PagesQualityParser;
import net.lamgc.cgj.util.TimeLimitThreadPoolExecutor;
import net.lamgc.utils.base.runner.ArgumentsRunner;
import net.lamgc.utils.base.runner.ArgumentsRunnerConfig;
import net.lamgc.utils.base.runner.exception.DeveloperRunnerException;
import net.lamgc.utils.base.runner.exception.NoSuchCommandException;
import net.lamgc.utils.base.runner.exception.ParameterNoFoundException;
import net.lamgc.utils.event.EventExecutor;
import net.lamgc.utils.event.EventHandler;
import net.lamgc.utils.event.*;
import net.lamgc.utils.event.EventObject;
import net.lamgc.utils.event.EventUncaughtExceptionHandler;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.JedisPool;
import java.lang.reflect.Method;
import java.net.URI;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.regex.Matcher;
@ -39,20 +38,17 @@ public class BotEventHandler implements EventHandler {
private final ArgumentsRunner processRunner;
private final ArgumentsRunner adminRunner;
private final static Logger log = LoggerFactory.getLogger("BotEventHandler");
private final static Logger log = LoggerFactory.getLogger(BotEventHandler.class);
/**
* 所有缓存共用的JedisPool
*/
private final static URI redisServerUri = URI.create("redis://" + System.getProperty("cgj.redisAddress"));
public final static JedisPool redisServer = new JedisPool(redisServerUri.getHost(), redisServerUri.getPort() == -1 ? 6379 : redisServerUri.getPort());
private final static Map<Long, AtomicBoolean> muteStateMap = new Hashtable<>();
/**
* 消息事件执行器
*/
public final static EventExecutor executor = new EventExecutor(new ThreadPoolExecutor(
(int) Math.ceil(Runtime.getRuntime().availableProcessors() / 2F),
Runtime.getRuntime().availableProcessors(),
private final static EventExecutor executor = new EventExecutor(new TimeLimitThreadPoolExecutor(
0,
Math.max(Runtime.getRuntime().availableProcessors(), 4),
Math.max(Math.max(Runtime.getRuntime().availableProcessors() * 2, 4), 32),
30L,
TimeUnit.SECONDS,
new LinkedBlockingQueue<>(1536),
@ -62,10 +58,10 @@ public class BotEventHandler implements EventHandler {
));
private static boolean initialled = false;
static {
initial();
}
/**
* 初始化BotEventHandler
*/
public synchronized static void initial() {
if(initialled) {
Logger logger = LoggerFactory.getLogger("BotEventHandler@<init>");
@ -88,25 +84,19 @@ public class BotEventHandler implements EventHandler {
});
try {
executor.addHandler(new BotEventHandler());
Thread shutdownThread = new Thread(() -> executor.shutdown(true));
shutdownThread.setName("Thread-EventHandlerShutdown");
Runtime.getRuntime().addShutdownHook(shutdownThread);
} catch (IllegalAccessException e) {
LoggerFactory.getLogger("BotEventHandler@Static").error("添加Handler时发生异常", e);
}
initialled = true;
}
private final static AtomicBoolean preLoaded = new AtomicBoolean();
/**
* 预加载
*/
public synchronized static void preLoad() {
if(preLoaded.get()) {
return;
}
try {
BotAdminCommandProcess.loadPushList();
} finally {
preLoaded.set(true);
} catch(Throwable e) {
log.error("加载推送列表失败", e);
}
initialled = true;
}
private BotEventHandler() {
@ -124,6 +114,29 @@ public class BotEventHandler implements EventHandler {
BotCommandProcess.initialize();
}
/**
* 投递消息事件
* @param event 事件对象
*/
@NotAccepted
public static void executeMessageEvent(MessageEvent event) {
String debuggerName = SettingProperties.getProperty(0, "debug.debugger");
if(!event.getMessage().startsWith(ADMIN_COMMAND_PREFIX) &&
!Strings.isNullOrEmpty(debuggerName)) {
try {
MessageEventExecutionDebugger debugger = MessageEventExecutionDebugger.valueOf(debuggerName.toUpperCase());
debugger.debugger.accept(executor, event, SettingProperties.getProperties(SettingProperties.GLOBAL),
MessageEventExecutionDebugger.getDebuggerLogger(debugger));
} catch(IllegalArgumentException e) {
log.warn("未找到指定调试器: '{}'", debuggerName);
} catch (Exception e) {
log.error("事件调试处理时发生异常", e);
}
} else {
BotEventHandler.executor.executor(event);
}
}
/**
* 以事件形式处理消息事件
* @param event 消息事件对象
@ -132,7 +145,10 @@ public class BotEventHandler implements EventHandler {
public void processMessage(MessageEvent event) {
String msg = event.getMessage();
log.debug(event.toString());
if(!match(msg)) {
if(mismatch(msg)) {
return;
} else if(isMute(event.getFromGroup())) {
log.debug("机器人已被禁言, 忽略请求.");
return;
}
@ -172,7 +188,8 @@ public class BotEventHandler implements EventHandler {
Object result;
try {
if(msg.toLowerCase().startsWith(ADMIN_COMMAND_PREFIX)) {
if(!String.valueOf(event.getFromQQ()).equals(BotCommandProcess.globalProp.getProperty("admin.adminId"))) {
if(!String.valueOf(event.getFromQQ())
.equals(SettingProperties.getProperty(0, "admin.adminId"))) {
result = "你没有执行该命令的权限!";
} else {
result = adminRunner.run(args.length <= 1 ? new String[0] : Arrays.copyOfRange(args, 1, args.length));
@ -185,18 +202,36 @@ public class BotEventHandler implements EventHandler {
} catch(ParameterNoFoundException e) {
result = "命令缺少参数: " + e.getParameterName();
} catch(DeveloperRunnerException e) {
log.error("执行命令时发生异常", e);
result = "命令执行时发生错误,无法完成!";
Throwable cause = e.getCause();
if (cause instanceof InterruptedException) {
log.error("命令执行超时, 终止执行.");
result = "色图姬发现这个命令的处理时间太久了!所以打断了这个命令。";
} else if(cause instanceof NoSuchElementException && cause.getMessage().startsWith("No work found: ")) {
String message = cause.getMessage();
log.error("指定作品不存在.(Id: {})", message.substring(message.lastIndexOf(": ") + 2));
result = "色图姬找不到这个作品!";
} else {
log.error("执行命令时发生异常", e);
result = "色图姬在执行命令时遇到了一个错误!";
}
}
log.info("命令处理完成.(耗时: {}ms)", System.currentTimeMillis() - time);
if(Objects.requireNonNull(result) instanceof String) {
long processTime = System.currentTimeMillis() - time;
if(!Objects.isNull(result) && result instanceof String && !isMute(event.getFromGroup())) {
try {
event.sendMessage((String) result);
int sendResult = event.sendMessage((String) result);
if(sendResult < 0) {
log.warn("消息发送失败, Sender {} 返回错误代码: {}", event.getClass().getName(), sendResult);
}
} catch (Exception e) {
log.error("发送消息时发生异常", e);
}
} else if(isMute(event.getFromGroup())) {
log.warn("命令反馈时机器人已被禁言, 跳过反馈.");
}
log.info("命令反馈完成.(耗时: {}ms)", System.currentTimeMillis() - time);
long totalTime = System.currentTimeMillis() - time;
log.info("命令反馈完成.(事件耗时: {}ms, P: {}%({}ms), R: {}%({}ms))", totalTime,
String.format("%.3f", ((double) processTime / (double)totalTime) * 100F), processTime,
String.format("%.3f", ((double) (totalTime - processTime) / (double)totalTime) * 100F), totalTime - processTime);
}
/**
@ -204,8 +239,44 @@ public class BotEventHandler implements EventHandler {
* @param message 要检查的消息
* @return 如果为true则提交
*/
public static boolean match(String message) {
return message.startsWith(COMMAND_PREFIX) || message.startsWith(ADMIN_COMMAND_PREFIX);
public static boolean mismatch(String message) {
return !message.startsWith(COMMAND_PREFIX) && !message.startsWith(ADMIN_COMMAND_PREFIX);
}
private static boolean isMute(long groupId) {
Boolean mute = isMute(groupId, false);
return mute != null && mute;
}
/**
* 查询某群是否被禁言.
* @param groupId 群组Id
* @param rawValue 是否返回原始值(当没有该群状态, 且本参数为true时, 将返回null)
* @return 返回状态值, 如无该群禁言记录且rawValue = true, 则返回null
*/
public static Boolean isMute(long groupId, boolean rawValue) {
if(groupId <= 0) {
return false;
}
AtomicBoolean state = muteStateMap.get(groupId);
if(state == null && rawValue) {
return null;
}
return state != null && state.get();
}
/**
* 设置机器人禁言状态.
* <p>设置该项可防止因机器人在禁言期间反馈请求导致被封号.</p>
* @param mute 如果被禁言, 传入true
*/
public static void setMuteState(long groupId, boolean mute) {
if(!muteStateMap.containsKey(groupId)) {
muteStateMap.put(groupId, new AtomicBoolean(mute));
} else {
muteStateMap.get(groupId).set(mute);
}
log.warn("群组 {} 机器人禁言状态已变更: {}", groupId, mute ? "已禁言" : "已解除");
}
}

View File

@ -0,0 +1,70 @@
package net.lamgc.cgj.bot.event;
import java.util.Objects;
public class BufferMessageEvent extends MessageEvent {
private final StringBuffer buffer = new StringBuffer();
private final MessageEvent parent;
/**
* 以空消息空Id生成BufferMessageEvent
*/
public BufferMessageEvent() {
super(0, 0, "");
parent = null;
}
/**
* 提供消息内容构造BufferMessageEvent
* @param message 传入的消息内容
*/
public BufferMessageEvent(String message) {
super(0, 0, message);
parent = null;
}
/**
* 提供消息内容构和Id信息造BufferMessageEvent
* @param groupId 群组Id
* @param qqId 发送者Id
* @param message 传入的消息内容
*/
public BufferMessageEvent(int groupId, int qqId, String message) {
super(groupId, qqId, message);
parent = null;
}
/**
* 使用事件构造BufferMessageEvent
* @param parentEvent 父级消息事件对象
*/
public BufferMessageEvent(MessageEvent parentEvent) {
super(parentEvent.getFromGroup(), parentEvent.getFromQQ(), parentEvent.getMessage());
parent = parentEvent;
}
@Override
public int sendMessage(String message) {
buffer.append(message);
return 0;
}
/**
* 当提供了父级消息事件时, 本方法调用父级消息事件对象的{@code getImageUrl(String)}, 如果没有, 返回{@code null}
*/
@Override
public String getImageUrl(String image) {
return Objects.isNull(this.parent) ? null : this.parent.getImageUrl(image);
}
/**
* 获取缓冲区消息内容
* @return 消息内容
*/
public String getBufferMessage() {
return buffer.toString();
}
}

View File

@ -12,16 +12,17 @@ public abstract class MessageEvent implements EventObject, MessageSender {
public MessageEvent(long fromGroup, long fromQQ, String message) {
this.fromGroup = fromGroup;
this.fromQQ = fromQQ;
this.message = message;
this.message = message.trim();
}
/**
* 发送消息
* @param message 消息内容
* @return 成功返回MessageId, 如没有MessageId则返回0, 失败返回负数错误码
* @throws Exception 该方法根据不同实现, 可能会抛出不同异常, 详见实现所标识的文档内容.
*/
@Override
public abstract int sendMessage(final String message);
public abstract int sendMessage(final String message) throws Exception;
/**
* 获取图片下载地址.

View File

@ -5,6 +5,28 @@ package net.lamgc.cgj.bot.event;
*/
public class VirtualLoadMessageEvent extends MessageEvent {
/**
* 将任意消息事件转换为假负载消息事件.
* <p>转换之后, 除了fromGroup, fromQQ, message外其他信息不会保留</p>
* @param event 待转换的消息事件
* @param inheritImpl 是否继承除 sendMessage 外的其他 MessageEvent 实现
* @return 转换后的消息事件
*/
public static VirtualLoadMessageEvent toVirtualLoadMessageEvent(MessageEvent event, boolean inheritImpl) {
if(event instanceof VirtualLoadMessageEvent) {
return (VirtualLoadMessageEvent) event;
} else if(!inheritImpl) {
return new VirtualLoadMessageEvent(event.getFromGroup(), event.getFromQQ(), event.getMessage());
} else {
return new VirtualLoadMessageEvent(event.getFromGroup(), event.getFromQQ(), event.getMessage()) {
@Override
public String getImageUrl(String image) {
return event.getImageUrl(image);
}
};
}
}
public VirtualLoadMessageEvent(long fromGroup, long fromQQ, String message) {
super(fromGroup, fromQQ, message);
}

View File

@ -0,0 +1,30 @@
package net.lamgc.cgj.bot.framework.cli;
import net.lamgc.cgj.bot.boot.ApplicationBoot;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import java.util.Scanner;
public class ConsoleMain {
public static void start() {
MessageSenderBuilder.setCurrentMessageSenderFactory(new ConsoleMessageSenderFactory());
ApplicationBoot.initialBot();
Scanner scanner = new Scanner(System.in);
boolean isGroup = false;
do {
String input = scanner.nextLine();
if(input.equalsIgnoreCase("#exit")) {
System.out.println("退出应用...");
break;
} else if(input.equalsIgnoreCase("#setgroup")) {
isGroup = !isGroup;
System.out.println("System: 群模式状态已变更: " + isGroup);
continue;
}
BotEventHandler.executeMessageEvent(new ConsoleMessageEvent(input, isGroup));
} while(true);
}
}

View File

@ -0,0 +1,23 @@
package net.lamgc.cgj.bot.framework.cli;
import net.lamgc.cgj.bot.event.MessageEvent;
import java.util.Date;
public class ConsoleMessageEvent extends MessageEvent {
public ConsoleMessageEvent(String message, boolean isGroup) {
super(isGroup ? 1 : 0, 1, message);
}
@Override
public int sendMessage(String message) {
System.out.println(new Date() + " Bot: " + message);
return 0;
}
@Override
public String getImageUrl(String image) {
return null;
}
}

View File

@ -0,0 +1,13 @@
package net.lamgc.cgj.bot.framework.cli;
import net.lamgc.cgj.bot.message.MessageSender;
import java.util.Date;
public class ConsoleMessageSender implements MessageSender {
@Override
public synchronized int sendMessage(String message) {
System.out.println(new Date() + " Bot: " + message);
return 0;
}
}

View File

@ -0,0 +1,15 @@
package net.lamgc.cgj.bot.framework.cli;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSource;
public class ConsoleMessageSenderFactory implements MessageSenderFactory {
private final static ConsoleMessageSender sender = new ConsoleMessageSender();
@Override
public MessageSender createMessageSender(MessageSource source, long id) {
return sender;
}
}

View File

@ -1,5 +1,6 @@
package net.lamgc.cgj.bot.framework.coolq;
import net.lamgc.cgj.bot.boot.ApplicationBoot;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.framework.coolq.message.SpringCQMessageEvent;
import net.lamgc.utils.event.EventHandler;
@ -17,34 +18,38 @@ public class CQPluginMain extends CQPlugin implements EventHandler {
public CQPluginMain() {
// TODO(LamGC, 2020.04.21): SpringCQ无法适配MessageSenderBuilder
BotEventHandler.preLoad();
LoggerFactory.getLogger(this.toString())
// MessageSenderBuilder.setCurrentMessageSenderFactory(new SpringCQMessageSenderFactory());
ApplicationBoot.initialBot();
LoggerFactory.getLogger(CQPluginMain.class)
.info("BotEventHandler.COMMAND_PREFIX = {}", BotEventHandler.COMMAND_PREFIX);
}
@Override
public int onPrivateMessage(CoolQ cq, CQPrivateMessageEvent event) {
//log.info("私聊消息到达: 发送者[{}], 消息内容: {}", event.getSender().getUserId(), event.getMessage());
return processMessage(cq, event);
}
@Override
public int onGroupMessage(CoolQ cq, CQGroupMessageEvent event) {
//log.info("群消息到达: 群[{}], 发送者[{}], 消息内容: {}", event.getGroupId(), event.getSender().getUserId(), event.getMessage());
return processMessage(cq, event);
}
@Override
public int onDiscussMessage(CoolQ cq, CQDiscussMessageEvent event) {
//log.info("讨论组消息到达: 群[{}], 发送者[{}], 消息内容: {}", event.getDiscussId(), event.getSender().getUserId(), event.getMessage());
return processMessage(cq, event);
}
public int processMessage(CoolQ cq, CQMessageEvent event) {
if(!BotEventHandler.match(event.getMessage())) {
/**
* 处理消息
* @param cq CoolQ机器人对象
* @param event 消息事件
* @return 是否拦截消息
*/
private static int processMessage(CoolQ cq, CQMessageEvent event) {
if(BotEventHandler.mismatch(event.getMessage())) {
return MESSAGE_IGNORE;
}
BotEventHandler.executor.executor(new SpringCQMessageEvent(cq, event));
BotEventHandler.executeMessageEvent(new SpringCQMessageEvent(cq, event));
return MESSAGE_BLOCK;
}

View File

@ -36,7 +36,7 @@ public class SpringCQMessageEvent extends MessageEvent {
}
@Override
public int sendMessage(final String message) {
public int sendMessage(final String message) throws Exception {
return messageSender.sendMessage(message);
}

View File

@ -5,16 +5,14 @@ import net.lamgc.cgj.bot.message.MessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lz1998.cq.robot.CoolQ;
import java.util.Objects;
public class SpringCQMessageSenderFactory implements MessageSenderFactory {
private final CoolQ coolQ;
public SpringCQMessageSenderFactory(CoolQ coolQ) {
this.coolQ = coolQ;
}
private final static ThreadLocal<CoolQ> threadCoolQ = new ThreadLocal<>();
@Override
public MessageSender createMessageSender(MessageSource source, long id) {
return new SpringCQMessageSender(coolQ, source, id);
return new SpringCQMessageSender(
Objects.requireNonNull(threadCoolQ.get(), "CoolQ object is not included in ThreadLocal"), source, id);
}
}

View File

@ -1,14 +1,17 @@
package net.lamgc.cgj.bot.framework.mirai;
import net.lamgc.cgj.bot.boot.ApplicationBoot;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.framework.mirai.message.MiraiMessageEvent;
import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import net.lamgc.cgj.bot.framework.mirai.message.MiraiMessageSenderFactory;
import net.mamoe.mirai.Bot;
import net.mamoe.mirai.BotFactoryJvm;
import net.mamoe.mirai.event.events.BotMuteEvent;
import net.mamoe.mirai.event.events.BotUnmuteEvent;
import net.mamoe.mirai.japt.Events;
import net.mamoe.mirai.message.FriendMessage;
import net.mamoe.mirai.message.GroupMessage;
import net.mamoe.mirai.message.*;
import net.mamoe.mirai.utils.BotConfiguration;
import org.apache.commons.net.util.Base64;
import org.slf4j.Logger;
@ -19,7 +22,7 @@ import java.util.Properties;
public class MiraiMain implements Closeable {
private final Logger log = LoggerFactory.getLogger(this.toString());
private final Logger log = LoggerFactory.getLogger(MiraiMain.class);
private Bot bot;
@ -34,7 +37,7 @@ public class MiraiMain implements Closeable {
return;
}
File botPropFile = new File("./bot.properties");
File botPropFile = new File(BotGlobal.getGlobal().getDataStoreDir(), "./bot.properties");
try (Reader reader = new BufferedReader(new FileReader(botPropFile))) {
botProperties.load(reader);
} catch (IOException e) {
@ -42,18 +45,47 @@ public class MiraiMain implements Closeable {
return;
}
bot = BotFactoryJvm.newBot(Long.parseLong(botProperties.getProperty("bot.qq", "0")), Base64.decodeBase64(botProperties.getProperty("bot.password", "")), new BotConfiguration());
Events.subscribeAlways(GroupMessage.class, (msg) -> BotEventHandler.executor.executor(new MiraiMessageEvent(msg)));
Events.subscribeAlways(FriendMessage.class, (msg) -> BotEventHandler.executor.executor(new MiraiMessageEvent(msg)));
BotConfiguration configuration = new BotConfiguration();
configuration.setProtocol(BotConfiguration.MiraiProtocol.ANDROID_PAD);
bot = BotFactoryJvm.newBot(Long.parseLong(botProperties.getProperty("bot.qq", "0")), Base64.decodeBase64(botProperties.getProperty("bot.password", "")), configuration);
Events.subscribeAlways(GroupMessageEvent.class, this::executeMessageEvent);
Events.subscribeAlways(FriendMessageEvent.class, this::executeMessageEvent);
Events.subscribeAlways(TempMessageEvent.class, this::executeMessageEvent);
Events.subscribeAlways(BotMuteEvent.class,
event -> BotEventHandler.setMuteState(event.getGroup().getId(), true));
Events.subscribeAlways(BotUnmuteEvent.class,
event -> BotEventHandler.setMuteState(event.getGroup().getId(), false));
bot.login();
MessageSenderBuilder.setCurrentMessageSenderFactory(new MiraiMessageSenderFactory(bot));
BotEventHandler.preLoad();
ApplicationBoot.initialBot();
bot.join();
}
public void close() {
/**
* 处理消息事件
* @param message 消息事件对象
*/
private void executeMessageEvent(MessageEvent message) {
if(message instanceof GroupMessageEvent) {
GroupMessageEvent GroupMessageEvent = (GroupMessageEvent) message;
if(BotEventHandler.isMute(GroupMessageEvent.getGroup().getId(), true) == null) {
BotEventHandler.setMuteState(GroupMessageEvent.getGroup().getId(),
((GroupMessageEvent) message).getGroup().getBotMuteRemaining() != 0);
}
}
BotEventHandler.executeMessageEvent(MiraiMessageEvent.covertEventObject(message));
}
/**
* 关闭机器人
*/
public synchronized void close() {
if(bot == null) {
return;
}
log.warn("正在关闭机器人...");
bot.close(null);
bot = null;
log.warn("机器人已关闭.");
}

View File

@ -1,33 +1,92 @@
package net.lamgc.cgj.bot.framework.mirai.message;
import net.lamgc.cgj.bot.event.MessageEvent;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.message.ContactMessage;
import net.mamoe.mirai.message.GroupMessage;
import net.mamoe.mirai.message.FriendMessageEvent;
import net.mamoe.mirai.message.GroupMessageEvent;
import net.mamoe.mirai.message.MessageEvent;
import net.mamoe.mirai.message.TempMessageEvent;
import net.mamoe.mirai.message.data.MessageChain;
import net.mamoe.mirai.message.data.MessageUtils;
import java.util.Objects;
public class MiraiMessageEvent extends MessageEvent {
public class MiraiMessageEvent extends net.lamgc.cgj.bot.event.MessageEvent {
private final ContactMessage messageObject;
private final MessageEvent messageObject;
private final MessageSender messageSender;
public MiraiMessageEvent(ContactMessage message) {
super(message instanceof GroupMessage ? ((GroupMessage) message).getGroup().getId() : 0,
message.getSender().getId(), message.getMessage().contentToString());
/**
* 通过Mirai的MessageEvent转换成应用支持的MessageEvent.
* @deprecated 请使用 {@link #covertEventObject(MessageEvent)}方法转换.
* @param message 消息对象
* @see #covertEventObject(MessageEvent)
*/
@Deprecated
public MiraiMessageEvent(MessageEvent message) {
super(message instanceof GroupMessageEvent ? ((GroupMessageEvent) message).getGroup().getId() : 0,
message.getSender().getId(), getMessageBodyWithoutSource(message.getMessage().toString()));
this.messageObject = Objects.requireNonNull(message);
if(message instanceof GroupMessage) {
messageSender = new MiraiMessageSender(((GroupMessage) message).getGroup(), MessageSource.Group);
if(message instanceof GroupMessageEvent) {
messageSender = new MiraiMessageSender(((GroupMessageEvent) message).getGroup(), MessageSource.Group);
} else {
messageSender = new MiraiMessageSender(message.getSender(), MessageSource.Private);
}
}
/**
* 通过解析好的信息构造MessageEvent
* @param messageObject 消息原始对象
* @param groupId 群组Id, 非群聊或无需使用群聊时, 该参数为0
* @param qqId 发送者Id, 不能为0
* @param message 原始消息内容对象, 由构造方法内部解析
*/
private MiraiMessageEvent(MessageEvent messageObject, long groupId, long qqId, MessageChain message) {
super(groupId, qqId, getMessageBodyWithoutSource(message.toString()));
this.messageObject = Objects.requireNonNull(messageObject, "messageObject is null");
if(groupId != 0) {
this.messageSender = new MiraiMessageSender(((GroupMessageEvent) messageObject).getGroup(), MessageSource.Group);
} else {
this.messageSender = new MiraiMessageSender(messageObject.getSender(), MessageSource.Group);
}
}
/**
* 将Mirai原始MessageEvent转换成应用支持的MessageEvent对象
* @param event 原始消息对象
* @return 原始消息对象所对应的应用MessageEvent对象.
* @throws IllegalArgumentException 当出现不支持的Mirai {@link MessageEvent}实现时将抛出异常.
* @see MessageEvent 原始消息对象
* @see net.lamgc.cgj.bot.event.MessageEvent 应用消息对象
*/
public static MiraiMessageEvent covertEventObject(MessageEvent event) throws IllegalArgumentException {
if(event instanceof GroupMessageEvent) {
return new MiraiMessageEvent(event,
((GroupMessageEvent) event).getGroup().getId(), event.getSender().getId(), event.getMessage());
} else if(event instanceof FriendMessageEvent) {
return new MiraiMessageEvent(event, 0, event.getSender().getId(), event.getMessage());
} else if(event instanceof TempMessageEvent) {
return new MiraiMessageEvent(event, 0, event.getSender().getId(), event.getMessage());
} else {
throw new IllegalArgumentException("Unsupported event type: " + event.toString());
}
}
/**
* 将ContactMessage获得的消息内容删除 Mirai:source 并返回.
* <p>该做法比较保守, 防止Mirai:source位置出现变动.</p>
* @param message ContactMessage的消息内容;
* @return 返回删除了Mirai:source的消息
*/
private static String getMessageBodyWithoutSource(String message) {
StringBuilder builder = new StringBuilder(message);
int startIndex = builder.indexOf("[mirai:source:");
int endIndex = builder.indexOf("]", startIndex) + 1;
return builder.delete(startIndex, endIndex).toString();
}
@Override
public int sendMessage(final String message) {
public int sendMessage(final String message) throws Exception {
return messageSender.sendMessage(message);
}

View File

@ -2,19 +2,16 @@ package net.lamgc.cgj.bot.framework.mirai.message;
import com.google.common.base.Strings;
import net.lamgc.cgj.bot.BotCode;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.cache.CacheStore;
import net.lamgc.cgj.bot.cache.HotDataCacheStore;
import net.lamgc.cgj.bot.cache.LocalHashCacheStore;
import net.lamgc.cgj.bot.cache.StringRedisCacheStore;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.Bot;
import net.mamoe.mirai.contact.Contact;
import net.mamoe.mirai.message.data.Image;
import net.mamoe.mirai.message.data.Message;
import net.mamoe.mirai.message.data.MessageChain;
import net.mamoe.mirai.message.data.MessageUtils;
import net.mamoe.mirai.message.data.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@ -29,11 +26,11 @@ public class MiraiMessageSender implements MessageSender {
private final Contact member;
private final MessageSource source;
private final static Logger log = LoggerFactory.getLogger("MiraiMessageSender");
private final static Logger log = LoggerFactory.getLogger(MiraiMessageSender.class);
private final static CacheStore<String> imageIdCache = new HotDataCacheStore<>(
new StringRedisCacheStore(BotEventHandler.redisServer, "mirai.imageId"),
new StringRedisCacheStore(BotGlobal.getGlobal().getRedisServer(), "mirai.imageId"),
new LocalHashCacheStore<>(),
5400000, 1800000);
5400000, 1800000, true);
/**
* 使用id构造发送器
@ -60,7 +57,7 @@ public class MiraiMessageSender implements MessageSender {
public int sendMessage(final String message) {
log.debug("处理前的消息内容:\n{}", message);
Message msgBody = processMessage(Objects.requireNonNull(message));
log.debug("处理后的消息内容(可能出现乱序的情况, 但实际上顺序是没问题的):\n{}", msgBody);
log.debug("处理后的消息内容(可能出现乱序的情况, 但实际上顺序是没问题的):\n{}", msgBody.contentToString());
member.sendMessage(msgBody);
return 0;
}
@ -80,7 +77,7 @@ public class MiraiMessageSender implements MessageSender {
.replaceAll("&38", "&")
.split("\\|");
MessageChain messages = MessageUtils.newChain().plus("");
MessageChain messages = MessageUtils.newChain();
int codeIndex = 0;
for(String text : texts) {
if(text.equals("{BotCode}")) {
@ -111,11 +108,21 @@ public class MiraiMessageSender implements MessageSender {
} else {
return MessageUtils.newChain("(参数不存在)");
}
if(code.getParameter("flashImage").equalsIgnoreCase("true")) {
if(Strings.nullToEmpty(code.getParameter("flashImage"))
.equalsIgnoreCase("true")) {
return MessageUtils.flash(img);
} else {
return img;
}
case "face":
if(!code.containsParameter("id")) {
return MessageUtils.newChain("(无效的表情Id)");
}
int faceId = Integer.parseInt(code.getParameter("id"));
if(faceId <= 0) {
return MessageUtils.newChain("(无效的表情Id)");
}
return new Face(faceId);
default:
log.warn("解析到不支持的BotCode: {}", code);
return MessageUtils.newChain("(不支持的BotCode)");
@ -155,7 +162,7 @@ public class MiraiMessageSender implements MessageSender {
}
}
imageIdCache.update(imageName, image.getImageId(), expireTime);
log.info("imageName [{}] 缓存更新完成.(有效时间: {})", imageName, expireTime);
log.debug("imageName [{}] 缓存更新完成.(有效时间: {})", imageName, expireTime);
} else {
log.debug("ImageName: [{}] 缓存命中.", imageName);
}

View File

@ -5,6 +5,8 @@ import net.lamgc.cgj.bot.message.MessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.Bot;
import java.util.Objects;
public class MiraiMessageSenderFactory implements MessageSenderFactory {
private final Bot bot;
@ -14,15 +16,11 @@ public class MiraiMessageSenderFactory implements MessageSenderFactory {
}
@Override
public MessageSender createMessageSender(MessageSource source, long id) throws Exception {
switch(source) {
case Group:
case Discuss:
return new MiraiMessageSender(bot.getGroup(id), source);
case Private:
return new MiraiMessageSender(bot.getFriend(id), source);
default:
throw new NoSuchFieldException(source.toString());
public MessageSender createMessageSender(MessageSource source, long id) {
Objects.requireNonNull(source);
if(id <= 0) {
throw new IllegalArgumentException("id cannot be 0 or negative: " + id);
}
return new MiraiMessageSender(bot, source, id);
}
}

View File

@ -9,7 +9,8 @@ public interface MessageSender {
* 返回0则发送器不支持消息Id,
* 返回非0正整数则为消息Id,
* 返回负数则为错误.
* @throws Exception 该方法根据不同实现, 可能会抛出不同异常, 详见实现所标识的文档内容.
*/
int sendMessage(final String message);
int sendMessage(final String message) throws Exception;
}

View File

@ -18,7 +18,7 @@ public interface AdultContentDetector {
* @param pageIndex 指定页数, 设为0或负数则视为单页面作品
* @return 如果为true则为成人作品, 该方法将由检测器决定如何为成人作品.
*/
boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex) throws Exception;
boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex) throws Exception;
/**
* 检查某一作品是否为成人内容
@ -26,8 +26,8 @@ public interface AdultContentDetector {
* @param isUgoira 是否为动图
* @param pageIndex 指定页数, 设为0或负数则视为单页面作品
* @param threshold 指数阀值, 当等于或大于该阀值时返回true.
* @return 如果为true则为成人作品, 该方法将由 threshold 参数决定是否为成人作品.
* @return 如果为true则为成人作品, 该方法将由 threshold 参数决定是否为成人作品(如果超过阈值, 则为true).
*/
boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex, double threshold) throws Exception;
boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex, double threshold) throws Exception;
}

View File

@ -1,69 +0,0 @@
package net.lamgc.cgj.pixiv;
/**
* 插图(集)的信息
*/
public class IllustInfo{
/**
* 插图(集)ID
*/
public final int illustID;
/**
* 第几页
*/
public final int page;
/**
* 插图标题
*/
public final String title;
/**
* 插图说明
*/
public final String description;
/**
* 插图标签
*/
public final String[] tags;
/**
* 插图图片长度
*/
public final int width;
/**
* 插图图片高度
*/
public final int height;
/**
* 作者名
*/
public final String authorName;
/**
* 作者用户ID
*/
public final int authorUserID;
public IllustInfo(int illustID, String title, String description, String[] tags, int width, int height, String authorName, int authorUserID){
this(illustID, 0, title, description, tags, width, height, authorName, authorUserID);
}
public IllustInfo(int illustID, int p, String title, String description, String[] tags, int width, int height, String authorName, int authorUserID){
this.illustID = illustID;
this.page = p;
this.title = title;
this.description = description;
this.tags = tags;
this.width = width;
this.height = height;
this.authorName = authorName;
this.authorUserID = authorUserID;
}
}

View File

@ -1,267 +0,0 @@
package net.lamgc.cgj.pixiv;
import com.google.gson.JsonArray;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.conn.ConnectionPoolTimeoutException;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.*;
public class Pixiv {
/**
* illust Link
*/
public final static String ATTR_LINK = "link";
/**
* illust Id
*/
public final static String ATTR_ILLUST_ID = "illustId";
/**
* illust Title
*/
public final static String ATTR_TITLE = "title";
/**
* illust Author Name
*/
public final static String ATTR_AUTHOR_NAME = "authorName";
/**
* illust Author UserID
*/
public final static String ATTR_AUTHOR_ID = "authorId";
private final HttpClient httpClient;
public Pixiv(HttpClient client){
this.httpClient = client;
}
/**
* 使用帐号密码登录Pixiv
* @param PixivID Pixiv账户登录名
* @param Password Pixiv帐号密码
* @throws IOException 当登录连接出现异常时抛出
* @deprecated {@link PixivSession#Login(String, String)} 已经废除, 故本方法不可用
*/
public Pixiv(String PixivID, String Password) throws IOException {
this(PixivID, Password, null);
}
/**
* 使用帐号密码登录Pixiv
* @param PixivID Pixiv账户登录名
* @param Password Pixiv帐号密码
* @param proxy 代理设置
* @throws IOException 当登录连接出现异常时抛出
* @deprecated {@link PixivSession#Login(String, String)} 已经废除, 故本方法不可用
*/
public Pixiv(String PixivID, String Password, HttpHost proxy) throws IOException {
PixivSession pixivSession = new PixivSession(proxy, null);
if(pixivSession.Login(PixivID, Password)){
System.out.println("P站登录成功!");
}else{
System.out.println("P站登录失败!错误信息: " + pixivSession.getErrMsg());
throw new RuntimeException(pixivSession.getErrMsg());
}
//httpClient = pixivSession.getHttpClient();
httpClient = HttpClientBuilder.create()
.setDefaultCookieStore(pixivSession.getCookieStore())
.build();
}
/**
* 获取首页推荐列表
* @return 首页推荐列表, 一个Map对应一个推荐项, 使用<code>ATTR_</code>开头常量访问即可
* @throws IOException
*/
public List<Map<String, String>> getRecommend() throws IOException {
HttpGet getRequest = new HttpGet(PixivURL.PIXIV_INDEX_URL);
HttpResponse response = httpClient.execute(getRequest);
String pageAsXML = EntityUtils.toString(response.getEntity(),"utf-8");
//获取推荐图列表(li)
//System.out.println(pageAsXML);
Document document = Jsoup.parse(pageAsXML);
List<String> links = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-thumbnail-link").eachAttr("href");
List<String> illustId = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-thumbnail-link").eachAttr("data-gtm-recommend-illust-id");
List<String> title = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-title>h1").eachAttr("title");
List<String> authorName = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-user-name").eachText();
List<String> authorId = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-user-name").eachAttr("data-user_id");
List<Map<String, String>> recommendList = new ArrayList<>();
for(int i = 0; i < links.size(); i++){
//System.out.println(links.get(i));
Map<String, String> info = new HashMap<>();
info.put(ATTR_LINK, PixivURL.PIXIV_INDEX_URL + links.get(i));
info.put(ATTR_ILLUST_ID, illustId.get(i));
info.put(ATTR_TITLE, title.get(i));
info.put(ATTR_AUTHOR_NAME, authorName.get(i));
info.put(ATTR_AUTHOR_ID, authorId.get(i));
recommendList.add(info);
}
return recommendList;
}
public String[] getAllDownloadLink(int illustID) throws IOException {
HttpGet illustPage = new HttpGet(PixivURL.PIXIV_ILLUST_API_URL.replaceAll("\\{illustId}", String.valueOf(illustID)));
HttpResponse response = httpClient.execute(illustPage);
String pageAsXML = EntityUtils.toString(response.getEntity(),"utf-8");
//System.out.println(pageAsXML);
JsonObject resultObj = (JsonObject) new JsonParser().parse(pageAsXML);
if(!resultObj.get("error").getAsBoolean()){
JsonArray bodyArray = resultObj.get("body").getAsJsonArray();
int length = bodyArray.size();
String[] result = new String[length];
for(int i = 0; i < length; i++){
JsonObject childObj = bodyArray.get(i).getAsJsonObject();
result[i] = childObj.get("urls").getAsJsonObject().get("original").getAsString();
}
return result;
}else{
return null;
}
}
/**
* 下载P站图片
* @param illustID 插图ID
* @return 成功返回图片输入流,失败或为多图则返回null
*/
public InputStream[] downloadIllustImage(int illustID) throws IOException {
String[] links = getAllDownloadLink(illustID);
List<InputStream> inputStreamList = new ArrayList<>();
int count = 1;
boolean retry = false;
for(int i = 0; i < links.length; i++){
try {
long sleepTime = (new Random().nextInt(4) + 2) * 1000;
System.out.println("nextTime: " + (float)(sleepTime / 1000));
Thread.sleep(sleepTime);
} catch (InterruptedException ignored) {}
String link = links[i];
System.out.print("page:" + count++ + "/" + links.length + " ...");
HttpGet imgGet = new HttpGet(link);
//关键!如果不加上Referer的话,会返回403
imgGet.setHeader("Referer", PixivURL.PIXIV_ILLUST_MEDIUM_URL.replaceAll("\\{illustId}", String.valueOf(illustID)));
RequestConfig config = RequestConfig.custom()
.setConnectTimeout(20 * 1000)
.setConnectionRequestTimeout(20 * 1000)
.setSocketTimeout(30 * 1000)
.build();
imgGet.setConfig(config);
HttpResponse response;
try {
response = httpClient.execute(imgGet);
}catch(ConnectionPoolTimeoutException e){
if(retry){
retry = false;
System.out.println("获取失败,跳过...");
continue;
}
System.out.println("连接超时,重新获取...");
retry = true;
i--;
continue;
}
retry = false;
ByteArrayOutputStream cacheOutputStream = new ByteArrayOutputStream((int)response.getEntity().getContentLength());
InputStream content = response.getEntity().getContent();
int readLength;
byte[] cache = new byte[4096];
while((readLength = content.read(cache)) != -1){
cacheOutputStream.write(cache, 0, readLength);
}
byte[] data = cacheOutputStream.toByteArray();
//System.out.println("读到数据: " + data.length);
inputStreamList.add(new ByteArrayInputStream(data));
System.out.println("done!length: " + response.getEntity().getContentLength() + ")");
}
return inputStreamList.toArray(new InputStream[0]);
}
/**
* 下载P站图片
* @param illustID 插图ID
* @return 成功返回图片输入流,失败或为多图则返回null
*/
public InputStream downloadIllustImages(int illustID){
throw new UnsupportedOperationException();
}
/**
* 通过解析插图详情页获取
* - 插图标题
* - 插图作者(及其UserId)
* - 插图上传时间
* - 插图标签(原始标签)
* ...
* @return 成功返回IllustInfo对象,失败返回null
*/
public IllustInfo[] getIllustInfo(int[] illustIDs) throws IOException {
//获取Api
HttpGet apiRequest = new HttpGet(PixivURL.getPixivIllustInfoAPI(illustIDs));
HttpResponse response = httpClient.execute(apiRequest);
String resultText = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
System.out.println(resultText);
JsonObject resultObj = ((JsonObject) new JsonParser().parse(resultText));
if(resultObj.get("error").getAsBoolean()){
System.err.println("获取失败!");
return null;
}
List<IllustInfo> illustInfoList = new ArrayList<>();
JsonArray illustArray = resultObj.get("body").getAsJsonObject().get("illusts").getAsJsonArray();
illustArray.forEach(jsonElement -> {
JsonObject illustInfoObj = jsonElement.getAsJsonObject();
JsonArray tagsArray = illustInfoObj.get("tags").getAsJsonArray();
String[] tags = new String[tagsArray.size()];
for(int i = 0; i < tags.length; i++){
tags[i] = tagsArray.get(i).getAsString();
}
//TODO: 通过不需要作者id就能获取图片信息的api无法获取图片尺寸
IllustInfo illustInfo = new IllustInfo(
illustInfoObj.get("workId").getAsInt(),
illustInfoObj.get("title").getAsString(),
null,
tags,
-1,
-1,
illustInfoObj.get("userName").getAsString(),
illustInfoObj.get("userId").getAsInt()
);
});
return null;
}
/**
* 获取指定用户的所有插画
*/
public void getUserAllIllustTest() {
}
}

View File

@ -33,7 +33,7 @@ import java.util.function.BiConsumer;
public class PixivDownload {
private final static Logger log = LoggerFactory.getLogger("PixivDownload");
private final static Logger log = LoggerFactory.getLogger(PixivDownload.class);
private final HttpClient httpClient;
@ -285,7 +285,8 @@ public class PixivDownload {
}
/**
* 获取排行榜
* 获取排行榜.
* <p>注意: 如果范围实际上没超出, 但返回排行榜不足, 会导致与实际请求的数量不符, 需要检查</p>
* @param contentType 排行榜类型
* @param mode 排行榜模式
* @param time 查询时间
@ -316,7 +317,8 @@ public class PixivDownload {
int count = 0;
Gson gson = new Gson();
ArrayList<JsonObject> results = new ArrayList<>(range);
for (int pageIndex = startPages; pageIndex <= endPages && count < range; pageIndex++) {
boolean canNext = true;
for (int pageIndex = startPages; canNext && pageIndex <= endPages && count < range; pageIndex++) {
HttpGet request = createHttpGetRequest(PixivURL.getRankingLink(contentType, mode, time, pageIndex, true));
log.debug("RequestUri: {}", request.getURI());
HttpResponse response = httpClient.execute(request);
@ -326,10 +328,13 @@ public class PixivDownload {
throw new IOException("Http Response Error: '" + response.getStatusLine() + "', ResponseBody: '" + responseBody + '\'');
}
JsonArray resultArray = gson.fromJson(responseBody, JsonObject.class).getAsJsonArray("contents");
JsonObject resultObject = gson.fromJson(responseBody, JsonObject.class);
canNext = resultObject.get("next").getAsJsonPrimitive().isNumber();
JsonArray resultArray = resultObject.getAsJsonArray("contents");
for (int resultIndex = startIndex; resultIndex < resultArray.size() && count < range; resultIndex++, count++) {
results.add(resultArray.get(resultIndex).getAsJsonObject());
}
// 重置索引
startIndex = 0;
}
@ -537,8 +542,9 @@ public class PixivDownload {
* }
* </pre>
* @throws IOException 当请求发生异常, 或接口返回错误信息时抛出.
* @throws NoSuchElementException 当该作品不存在时抛出异常
*/
public JsonObject getIllustInfoByIllustId(int illustId) throws IOException {
public JsonObject getIllustInfoByIllustId(int illustId) throws IOException, NoSuchElementException {
HttpGet request = createHttpGetRequest(PixivURL.getPixivIllustInfoAPI(illustId));
HttpResponse response = httpClient.execute(request);
String responseStr = EntityUtils.toString(response.getEntity());
@ -553,7 +559,7 @@ public class PixivDownload {
if(illustsArray.size() == 1) {
return illustsArray.get(0).getAsJsonObject();
} else {
return null;
throw new NoSuchElementException("No work found: " + illustId);
}
}

View File

@ -9,6 +9,13 @@ import java.util.Date;
import java.util.HashSet;
import java.util.Objects;
/**
* Pixiv搜索URL构造器
* <p>该构造器通过分析Pixiv搜索链接可用的参数而开发, 对搜索链接的构造有高度自定义能力.</p>
* @author LamGC
* @see PixivURL#PIXIV_SEARCH_CONTENT_URL
*/
@SuppressWarnings("ALL")
public class PixivSearchBuilder {
private final String content;

View File

@ -1,171 +0,0 @@
package net.lamgc.cgj.pixiv;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import org.apache.http.Header;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.CookieStore;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicHeader;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
public class PixivSession {
public static final String USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36";
/**
* 全登陆过程的关键,
* 保存会话用的cookieStore!
*/
private CookieStore cookieStore = new BasicCookieStore();
/**
* 可以直接使用的HttpClient对象
*/
private HttpClient httpClient;
/**
* 最后一次登录的错误信息
*/
private String errMsg;
public PixivSession(){
this(null);
}
public PixivSession(CookieStore cookieStore){
this(null, cookieStore);
}
/**
* 创建一个Pixiv登录会话
*/
public PixivSession(HttpHost proxy, CookieStore cookieStore) {
if(cookieStore != null){
this.cookieStore = cookieStore;
}
List<Header> defaultHeader = new ArrayList<>();
defaultHeader.add(new BasicHeader("accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"));
defaultHeader.add(new BasicHeader("user-agent", PixivSession.USER_AGENT));
defaultHeader.add(new BasicHeader("accept-encoding", "gzip, deflate, br"));
defaultHeader.add(new BasicHeader("accept-language", "zh-CN,zh;q=0.9"));
/*defaultHeader.add(new BasicHeader("sec-fetch-mode", "navigate"));
defaultHeader.add(new BasicHeader("sec-fetch-site", "same-origin"));
defaultHeader.add(new BasicHeader("upgrade-insecure-requests", "1"));*/
//创建一个Http访问器
httpClient = HttpClients.custom()
.setDefaultCookieStore(cookieStore)
.setDefaultHeaders(defaultHeader)
.setProxy(proxy)
.build();
}
/**
* 程序自行通过帐号密码登录Pixiv.
* @param PixivID Pixiv帐号
* @param Password Pixiv密码
* @return 登录成功返回true
* @throws IOException 当登录抛出异常时返回
* @deprecated Pixiv已经新增Google人机验证, 程序已无法自行登录Pixiv
*/
public boolean Login(String PixivID, String Password) throws IOException {
// 获取登录接口所需的PostKey
String post_key = getPostKey();
HttpPost postRequest = new HttpPost(PixivURL.PIXIV_LOGIN_URL); //https://accounts.pixiv.net/api/login?lang=zh
List<NameValuePair> params = new ArrayList<>();
params.add(new BasicNameValuePair("pixiv_id", PixivID));
params.add(new BasicNameValuePair("password", Password));
params.add(new BasicNameValuePair("post_key", post_key));
//Form编码表单,作为Post的数据
postRequest.setEntity(new UrlEncodedFormEntity(params, StandardCharsets.UTF_8));
//启动访问
HttpResponse response = httpClient.execute(postRequest);
//获取接口返回数据
String httpXML = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
System.out.println(httpXML);
JsonObject responseJson = (JsonObject) new JsonParser().parse(httpXML);
if(!responseJson.get("error").getAsBoolean() && !responseJson.get("body").getAsJsonObject().has("validation_errors")){
errMsg = null;
return true;
}else{
errMsg = responseJson.get("body").getAsJsonObject().get("validation_errors").toString();
//System.err.println("登录失败MSG: " + errMsg);
return false;
}
}
/**
* 登录前准备, 获取PostKey
* @return Post_Key
*/
private String getPostKey() throws IOException {
//创建请求,获取PostKey
HttpGet getRequest = new HttpGet(PixivURL.PIXIV_LOGIN_PAGE_URL);
//设置请求
//getRequest.setConfig(config);
getRequest.setHeader("User-Agent", USER_AGENT);
//启动访问
HttpResponse response = httpClient.execute(getRequest);
//获取网页内容
String pageAsXML = EntityUtils.toString(response.getEntity(),"utf-8");
//创建Http解析器
Document document = Jsoup.parse(pageAsXML);
//获取init-config内容
String init_config = document.getElementById("init-config").val();
//System.out.println(init_config);
//创建Json解析器解析init-config
JsonObject initConfigObj = (JsonObject) new JsonParser().parse(init_config);
//检查是否有postKey
if(!initConfigObj.has("pixivAccount.postKey")){
throw new RuntimeException("postKey获取失败!可能是Pixiv修改了登录过程!");
}
//获取postKey
return initConfigObj.get("pixivAccount.postKey").getAsString();
}
/**
* 获取CookieStore
* @return CookieStore
*/
public CookieStore getCookieStore(){
return cookieStore;
}
/**
* 获取可直接使用的HttpClient对象
* @return 已配置好的HttpClient对象
*/
public HttpClient getHttpClient(){
return this.httpClient;
}
public boolean hasError(){
return errMsg == null;
}
/**
* 获取错误信息
* @return 返回登录错误信息
* @deprecated {@link #Login(String, String)}已经废除, 故本接口废除
*/
public String getErrMsg(){
return errMsg;
}
}

View File

@ -1,16 +0,0 @@
package net.lamgc.cgj.pixiv;
public class PixivTag {
public static PixivTag create(String tagName) {
return null;
}
private PixivTag() {
}
}

View File

@ -9,6 +9,7 @@ import java.util.GregorianCalendar;
/**
* 目前已整理的一些Pixiv接口列表
*/
@SuppressWarnings("unused")
public class PixivURL {
@ -26,21 +27,30 @@ public class PixivURL {
/**
* P站搜索请求url
* @deprecated 该接口已被替换, 请使用{@link PixivSearchBuilder}构造搜索Url
* @see PixivSearchBuilder
*/
@Deprecated
private static final String PIXIV_SEARCH_URL = "https://www.pixiv.net/search.php";
/**
* P站搜索用户url
* 需要替换的参数:
* {nick} - 用户昵称、部分名称
* @deprecated 该接口已被替换, 请使用{@link PixivSearchBuilder}构造搜索Url
* @see PixivSearchBuilder
*/
@Deprecated
public static final String PIXIV_SEARCH_USER_URL = PIXIV_SEARCH_URL + "?s_mode=s_usr&nick={nick}";
/**
* P站搜索插画url
* 需要替换的参数:
* {word} - 插画相关文本
* @deprecated 该接口已被替换, 请使用{@link PixivSearchBuilder}构造搜索Url
* @see PixivSearchBuilder
*/
@Deprecated
public static final String PIXIV_SEARCH_TAG_URL = PIXIV_SEARCH_URL + "?s_mode=s_tag&word={word}";
/**
@ -52,10 +62,10 @@ public class PixivURL {
/**
* P站用户插图列表获取API
* <p>所需数据在 body属性内的 illusts(属性名,属性值不重要), manga(多图) pickup(精选)</p>
* 需要替换的文本:
* {userId} - 用户ID
*/
//TODO: 所需数据在 body属性内的 illusts(属性名,属性值不重要), manga(多图) pickup(精选)
//{"error":false,"message":"","body":{"illusts":{"74369837":null,"70990542":null,"70608653":null,"69755191":null,"69729450":null,"69729416":null,"69503608":null,"69288766":null,"69083882":null,"69051458":null,"68484200":null,"68216927":null,"68216866":null,"68192333":null,"67915106":null,"67914932":null,"67854803":null,"67854745":null,"67854670":null,"67787211":null,"67772199":null,"67770637":null,"67754861":null,"67754804":null,"67754726":null,"67740486":null,"67740480":null,"67740450":null,"67740434":null,"67726337":null,"67499196":null,"67499163":null,"67499145":null,"67499111":null,"67499085":null,"67499038":null,"67498987":null,"67473178":null,"66271465":null,"63682753":null,"63682697":null,"59385148":null,"59383265":null,"59383240":null,"59383227":null,"59383173":null},"manga":[],"novels":[],"mangaSeries":[],"novelSeries":[],"pickup":[],"bookmarkCount":{"public":{"illust":1,"novel":0},"private":{"illust":0,"novel":0}}}}
public static final String PIXIV_USER_ILLUST_LIST_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/all";
@ -163,6 +173,7 @@ public class PixivURL {
/**
* 查询用户收藏.<br/>
* 该URL返回HTML页面需要进行解析.<br/>
* <p>注意: 该接口需要登陆</p>
* 需要替换的文本:<br/>
* {pageIndex} - 页数, 超出了则结果为空<br/>
*/
@ -335,6 +346,7 @@ public class PixivURL {
* @param mode 要检查的RankingMode项
* @return 如果支持返回true
*/
@SuppressWarnings("BooleanMethodIsAlwaysInverted")
public boolean isSupportedMode(RankingMode mode) {
return Arrays.binarySearch(supportedMode, mode) >= 0;
}

View File

@ -26,12 +26,11 @@ import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
/**
* Pixiv动图构建器.
* 可便捷的接收并处理动图.
* Pixiv动图构建器
*/
public final class PixivUgoiraBuilder {
private final Logger log = LoggerFactory.getLogger(PixivUgoiraBuilder.class.getSimpleName() + "@" + Integer.toHexString(this.hashCode()));
private final Logger log = LoggerFactory.getLogger(this.toString());
private final HttpClient httpClient;
private final JsonObject ugoiraMeta;

View File

@ -27,7 +27,7 @@ import java.util.List;
*/
public class PixivAccessProxyServer {
private final Logger log = LoggerFactory.getLogger(this.getClass().getSimpleName());
private final Logger log = LoggerFactory.getLogger(PixivAccessProxyServer.class);
private final HttpProxyServer proxyServer;
@ -99,7 +99,7 @@ public class PixivAccessProxyServer {
}*/
log.info("Response Cookie: " + value);
BasicClientCookie cookie = parseRawCookie(value);
cookieStore.addCookie(null);
cookieStore.addCookie(cookie);
});
httpResponse.headers().remove(HttpHeaderNames.SET_COOKIE);
super.afterResponse(clientChannel, proxyChannel, httpResponse, pipeline);

View File

@ -1,31 +0,0 @@
package net.lamgc.cgj.util;
import org.apache.http.client.CookieStore;
import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.cookie.BasicClientCookie;
import java.util.Date;
public class CookieUtil {
/**
* 将{@link java.net.CookieStore}转换到{@link CookieStore}
* @param netCookieStore java.net.CookieStore
* @return org.apache.http.client.CookieStore
*/
public static CookieStore NAParse(java.net.CookieStore netCookieStore){
CookieStore apacheCookieStore = new BasicCookieStore();
netCookieStore.getCookies().forEach(netCookie -> {
BasicClientCookie aCookie = new BasicClientCookie(netCookie.getName(), netCookie.getValue());
aCookie.setComment(netCookie.getComment());
aCookie.setDomain(netCookie.getDomain());
aCookie.setExpiryDate(new Date(netCookie.getMaxAge()));
aCookie.setPath(netCookie.getPath());
aCookie.setSecure(netCookie.getSecure());
aCookie.setVersion(netCookie.getVersion());
apacheCookieStore.addCookie(aCookie);
});
return apacheCookieStore;
}
}

View File

@ -1,13 +1,12 @@
package net.lamgc.cgj.util;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.utils.base.runner.StringParameterParser;
public class PagesQualityParser implements StringParameterParser<PixivDownload.PageQuality> {
@Override
public PixivDownload.PageQuality parse(String strValue) throws Exception {
public PixivDownload.PageQuality parse(String strValue) {
return PixivDownload.PageQuality.valueOf(strValue.toUpperCase());
}

View File

@ -0,0 +1,51 @@
package net.lamgc.cgj.util;
import net.lamgc.utils.base.ArgumentsProperties;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public final class PropertiesUtils {
private final static Logger log = LoggerFactory.getLogger(PropertiesUtils.class);
private PropertiesUtils() {}
/**
* 从ArgumentsProperties获取设置项到System Properties
* @param prop ArgumentsProperties对象
* @param key 设置项key
* @param defaultValue 默认值
* @return 如果成功从ArgumentsProperties获得设置项, 返回true, 如未找到(使用了defaultValue或null), 返回false;
*/
public static boolean getSettingToSysProp(ArgumentsProperties prop, String key, String defaultValue) {
if(prop.containsKey(key)) {
log.info("{}: {}", key, prop.getValue(key));
System.setProperty("cgj." + key, prop.getValue(key));
return true;
} else {
if(defaultValue != null) {
System.setProperty("cgj." + key, defaultValue);
}
return false;
}
}
/**
* 将环境变量的值读取并存入System Properties.
* @param envKey 待获取的环境变量Key
* @param sysPropKey 要设置的System Properties Key值
* @param defaultValue 默认值, 可选
* @return 如果设置成功, 返回true, 否则返回false
*/
public static boolean getEnvSettingToSysProp(String envKey, String sysPropKey, String defaultValue) {
String env = System.getenv(envKey);
if(env != null) {
System.setProperty("cgj." + sysPropKey, env);
return true;
} else if(defaultValue != null) {
System.setProperty("cgj." + sysPropKey, defaultValue);
}
return false;
}
}

View File

@ -0,0 +1,148 @@
package net.lamgc.cgj.util;
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicLong;
/**
* 带有时间限制的线程池.
* 当线程超出了限制时间时, 将会对该线程发出中断.
*/
public class TimeLimitThreadPoolExecutor extends ThreadPoolExecutor {
/**
* 执行时间限制, 单位毫秒.
* 默认30s.
*/
private final AtomicLong executeTimeLimit = new AtomicLong();
/**
* 检查间隔时间.
* 默认100ms.
*/
private final AtomicLong timeoutCheckInterval = new AtomicLong(100);
private final Map<Thread, AtomicLong> workerThreadMap = new Hashtable<>();
private final Thread timeoutCheckThread = createTimeoutCheckThread();
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
setInitialTime(executeLimitTime);
timeoutCheckThread.start();
}
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory);
setInitialTime(executeLimitTime);
timeoutCheckThread.start();
}
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, RejectedExecutionHandler handler) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, handler);
setInitialTime(executeLimitTime);
timeoutCheckThread.start();
}
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory, RejectedExecutionHandler handler) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler);
setInitialTime(executeLimitTime);
timeoutCheckThread.start();
}
private void setInitialTime(long executeLimitTime) {
if(executeLimitTime > 0) {
executeTimeLimit.set(executeLimitTime);
}
}
/**
* 设置执行时间.
* <p>注意: 该修改仅在线程池完全停止后才有效</p>
* @see #isTerminated()
* @param time 新的限制时间(ms)
*/
public void setExecuteTimeLimit(long time) {
if(time <= 0) {
throw new IllegalArgumentException("Time is not allowed to be set to 0 or less");
}
if(this.isTerminated()) {
executeTimeLimit.set(time);
}
}
/**
* 设置超时检查间隔.
* <p>该方法仅会在当前检查后生效.</p>
* @param time 新的检查间隔(ms)
*/
public void setTimeoutCheckInterval(long time) {
if(time <= 0) {
throw new IllegalArgumentException("Time is not allowed to be set to 0 or less");
}
timeoutCheckInterval.set(time);
}
/**
* 获取当前设置的执行时间限制.
* @return 执行时间限制(ms).
*/
public long getExecuteTimeLimit() {
return executeTimeLimit.get();
}
/**
* 获取当前设定的超时检查间隔
* @return 间隔时间(ms).
*/
public long getTimeoutCheckInterval() {
return timeoutCheckInterval.get();
}
private Thread createTimeoutCheckThread() {
Thread checkThread = new Thread(() -> {
if(executeTimeLimit.get() <= 0) {
return;
}
while (true) {
try {
long interval = this.timeoutCheckInterval.get();
Thread.sleep(interval);
// 检查是否存在超时的任务
workerThreadMap.forEach((thread, time) -> {
long currentTime = time.getAndAdd(interval);
if(currentTime > executeTimeLimit.get()) {
if(!thread.isInterrupted()) {
thread.interrupt();
}
}
});
} catch(InterruptedException ignored) {
break;
}
}
});
checkThread.setName("ThreadPool-" + Integer.toHexString(this.hashCode()) +"-TimeoutCheck");
return checkThread;
}
@Override
protected void beforeExecute(Thread t, Runnable r) {
workerThreadMap.put(t, new AtomicLong());
super.beforeExecute(t, r);
}
@Override
protected void afterExecute(Runnable r, Throwable t) {
workerThreadMap.remove(Thread.currentThread());
super.afterExecute(r, t);
}
@Override
protected void terminated() {
this.timeoutCheckThread.interrupt();
super.terminated();
}
}

View File

@ -4,6 +4,7 @@
<property name="logStorePath">./logs</property>
<property name="charset">UTF-8</property>
<property name="pattern">[%-d{HH:mm:ss.SSS} %5level][%logger.%method():%-3L][%thread]: %msg%n</property>
<property name="logsDir">${sys:cgj.logsPath:-logs}</property>
</properties>
<Appenders>
@ -20,7 +21,7 @@
</Filters>
</Console>
<RollingFile name="rollingFile" fileName="logs/latest.log" filePattern="logs/running.%-d{yyyy-MM-dd_HH-mm-ss}.log.gz">
<RollingFile name="rollingFile" fileName="${logsDir}/latest.log" filePattern="${logsDir}/running.%-d{yyyy-MM-dd_HH-mm-ss}.log.gz">
<PatternLayout pattern="${pattern}" charset="${charset}"/>
<Policies>
<OnStartupTriggeringPolicy />

View File

@ -4,6 +4,7 @@
<property name="logStorePath">./logs</property>
<property name="charset">UTF-8</property>
<property name="pattern">[%-d{HH:mm:ss.SSS} %5level][%logger.%method():%-3L][%thread]: %msg%n</property>
<property name="logsDir">${sys:cgj.logsPath:-logs}</property>
</properties>
<Appenders>
@ -20,7 +21,7 @@
</Filters>
</Console>
<RollingFile name="rollingFile" fileName="logs/latest.log" filePattern="logs/running.%-d{yyyy-MM-dd_HH-mm-ss}.log.gz">
<RollingFile name="rollingFile" fileName="${logsDir}/latest.log" filePattern="${logsDir}/running.%-d{yyyy-MM-dd_HH-mm-ss}.log.gz">
<PatternLayout pattern="${pattern}" charset="${charset}"/>
<Policies>
<OnStartupTriggeringPolicy />

View File

@ -31,9 +31,9 @@ public class PixivDownloadTest {
private static CookieStore cookieStore;
private final static Logger log = LoggerFactory.getLogger("PixivDownloadTest");
private final static Logger log = LoggerFactory.getLogger(PixivDownloadTest.class);
private static HttpHost proxy = new HttpHost("127.0.0.1", 1001);
private static final HttpHost proxy = new HttpHost("127.0.0.1", 1001);
@BeforeClass
public static void before() throws IOException, ClassNotFoundException {
@ -180,22 +180,8 @@ public class PixivDownloadTest {
log.info("正在调用方法...");
try {
pixivDownload.getRankingAsInputStream(null, null, queryDate, 5, 50, PixivDownload.PageQuality.ORIGINAL, (rank, link, rankInfo, inputStream) -> {
/*try {
ZipEntry entry = new ZipEntry("Rank" + rank + "-" + link.substring(link.lastIndexOf("/") + 1));
entry.setComment(rankInfo.toString());
log.info("正在写入: " + entry.getName());
zos.putNextEntry(entry);
IOUtils.copy(inputStream, zos);
zos.flush();
log.info("已成功写入 {}", entry.getName());
inputStream.close();
} catch (IOException e) {
log.error("写入文件项时发生异常", e);
}*/
log.info("空操作");
});
pixivDownload.getRankingAsInputStream(null, null, queryDate, 5, 50,
PixivDownload.PageQuality.ORIGINAL, (rank, link, rankInfo, inputStream) -> log.info("空操作"));
} finally {
zos.finish();
zos.flush();

View File

@ -0,0 +1,30 @@
package net.lamgc.cgj.util;
import org.junit.Assert;
import org.junit.Test;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.TimeUnit;
public class TimeLimitThreadPoolExecutorTest {
@Test
public void timeoutTest() throws InterruptedException {
TimeLimitThreadPoolExecutor executor = new TimeLimitThreadPoolExecutor(1000, 1, 1, 30, TimeUnit.SECONDS, new ArrayBlockingQueue<>(50));
System.out.println(executor.isTerminated());
System.out.println(executor.isShutdown());
executor.setTimeoutCheckInterval(150);
System.out.println("当前设定: ETL: " + executor.getExecuteTimeLimit() + "ms, TCI: " + executor.getTimeoutCheckInterval() + "ms");
executor.execute(() -> {
try {
Thread.sleep(5 * 1000);
} catch (InterruptedException e) {
System.out.println("线程 " + Thread.currentThread().getName() + " 被中断");
}
});
executor.shutdown();
Assert.assertTrue(executor.awaitTermination(5 * 1000, TimeUnit.MILLISECONDS));
}
}