Compare commits

...

109 Commits

Author SHA1 Message Date
75aa78a3d7 [Version] 更新版本(2.5.2-20200606.1-SNAPSHOT -> 2.5.2-20200609.1-SNAPSHOT); 2020-06-09 10:12:02 +08:00
18a8ad95a1 [Change] PixivUgoiraBuilderTest 调整测试细节; 2020-06-09 10:07:54 +08:00
438d0a95d3 [Add] pom.xml 增加JLine库;
[Update] ConsoleMain 增强Cli使用体验;
[Add] BotEventHandler 增加同步执行命令的方法'executeMessageEvent(MessageEvent, boolean)';
[Change] Main 适配ConsoleMain的更改;
2020-06-09 09:50:16 +08:00
6789b5b7c5 [Change] CacheStoreCentral 调整'getImageById'方法中对'pageIndex'的参数值检查时机;
[Change] MiraiMessageSender 增加警告忽略注释;
2020-06-08 19:13:51 +08:00
ad289f952f [Fix] log4j2.xml 修复SYSTEM_OUT级别限定错误的问题; 2020-06-08 18:47:53 +08:00
3ae0e4cd8d [Change] HotDataCacheStore 调整日志输出级别(DEBUG -> TRACE); 2020-06-08 16:08:13 +08:00
5550c7aef1 [Clear] CacheStoreCentral 删除无用的运行器代码; 2020-06-08 15:57:38 +08:00
d4d3432c76 [Add] PixivUgoiraBuilder 增加方法'buildUgoira(OutputStream, boolean)'以提供输出流给Builder输出已构建的动图数据;
[Add] PixivUgoiraBuilder 增加方法'getUgoiraMeta():JsonObject'方法, 可通过该方法获取动图元数据;
2020-06-08 15:30:22 +08:00
d1aeda012e [Fix] BotCommandProcess 修复因'getImageById'迁移导致'image'命令不可用的问题;
[Clear] CacheStoreCentral 清除无用代码;
2020-06-08 09:37:06 +08:00
683a38bc17 [Add] BotGlobal BotGlobal在初始化时将检查Redis连通性;
[Change] RandomRankingArtworksSender 支持外部设置groupId, 以使用群组配置;
[Change] BotCommandProcess, BotAdminCommandProcess 适配RandomRankingArtworksSender更改;
2020-06-08 09:24:17 +08:00
188309509b [Change] BotCommandProcess 简化Search参数'ContentOption'的名称('contentOption' -> 'option'); 2020-06-06 19:27:55 +08:00
3915712337 [Delete] search.txt 删除旧文档; 2020-06-06 17:57:24 +08:00
1e88ba70dd [Version] 更新版本(2.5.2-20200604.3-SNAPSHOT -> 2.5.2-20200606.1-SNAPSHOT); 2020-06-06 17:54:09 +08:00
e6b2544998 [Update] 更新接口文档; 2020-06-06 17:53:37 +08:00
a2f6f1d140 [Clear] BotCommandProcess 整理代码; 2020-06-06 17:07:40 +08:00
f54ed35a09 [Change] SettingProperties 增加群号检查, 防止出现非法群号; 2020-06-06 15:42:35 +08:00
a426f80ec5 [Fix] BotCommandProcess 修复Search命令对作品限制的管理不受'image.allowR18'选项控制的问题; 2020-06-06 11:50:29 +08:00
c1a21d1065 [Change] ConsoleMain, ConsoleMessageEvent 支持启动时设置会话群组Id和QQId; 2020-06-06 10:52:12 +08:00
e570ddbb53 [Change] PreLoadDataComparator 增加对JsonElement是否为JsonObject的检查; 2020-06-06 10:51:24 +08:00
223d78dbd6 [Change #5] 优化'getImageById'从文件读入缓存时, 检查图片完整性的速度; 2020-06-05 17:03:41 +08:00
ef5651be47 [Fix] CacheStore 修正Javadoc中的错误描述; 2020-06-05 16:29:16 +08:00
9a8aac1960 [Fix] BotGlobal 修复BotGlobal初始化失败的问题;
[Fix] BotCommandProcess 修复Search命令未找到相关作品时提示语无法触发的问题;
[Change] BotCommandProcess 调整'isNoSafe(int, Properties, boolean)'方法中对于作品限制的判断方式;
2020-06-05 16:23:13 +08:00
4bbed5fd55 [Change] BotCommandProcess, CacheStoreCentral 将ImageFile缓存管理转移到CacheStoreCentral;
[Change] RandomRankingArtworksSender 适配修改;
[Change] BotCommandProcess, BotGlobal 将'imageStoreDir'由BotCommandProcess转移到BotGlobal;
2020-06-05 10:07:13 +08:00
bcc21149b9 [Change #10] 将缓存存取部分从BotCommandProcess分离;
[Change] 整理代码;
2020-06-05 09:53:30 +08:00
e93c322c02 [Change] BotGlobal, BotCommandProcess 调整PixivDownload的初始化过程;
[Change] BotCommandProcess 将'Search'所属缓存部分抽出到单独的方法('getSearchBody');
2020-06-04 20:14:57 +08:00
feb51b8534 [Change] BotGlobal, BotCommandProcess 将'Gson'和'PixivDownload'对象纳入BotGlobal; 2020-06-04 19:47:20 +08:00
273dbd45b0 [Version] 更新版本(2.5.2-20200604.2-SNAPSHOT -> 2.5.2-20200604.3-SNAPSHOT); 2020-06-04 16:16:53 +08:00
b39a82b936 [Fix] BotGlobal 修复BotGlobal初始化失败的问题; 2020-06-04 16:15:27 +08:00
4589624b9b [Version] 更新版本(2.5.2-20200604.1-SNAPSHOT -> 2.5.2-20200604.2-SNAPSHOT); 2020-06-04 15:58:56 +08:00
033fd0188b [Change] 调整Logger创建方式(getLogger(String) -> getLogger(Class)); 2020-06-04 15:58:10 +08:00
4318869846 [Change] 将'Proxy', 'CookieStore'转移到BotGlobal类; 2020-06-04 15:54:52 +08:00
749b89e668 [Version] 更新版本(2.5.2-20200522.1-SNAPSHOT -> 2.5.2-20200604.1-SNAPSHOT); 2020-06-04 11:26:26 +08:00
9ce18469e6 [Add] 增加控制台框架以进行本地测试, 或非聊天机器人使用; 2020-06-04 11:25:08 +08:00
e803d2161b [Add] PropertiesUtils 增加一个工具类; 2020-06-04 11:24:40 +08:00
3fbf80f233 Merge remote-tracking branch 'origin/master' 2020-06-04 11:20:22 +08:00
8b8ef7e744 [Change] Issue #12 调整框架启动机器人应用的方式, 增加一个用于机器人应用内部共享数据的类;
[Fix] BotEventHandler, ImageCacheStore 增加ShutdownHook用于关闭线程池, 解决线程池阻塞关闭过程的问题;
[Change] BotEventHandler 调整'match(String)'方法;
[Change] BotAdminCommandProcess 调整'savePushList()'方法对文件创建失败的行为;
2020-06-04 11:19:18 +08:00
7fbdf28ec8 [Update] Bug_Report.md 增加默认标签; 2020-06-04 10:50:37 +08:00
b29482927c [Add] Bug_Report.md 添加Bug反馈模板; 2020-06-04 10:47:28 +08:00
69da2b02ac [Fix #11] 修复在图片缓存失效的情况下, 'getImageToBotCode'依然会尝试从缓存获取图片File对象导致NPE;
[Change] BotCommandProcess 调整字符串拼接形式, 统一错误提示语的格式;
2020-06-04 09:29:53 +08:00
eb2de09859 [Change] RandomIntervalSendTimer 将随机间隔发送器的Timer设为daemon thread; 2020-06-03 19:37:45 +08:00
ef70ac77cb [Update] HotDataCacheStore, Cleanable 更新Javadoc; 2020-06-03 17:03:54 +08:00
b4e9fdab7d [Change] TimeLimitThreadPoolExecutor 整理代码, 清除无用参数;
[Update] Pixiv排行榜接口.md 更新文档内容, 补充返回数据信息;
2020-06-03 17:02:46 +08:00
5c5827123d [Update] net.lamgc:java-utils 更新依赖项版本(1.2.0_20200514.1-SNAPSHOT -> 1.2.0_20200517.1-SNAPSHOT), 以修复作品无效检测不生效的问题; 2020-06-03 16:51:26 +08:00
2bfb78304c [Change] HotDataCacheStore, RandomIntervalSendTimer 支持浮动时间参数为0; 2020-06-03 14:50:45 +08:00
637ea46b9a [Fix] MiraiMessageEvent 修复群消息事件错误设置私聊Sender的问题; 2020-06-01 21:02:46 +08:00
f5140a7a1e [Change] SpringCQMessageSenderFactory 尝试通过ThreadLocal获取当前处理的CoolQ对象, 目前EventExecutor暂不支持, 仅预留;
[Change] CQPluginMain 调整'processMessage'方法修饰符, 补充Javadoc;
2020-06-01 20:15:59 +08:00
0fafddc715 [Change] AdultContentDetector, PixivURL 整理代码, 补充Javadoc; 2020-06-01 16:12:33 +08:00
4c03a0f7d9 [Change] HotDataCacheStore 调整'supportedList'的条件, HotDataCacheStore将支持使用ListCacheStore;
[Change] AutoSender 将AutoSender的messageSender增加final属性;
2020-06-01 16:01:17 +08:00
ee02072b2d [Deprecated] PixivURL 将旧版Search接口设为弃用; 2020-06-01 15:58:54 +08:00
3f6c87da3c [Add] MiraiMain 增加对临时消息的支持;
[Change] MiraiMessageEvent 调整消息对象转换的方式, 通过静态方法转换Mirai的MessageEvent;
2020-06-01 15:55:31 +08:00
d45cd8aab5 [Update] 更新文档, 增加排行榜接口文档, 待补充返回数据; 2020-05-28 16:20:16 +08:00
598d6ef205 [Fix] README.md 修复格式错误;
[Update] README.md 补充参数细节;
2020-05-27 19:30:04 +08:00
2190ef1b59 [Change] README.md 调整Readme内容以减少对用户选择带来的影响; 2020-05-27 19:27:29 +08:00
fcc522c057 Merge branch 'master' of github.com:LamGC/ContentGrabbingJi 2020-05-26 14:45:01 +08:00
5a81604d94 [Change] BotCommandProcess 调整'色图'命令处理方法的命名; 2020-05-26 14:31:12 +08:00
376306e404 [Update] README.md 更新文档说明, 提供详细命令说明和注意事项; 2020-05-26 10:37:40 +08:00
0fe69253ed [Change] .gitignore 将Idea项目文件由指定文件名改为匹配后缀名; 2020-05-26 10:28:46 +08:00
fe8bf16d3a [Clean] RandomRankingArtworksSender 清理无用导入;
[Change] AutoCleanTimer 统一LoggerName名称;
2020-05-25 20:23:14 +08:00
99c66babec [Add] BotEventHandler 增加对'Sender.sendMessage(String)'返回值的处理; 2020-05-25 09:45:09 +08:00
cb8b01fd74 Merge branch 'master' of github.com:LamGC/ContentGrabbingJi 2020-05-25 09:25:49 +08:00
1f48b3ffdc [Change] MessageSender, MessageEvent 允许'senderMessage(String)'抛出异常;
[Change] RandomRankingArtworksSender 调整异常捕获日志输出;
[Change] MiraiMessageEvent, MiraiMessageSender, SpringCQMessageEvent 适配更改;
2020-05-25 09:21:46 +08:00
68feed8d3c Merge remote-tracking branch 'origin/master' 2020-05-22 23:09:39 +08:00
d89711b0b9 [Version] 更新版本(2.5.2-20200520.1-SNAPSHOT -> 2.5.2-20200522.1-SNAPSHOT); 2020-05-22 20:42:29 +08:00
a3376e96ee [Add] BotCommandProcess, BotEventHandler, PixivDownload 对作品Id不存在的情况做出反应(而不是作为内部异常反馈); 2020-05-22 20:39:49 +08:00
33d18cef6b Merge remote-tracking branch 'origin/master' 2020-05-22 20:34:32 +08:00
7aa00ff98b [Fix] BotCommandProcess 修复ImageStore不允许使用软链接的问题; 2020-05-22 20:34:15 +08:00
e956e36584 [Fix] RankingUpdateTimer 修复首次执行时间计算错误的问题; 2020-05-22 20:23:46 +08:00
8e27221457 [Change] BufferMessageEvent 调整成员变量修饰符, 补充新的构造方法'<init>(int, int, String)'; 2020-05-22 08:49:48 +08:00
a5fca68ef5 [Change] MessageEventExcutionDebugger 在配置文件无法转换的情况下不再忽略'NumberFormatException'异常; 2020-05-20 15:31:43 +08:00
8edb728fe9 [Update] README.md 补充Readme内容 2020-05-20 14:34:55 +08:00
cc05af8a24 Create README.md 2020-05-20 11:00:25 +08:00
60fa641962 [Version] 更新版本(2.5.2-20200517.1-SNAPSHOT -> 2.5.2-20200520.1-SNAPSHOT); 2020-05-20 10:36:30 +08:00
3ab373cc70 [Update] .gitignore 补充VSCode忽略项; 2020-05-20 10:34:55 +08:00
461cd246d8 [Add] BotCommandProcess 新增'色图'功能以随机从排行榜中获取一张图片;
[Add] BufferMessageEvent 增加用于获取Sender发送内容的Event实现;
[Change] BotEventHandler Handler现在允许命令处理方法不返回消息(返回'null');
2020-05-20 09:55:50 +08:00
f040f130d7 [Update] Main, AutoArtworksSender, ImageCacheStore, PixivAccessProxyServer, PagesQualityParser 整理代码问题, 删除无用类; 2020-05-20 09:05:50 +08:00
1f402fbbac [Change] LocalHashCacheStore 实现'Cleanable'接口, 增加'<init>(int, int, boolean)'构造函数以允许自动清理
[Update] LocalHashCacheStore 补充Javadoc内容;
[Update] .gitignore 补充并整理忽略项;
2020-05-20 08:56:27 +08:00
f7f3c3beaf [Fix] BotCommandProcess 修复help命令中帮助信息的错误; 2020-05-17 19:38:17 +08:00
04c1753c22 [Version] 更新版本(2.5.1 -> 2.5.2-20200517.1-SNAPSHOT); 2020-05-17 17:45:06 +08:00
d993e9d719 [Add] MiraiMessageSender 增加对默认表情的支持; 2020-05-12 15:25:29 +08:00
60e91987d1 Merge branch 'mirai' 2020-05-12 15:14:58 +08:00
65392fc2fe [Fix] net.mamoe:mirai-core, net.mamoe:mirai-core-qqandroid 更新Mirai-Core版本号以修复VerifyError问题(1.0-RC2 -> 1.0-RC2-1);
[Fix] MiraiMessageSender 调整日志输出, 以修复日志打印处理后消息顺序混乱的问题;
2020-05-12 15:14:28 +08:00
d38934a0f4 [Change] RankingUpdateTimer 调整时间补齐的阀值(12:00 -> 11:30);
[Add] documents/interfaces/* 整理部分Pixiv接口的文档;
2020-05-12 15:03:08 +08:00
7943357d96 [Fix] MiraiMain 修复未登录时异常退出, 抛出NPE的问题; 2020-05-11 12:16:35 +08:00
a170ff040b [Update] net.mamoe:mirai-core, net.mamoe:mirai-core-qqandroid 更新依赖项版本(0.39.4 -> 1.0-RC2);
[Update] MiraiMain, MiraiMessageEvent 适配mirai新版本;
[Change] MiraiMessageSenderFactory 增加参数检查并优化过程;
[Change] MiraiMain 显性指定Mirai所用协议为'BotConfiguration.MiraiProtocol.ANDROID_PAD';
2020-05-11 11:36:37 +08:00
cb2ebfdb73 [Change] MessageEvent 对消息内容删除首尾空格以提高用户体验性; 2020-05-09 23:13:53 +08:00
66b22c543a [Fix] SettingProperties 修复了配置文件无法创建, 导致无法保存配置的问题;
[Fix] Main 修复了潜在的文件创建失败的问题;
2020-05-09 22:54:32 +08:00
6bace4b048 [Version] 更新版本(2.5.0 -> 2.5.1); 2020-05-09 22:35:46 +08:00
597aac4e95 [Fix] BotCommandProcess 修复部分指令无法使用群组配置的问题; 2020-05-09 22:35:21 +08:00
181d60285b [Version] 更新版本(2.5.0-20200509.1-SNAPSHOT -> 2.5.0); 2020-05-09 22:20:07 +08:00
c06b8717c3 [Add] CacheStoreUtils 增加用于缓存库的工具类;
[Update] BotCommandProcess 补充注释, 优化"info"命令输出内容的格式;
2020-05-09 22:19:29 +08:00
b4a28b6735 [Version] 更新版本(2.5.0-20200506.1-SNAPSHOT -> 2.5.0-20200509.1-SNAPSHOT); 2020-05-09 18:16:54 +08:00
40057c3683 [Fix] Issue #1 修复了ImageCache对重复缓存请求的策略(忽略并返回)导致的后续处理发生NPE的问题;
[Update] ImageCache 更新了ImageCache, 将该部分单独分离到 ImageCacheStore;
[Update] BotCommandProcess 适配新的ImageCache, 调整内容检查的时机以减少不必要的API访问;
2020-05-09 18:16:16 +08:00
4beb4d78fb [Fix] BotCommandProcess 修复Search命令格式不正确的问题; 2020-05-09 12:52:15 +08:00
d9edaa681f [Add] BotAdminCommandProcess 为 addPushGroup 增加参数检查; 2020-05-09 10:10:48 +08:00
7b52abde60 [Change] BotCommandProcess 增加Search命令的反馈内容(可能必须要QQ支持长文本才能使用); 2020-05-08 23:31:24 +08:00
2cb4fe1dbc [Fix] SettingProperties 修复无法设置全局配置项的问题; 2020-05-08 12:23:43 +08:00
5830327dad [Add] BotEventHandler 增加禁言记录, 防止应用因过多的在禁言状态反馈导致被封号;
[Update] MiraiMain 适配禁言记录功能;
2020-05-07 19:46:54 +08:00
cbd10ff281 [Fix] Issue #4 修复因Mirai API变动导致应用无法从消息获取ImageId的问题; 2020-05-07 18:24:09 +08:00
84b8006f89 [Change] RankingUpdateTimer 将定时更新时间调整为"+8 11:30", 并增加日期指定; 2020-05-07 15:21:05 +08:00
d32d891ad7 [Update] BotCommandProcess 补充Javadoc内容;
[Add] BotCommandProcess 在Ranking命令增加force参数以允许强制查询预计不在排行榜范围的日期;
2020-05-07 11:29:31 +08:00
aabd35ce71 [Add] AutoCleanTimer, Cleanable 增加缓存库自动清理;
[Update] HotDataCacheStore 增加对自动清理的支持;
[Change] RedisPoolCacheStore 将Redis缓存库抽象类设为default(原public);
[Change] MiraiMessageSender, BotCommandProcess 适配HotDataCacheStore的修改;
[Change] BotCommandProcess 调整ranking命令的输出格式;
2020-05-07 10:26:11 +08:00
a36eb713d7 [Version] 更新版本(2.5.0-20200505.1-SNAPSHOT -> 2.5.0-20200506.1-SNAPSHOT); 2020-05-06 11:09:18 +08:00
f452c1c128 Merge branch 'add-MultiProperties' 2020-05-06 11:08:16 +08:00
cb42aadb15 [Fix] SettingProperties 修复在初次设置群组配置时未创建该群组所属Properties导致NPE的问题;
[Update] BotAdminCommandProcess 更新日志内容及命令返回信息;
[Change] BotCommandProcess 调整配置项key格式;
2020-05-06 10:55:32 +08:00
b969bb29b2 Merge branch 'master' into add-MultiProperties 2020-05-06 10:18:00 +08:00
0d74007a98 [Change] search.txt 整理接口相关文档的存储目录; 2020-05-06 09:42:34 +08:00
edade24883 [Change] Issue #2 调整Logger Name以统一Logger Name格式; 2020-05-06 09:34:28 +08:00
34f57404ca [Update] 重构Setting Properties, 使其支持分群配置; 2020-05-06 00:57:48 +08:00
99b6e14ff7 [Change] RandomIntervalSendTimer 调整日志输出内容; 2020-05-05 17:09:45 +08:00
62 changed files with 3363 additions and 929 deletions

26
.github/ISSUE_TEMPLATE/Bug_Report.md vendored Normal file
View File

@ -0,0 +1,26 @@
---
name: Bug Report
about: Use this template to feedback bugs.
title: ''
labels: bug
assignees: ''
---
## Environmental information ##
OS(e.g: Windows 10 1909):
Java(e.g: Oracle Jdk 8.242):
Issue version(versionTag or commitId):
## Problem description ##
// Describe the problem in as much detail as possible here
## Expected behavior ##
// What will this function do under normal circumstances?
## Actual behavior ##
// But what does this feature actually look like?
## Recurrence steps ##
// What can we do to recreate this situation?
1.

14
.gitignore vendored
View File

@ -1,6 +1,16 @@
# Ignore test date folder
/pluginData/
/logs/
/.idea/
/CGJ_2.iml
/cookies.store
/target/
# Ignore Idea files
/.idea/
*.iml
# Ignore Visual Studio Code files
.classpath
.factorypath
.project
/.settings/
/.vscode/

39
README.md Normal file
View File

@ -0,0 +1,39 @@
# ContentGrabbingJi
Pixiv爬虫一只同时也是一个机器人/插件!
## 支持的机器人平台 ##
- [Mirai](https://github.com/mamoe/mirai)
- [CoolQ](https://cqp.cc)(基于[`SpringCQ`](https://github.com/lz1998/spring-cq), 不支持多账号使用, 需要使用[`CQHttp`](https://cqhttp.cc/)插件)
## Usage ##
> 注意: 运行色图姬前, 你需要准备一个Pixiv账号的会话Cookie存储文件, 否则色图姬将无法运行.
> 详见[PixivLoginProxyServer](https://github.com/LamGC/PixivLoginProxyServer)项目的[Readme](https://github.com/LamGC/PixivLoginProxyServer/blob/master/README.md).
### Arguments ###
> ENV参数名为环境变量名, 用于给Docker容器提供设置方式.
- 通用参数
- `-proxy` / `ENV: CGJ_PROXY`: (**可选**) 设置代理
- 格式: `协议://地址:端口`
- 示例: `socks5://127.0.0.1:1080`
- 机器人参数
- `-botDataDir` / `ENV: CGJ_BOT_DATA_DIR`: (**可选**) 设置`botMode`运行模式下机器人数据存储目录
- 格式: `路径`
- 示例: `./data`
- 默认: `./`
- `-redisAddress` / `ENV: CGJ_REDIS_URI`: (**必填, 计划支持可选**) Redis服务器地址
- 格式: `地址:端口`
- 示例: `127.0.0.1:6379`
> 例如以BotMode启动应用: `java -jar CGJ.jar botMode -proxy "socks5://127.0.0.1:1080 -redisAddress 127.0.0.1:6379`
### Commands ###
- `pluginMode`: CoolQ插件模式(依赖[酷Q机器人](https://cqp.cc/), 支持与CoolQ其他插件共存, 性能耗损大)
- `botMode`: Mirai独立模式(机器人独立运行, 不支持与其他插件共存, 性能耗损小)
- `collectionDownload`: 收藏下载, 以原图画质下载Cookie所属账号的所有收藏作品
- `getRecommends`: 将访问主页获得的推荐作品全部以原图画质下载
- `getRankingIllust`: 以原图画质下载指定排行榜类型的全部作品
- `search`: 搜索指定内容并获取相关作品信息(不下载)
> 注意: 除去机器人模式外, 其他功能后续可能会出现改动.

View File

@ -0,0 +1,108 @@
## Pixiv作品信息批量获取接口 ##
### 说明 ###
接口可一次获取多个作品的基础信息
### 接口地址 ###
```
GET https://www.pixiv.net/ajax/illust/recommend/illusts
```
- 需要登录: `否`
- 是否为Pixiv接口标准返回格式: `是`
### 参数 ###
- `illust_ids[]`: 作品Id, 可重复添加该参数
### 请求示例 ###
```
GET https://www.pixiv.net/ajax/illust/recommend/illusts?illust_ids[]=82030844&illust_ids[]=82029098&illust_ids[]=82028913
```
### 返回数据 ###
#### 数据示例 ####
```json
{
"error":false,
"message":"",
"body":{
"illusts":[
{
"illustId":"82030844",
"illustTitle":"3rd anniversary",
"id":"82030844",
"title":"3rd anniversary",
"illustType":0,
"xRestrict":0,
"restrict":0,
"sl":2,
"url":"https:\/\/i.pximg.net\/c\/360x360_70\/custom-thumb\/img\/2020\/06\/02\/11\/24\/49\/82030844_p0_custom1200.jpg",
"description":"",
"tags":[
"\u30a2\u30ba\u30fc\u30eb\u30ec\u30fc\u30f3",
"\u6bd4\u53e1(\u30a2\u30ba\u30fc\u30eb\u30ec\u30fc\u30f3)",
"\u6c34\u7740",
"\u8db3\u88cf",
"\u88f8\u8db3",
"\u30a2\u30ba\u30fc\u30eb\u30ec\u30fc\u30f35000users\u5165\u308a",
"\u30a2\u30ba\u30fc\u30eb\u30ec\u30fc\u30f310000users\u5165\u308a",
"\u30dd\u30cb\u30fc\u30c6\u30fc\u30eb",
"\u30db\u30eb\u30bf\u30fc\u30cd\u30c3\u30af"],
"userId":"6662895",
"userName":"ATDAN-",
"width":1500,
"height":844,
"pageCount":1,
"isBookmarkable":true,
"bookmarkData":null,
"alt":"#\u30a2\u30ba\u30fc\u30eb\u30ec\u30fc\u30f3 3rd anniversary - ATDAN-\u7684\u63d2\u753b",
"isAdContainer":false,
"titleCaptionTranslation":{
"workTitle":null,
"workCaption":null
},
"createDate":"2020-06-02T01:29:40+09:00",
"updateDate":"2020-06-02T11:24:49+09:00",
"profileImageUrl":"https:\/\/i.pximg.net\/user-profile\/img\/2016\/01\/11\/21\/46\/50\/10371466_80f6ad67eab3b8abd44a2fb74ddd1ba1_50.jpg",
"type":"illust"
}, // ...
]
}
}
```
#### 参数详解 ####
- `illusts`: (`Object[]`) 存储查询作品信息的数组
- `illustId`: (`string` -> `number`) 作品Id
- `illustTitle`: (`string`) 作品标题
- `id`: (`string` -> `number`) 与`illustId`一致, 猜测是以兼容旧版本为目录而存在
- `title`: (`string`) 与`illustTitle`一致, 猜测是以兼容旧版本为目录而存在
- `illustType`: (`number`) 作品类型
- `0`: 插画作品
- `1`: 漫画作品
- `2`: 动图作品
- `xRestrict`: (`number`) 作品是否为限制级, 基本准确, 少部分不一定(看Pixiv审核怎么理解了)
- `0`: 非限制级内容(即非R18作品)
- `1`: 限制级内容(即R18作品)
- `restrict`: (`number`) 作品限制级(意义不明, 可能是兼容性问题?)?
- `sl`: (`number`) 不明?
- `url`: (`string`) 作品预览图链接, 需要`Referer`请求头
- `description`: (`string`) 作品说明
- `tags`: (`string[]`) 作品标签数组
- `userId`: (`string` -> `number`) 作者用户Id
- `userName`: (`string`) 作者用户名
- `width`: (`number`) 作品长度
- `height`: (`number`) 作品高度
- `pageCount`: (`number`) 作品页数
- `isBookmarkable`: (`boolean`) 不明?
- `alt`: (`string`) 简略介绍信息(在图片加载失败时可提供给`img`标签使用)
- `isAdContainer`: (`boolean`) 不明?
- `titleCaptionTranslation`: (`Object`) 不明?
- `workTitle`: (`Unknown`) 不明?
- `workCaption`: (`Unknown`) 不明?
- `createDate`: (`string`) 作品创建时间(或者是完成时间?)
- `updateDate`: (`string`) 作品上传时间
- `profileImageUrl`: (`string`) 作者用户头像图片链接
- `type`: (`string`) 作品类型名

View File

@ -0,0 +1,166 @@
## Pixiv 排行榜获取接口 ##
### 接口地址 ###
```
GET https://www.pixiv.net/ranking.php
```
- 是否需要登录: `是`
- 是否为Pixiv标准接口返回格式: `否`
- 是否需要Referer请求头: `否`
### 参数 ###
> 提示: 该接口参数较为复杂,请结合表格查看
- `date`: 排行榜时间与Mode有关 (格式: yyyy-MM-dd)
- `mode`: 排行榜模式
- `daily`: 每天
- `weekly`: 每周
- `monthly`: 每月
- `rookie`: 新人
- `original`: 原创
- `male`: 男性偏好
- `female`: 女性偏好
- `daily_r18`: 每天 - 仅成人内容
- `weekly_r18`: 每周 - 仅成人内容
- `male_r18`: 男性偏好 - 仅成人内容
- `female_r18`: 女性偏好 - 仅成人内容
- `content`: 排行榜内容类型
- `all`: 全部内容 (实际使用请直接省略`content`参数)
- `illust`: 插画
- `ugoira`: 动图
- `manga`: 漫画
- `p`: 排行榜页数 (如超出范围,则返回错误信息)
- `format`: 格式
- `json`: 以Json返回数据
- (留空): 返回完整的排行榜网页
#### 参数关系表 ####
`mode`参数与`content`参数有一些支持关系,并不是所有的`mode`参数都能被所有的`content`参数支持,故附下表。
参数 |all|illust|ugoira|manga
:-: |:-:| :-: | :-: | :-:
daily |`√`|`√`|`√`|`√`
weekly |`√`|`√`|`√`|`√`
monthly |`√`|`√`|×|`√`
rookie |`√`|`√`|×|`√`
original |`√`|×|×|×
male |`√`|×|×|×
female |`√`|×|×|×
daily_r18 |`√`|`√`|`√`|`√`
weekly_r18|`√`|`√`|`√`|`√`
male_r18 |`√`|×|×|×
female_r18|`√`|×|×|×
### 返回数据 ###
#### 数据示例 ####
```json
{
"contents":[
{
"title":"【伊アオ】髪結い。",
"date":"2020年05月31日 14:26",
"tags":[
"鬼滅の刃",
"伊アオ",
"嘴平伊之助",
"神崎アオイ",
"鬼滅の刃1000users入り"
],
"url":"https:\/\/i.pximg.net\/c\/240x480\/img-master\/img\/2020\/05\/31\/14\/26\/41\/81987309_p0_master1200.jpg",
"illust_type":"0",
"illust_book_style":"0",
"illust_page_count":"1",
"user_name":"シロウ",
"profile_img":"https:\/\/i.pximg.net\/user-profile\/img\/2020\/05\/01\/02\/18\/18\/18450100_ac34872504959f8cc26f086248066b39_50.png",
"illust_content_type":{
"sexual":0,
"lo":false,
"grotesque":false,
"violent":false,
"homosexual":false,
"drug":false,
"thoughts":false,
"antisocial":false,
"religion":false,
"original":false,
"furry":false,
"bl":false,
"yuri":false
},
"illust_series":false,
"illust_id":81987309,
"width":600,
"height":2226,
"user_id":174995,
"rank":51,
"yes_rank":83,
"rating_count":707,
"view_count":19759,
"illust_upload_timestamp":1590902801,
"attr":"",
"is_bookmarked":false,
"bookmarkable":true
}, // ....
],
"mode":"daily",
"content":"all",
"page":2,
"prev":1,
"next":3,
"date":"20200601",
"prev_date":"20200531",
"next_date":false,
"rank_total":500
}
```
#### 字段说明 ####
- `contents`: (`Object[]`) 排行榜数组, 最多50行排行榜信息
- `illust_id`: (`number`) 作品Id
- `title`: (`string`) 作品标题
- `attr`: (`string`) 不明?
- `tags`: (`string[]`) 原始标签数组
- `url`: (`string`) 预览画质的原始尺寸图下载链接(存在防盗链)
- `illust_type`: (`string` -> `number`) 作品类型
- `illust_book_style`: (`string` -> `number`) 不明?
- `illust_page_count`: (`string` -> `number`) 作品页数
- `user_name`: (`string`) 画师用户名
- `user_id`: (`number`) 画师用户Id
- `profile_img`: (`string`) 画师用户头像
- `illust_content_type`: (`Object`) 作品内容信息(警告: 文档内容仅作为开发参考, 并不传播相关内容!!!)
- `sexual`: (`number`) 作品内容分级
- `0`: 全年龄
- `1`: 青少年
- `2`: 成人级
- `lo`: (`boolean`) 是否为loli作品
- `grotesque`: (`boolean`) 是否为怪诞作品
- `violent`: (`boolean`) 作品是否含有暴力/强暴相关元素
- `homosexual`: (`boolean`) 作品是否含有同性恋相关元素
- `drug`: (`boolean`) 作品是否含有药物相关元素
- `thoughts`: (`boolean`) 作品是否含有思维/记忆相关元素(这个属性翻译起来有些问题, 待纠正)?
- `antisocial`: (`boolean`) 作品是否含有反社会, 令人厌恶的相关元素
- `religion`: (`boolean`) 作品是否含有宗教, 信仰相关元素
- `original`: (`boolean`) 作品是否为原创作品
- `furry`: (`boolean`) 作品是否有兽人相关元素
- `bl`: (`boolean`) 作品是否有耽美相关元素
- `yuri`: (`boolean`) 作品是否有百合相关元素
- `illust_series`: (`boolean`) 是否为系列作品
- `width`: (`number`) 作品宽度(建议以原图为准)
- `height`: (`number`) 作品高度(建议以原图为准)
- `rank`: (`number`) 本期排行榜排名
- `yes_rank`: (`number`) 上期同排行榜排名
- `rating_count`:
- `view_count`: (`number`) 浏览量
- `illust_upload_timestamp`: (`number`) 作品上传时间戳(10位)
- `is_bookmarked`: (`boolean`) 不明?
- `bookmarkable`: (`boolean`) 不明?
- `mode`: (`string`) 请求的排行榜模式字段
- `content`: (`string`) 请求的内容类型
- `page`: (`number`) 当前排行榜页数
- `prev`: (`string` / `boolean`) 上一页排行榜页数, 如果该请求的页数为首页, 则为`false`
- `next`: (`string` / `boolean`) 下一页排行榜页数, 如果该请求的页数为页尾, 则为`false`
- `date`: (`string`) 排行榜日期(格式:`yyyyMMdd`)
- `prev_date`: (`string` / `boolean`) 如果存在上一期排行榜, 则该属性为上期排行榜日期字符串, 否则为`false`
- `next_date`: (`string` / `boolean`) 如果存在下一期排行榜, 则该属性为下期排行榜日期字符串, 否则为`false`
- `rank_total`: (`number`) 该排行榜的总排行数

View File

@ -0,0 +1,32 @@
## 接口名 ##
### 说明 ###
### 接口地址 ###
```
GET/POST https://www.pixiv.net/...
```
- 是否需要登录: `是/否`
- 是否为Pixiv标准接口返回格式: `是/否`
- 是否需要Referer请求头: `是/否`
### 参数 ###
### 请求示例 ###
```
GET/POST
---- Request Body ---- // 如果没有, 可以不写, 如没有记得删除
```
### 返回数据 ###
#### 数据示例 ####
```json
// Object[] 中只留一个, 其他删除后保留逗号, 增加 '// ...' 注释
```
#### 字段说明 ####
- `属性名`: (`JS类型`) 属性说明
- `对象属性名`: (`原始JS类型` -> `可转换JS类型`) 属性说明
- `属性名`: (`JS类型1` / `JS类型2` / ...) 属性说明, 需要清晰说明在什么情况下类型为`JS类型1`, 什么情况下是`JS类型2`.
- `不明属性`: (`Unknown`) 如果属性用途不明, 则在说明后面加上`?`符号, 类型不明则填`Unknown`.

View File

@ -0,0 +1,18 @@
## Pixiv接口标准返回格式 ##
Pixiv大部分接口在返回数据时都会遵循以下格式
```json
{
"error": false,
"message": "",
"body": {
}
}
```
大部分是如此(部分接口比较特别, 不是这个格式)
属性|类型|说明
---|---|---
error|Boolean|如果接口返回错误信息, 该属性为`true`
message|String|如果`error`属性为`true`, 则该属性不为空字符串
body|Object/Array|如果`error`不为`true`, 该属性有数据, 否则属性可能不存在

View File

@ -0,0 +1,365 @@
## Pixiv内容搜索接口 ##
### 说明 ###
> 注意: 本接口可能会影响Pixiv对账号的行为判断猜测不一定会
该接口用于在Pixiv搜索内容。
### 接口地址 ###
```
GET https://www.pixiv.net/ajax/search/{Type}/{Content}?{Param...}
```
- 是否需要登录: `是/否`
- 是否为Pixiv标准接口返回格式: `是/否`
- 是否需要Referer请求头: `否`
### 参数 ###
#### Url参数 ####
- `Type`: 内容类型
- illustrations(插画)
- top(推荐)
- manga(漫画)
- novels(小说)
- `Content`: 搜索内容
#### GET参数 ####
##### 必填 #####
- `word`: 与搜索内容一致 (经测试似乎可以省略)
- `s_mode`: 匹配模式
- `s_tag`: 标签,部分一致
- `s_tag_full`: 标签,完全一致
- `s_tc`: 标签和说明文字
- `type`: 作品类型
- `all`: 插画、漫画和动图
- `illust_and_ugoira`: 插画和动图
- `illust`: 仅插画
- `manga`: 仅漫画
- `ugoira`: 仅动图
- `p`: 指定页数 (当指定页数超出范围后,`body.illust.data`为空数组)
- `order`: 排序设置
- `date`: 按时间从旧到新
- `date_d`: 按时间从新到旧
- `(Unknown)`: 未知, 猜测是会员功能的热门搜索
- `mode`: 内容分级设置
- `all`: 全部内容
- `safe`: 排除成人内容
- `r18`: 仅成人内容
##### 选填 #####
- `wlt`: 作品最低宽度(px)
- `wgt`: 作品最高宽度(px)
- `hlt`: 作品最低高度(px)
- `hgt`: 作品最高高度(px)
- `ratio`: 作品横宽比过滤 (初步测试表明,该参数无法指定横宽比,可能暂不支持该功能)
- `0.5`: 仅横图
- `-0.5`: 仅纵图
- `0`: 仅正方形图
- `tool`: 限定作品绘制工具
- `scd`: 过滤作品发布时间 - 开始时间(yyyy-MM-dd)
- `scd`: 过滤作品发布时间 - 结束时间(yyyy-MM-dd)
- `(Unknown)`: 最小收藏数 (该参数为会员限定功能,后续补充)
### 返回数据 ###
#### 数据示例 ####
```json
{
"error":false,
"body":{
"illustManga":{
"data":[
{
"illustId":"82130571",
"illustTitle":"空の絵",
"id":"82130571",
"title":"空の絵",
"illustType":0,
"xRestrict":0,
"restrict":0,
"sl":2,
"url":"https:\/\/i.pximg.net\/c\/250x250_80_a2\/img-master\/img\/2020\/06\/06\/17\/51\/14\/82130571_p0_square1200.jpg",
"description":"",
"tags":[
"風景",
"空",
"草",
"雲"],
"userId":"31507675",
"userName":"昏omeme",
"width":1600,
"height":1600,
"pageCount":2,
"isBookmarkable":true,
"bookmarkData":null,
"alt":"#風景 空の絵 - 昏omeme的插画",
"isAdContainer":false,
"titleCaptionTranslation":{
"workTitle":null,
"workCaption":null
},
"createDate":"2020-06-06T17:51:14+09:00",
"updateDate":"2020-06-06T17:51:14+09:00",
"profileImageUrl":"https:\/\/i.pximg.net\/user-profile\/img\/2020\/05\/06\/19\/21\/04\/18509741_e3166e69809c44d6926454ecaac89590_50.png"
}, // ...
],
"total":165875,
"bookmarkRanges":[
{
"min":null,
"max":null
},
{
"min":10000,
"max":null
},
{
"min":5000,
"max":null
},
{
"min":1000,
"max":null
},
{
"min":500,
"max":null
},
{
"min":300,
"max":null
},
{
"min":100,
"max":null
},
{
"min":50,
"max":null
}
]
},
"popular":{
"recent":[
{
"illustId":"82062770",
"illustTitle":"Still you remember",
"id":"82062770",
"title":"Still you remember",
"illustType":0,
"xRestrict":0,
"restrict":0,
"sl":2,
"url":"https:\/\/i.pximg.net\/c\/250x250_80_a2\/img-master\/img\/2020\/06\/03\/18\/02\/15\/82062770_p0_square1200.jpg",
"description":"",
"tags":[
"オリジナル",
"女の子",
"カラス",
"风景",
"線路"],
"userId":"1069005",
"userName":"へちま",
"width":2000,
"height":1415,
"pageCount":1,
"isBookmarkable":true,
"bookmarkData":null,
"alt":"#オリジナル Still you remember - へちま的插画",
"isAdContainer":false,
"titleCaptionTranslation":{
"workTitle":null,
"workCaption":null
},
"createDate":"2020-06-03T18:02:15+09:00",
"updateDate":"2020-06-03T18:02:15+09:00",
"profileImageUrl":"https:\/\/i.pximg.net\/user-profile\/img\/2013\/05\/10\/00\/38\/05\/6213543_c94edc0d13776214467bd0c47ee6491a_50.jpg"
}, // ...
],
"permanent":[
{
"illustId":"60993044",
"illustTitle":"無題",
"id":"60993044",
"title":"無題",
"illustType":0,
"xRestrict":0,
"restrict":0,
"sl":2,
"url":"https:\/\/i.pximg.net\/c\/250x250_80_a2\/img-master\/img\/2017\/01\/18\/13\/07\/46\/60993044_p0_square1200.jpg",
"description":"",
"tags":[
"少女",
"女の子",
"原创",
"オリジナル",
"场景",
"落書き",
"創作",
"背景",
"风景",
"オリジナル7500users入り"],
"userId":"18811972",
"userName":"淅洵",
"width":3507,
"height":2480,
"pageCount":1,
"isBookmarkable":true,
"bookmarkData":null,
"alt":"#少女 無題 - 淅洵的插画",
"isAdContainer":false,
"titleCaptionTranslation":{
"workTitle":null,
"workCaption":null
},
"createDate":"2017-01-18T13:07:46+09:00",
"updateDate":"2017-01-18T13:07:46+09:00",
"profileImageUrl":"https:\/\/i.pximg.net\/user-profile\/img\/2017\/05\/29\/17\/17\/49\/12623968_6cf3f1979e10643425972ae205a7920d_50.jpg"
}, // ...
]
},
"relatedTags":[
"風景",
"背景",
"風景画",
"空",
"雲",
"創作",
"ファンタジー",
"夏",
"青",
"建物",
"青空",
"少女",
"東京",
"抽象画",
"男の子",
"透明水彩"
],
"tagTranslation":{
"風景":{
"zh":"风景"
},
"背景":{
"zh":"background"
},
"風景画":{
"zh":"landscape painting"
},
"空":{
"zh":"sky"
},
"雲":{
"zh":"云"
},
"創作":{
"zh":"原创"
},
"ファンタジー":{
"zh":"奇幻"
},
"夏":{
"zh":"夏天"
},
"青":{
"zh":"蓝"
},
"建物":{
"zh":"building"
},
"青空":{
"zh":"蓝天"
},
"少女":{
"zh":"young girl"
},
"東京":{
"zh":"tokyo"
},
"抽象画":{
"zh":"abstract art"
},
"男の子":{
"zh":"男孩子"
},
"透明水彩":{
"zh":"transparent watercolor"
}
},
"zoneConfig":{
"header":{
"url":"https:\/\/pixon.ads-pixiv.net\/show?zone_id=header&format=js&s=1&up=0&a=22&ng=g&l=zh&uri=%2Fajax%2Fsearch%2Fartworks%2F_PARAM_&is_spa=1&K=59bba275c645c&ab_test_digits_first=20&yuid=FwdzEnA&suid=Pgfip96ymw5tvu9l9&num=5edb6277927"
},
"footer":{
"url":"https:\/\/pixon.ads-pixiv.net\/show?zone_id=footer&format=js&s=1&up=0&a=22&ng=g&l=zh&uri=%2Fajax%2Fsearch%2Fartworks%2F_PARAM_&is_spa=1&K=59bba275c645c&ab_test_digits_first=20&yuid=FwdzEnA&suid=Pgfip96yn1fgocj2&num=5edb6277775"
},
"infeed":{
"url":"https:\/\/pixon.ads-pixiv.net\/show?zone_id=illust_search_grid&format=js&s=1&up=0&a=22&ng=g&l=zh&uri=%2Fajax%2Fsearch%2Fartworks%2F_PARAM_&is_spa=1&K=59bba275c645c&ab_test_digits_first=20&yuid=FwdzEnA&suid=Pgfip96yn4t7cho88&num=5edb6277137"
}
},
"extraData":{
"meta":{
"title":"#风景のイラスト・マンガ作品(投稿超过10万件 - pixiv",
"description":"pixiv",
"canonical":"https:\/\/www.pixiv.net\/tags\/%E9%A3%8E%E6%99%AF",
"alternateLanguages":{
"ja":"https:\/\/www.pixiv.net\/tags\/%E9%A3%8E%E6%99%AF",
"en":"https:\/\/www.pixiv.net\/en\/tags\/%E9%A3%8E%E6%99%AF"
},
"descriptionHeader":"pixiv"
}
}
}
}
```
#### 字段说明 ####
- `novel`: (`Object`) 小说搜索结果
- `data`: (`Object`) 搜索结果(仅当前页数)
- (待补充)
- `total`: (`number`) 搜索结果总量
- `popular`: (`Object`) 受欢迎的搜索结果
- `relatedTags`: (`string[]`) 与搜索结果相关的标签
- `tagTranslation`: (`Object`) 相关标签的翻译信息
- `{Attr: 标签名}`: 标签名为属性名
- `语言名(例如 中文是 zh)`: (`string`) 标签翻译名
- `zoneConfig`: (`Object`) 猜测是广告相关信息?
- `extraData`: (`Object`) 扩展信息
- `meta`: (`Object`) 网页元数据
- `title`: (`string`) 网页标题
- `description`: 搜索结果说明内容
- `descriptionHeader`: (`string`) 说明内容的Html代码
- `alternateLanguages`: (`Object`) 不明链接?
- `{语言名}`: 对应语言的链接
- `illustManga`: (`Object`) 漫画和插画的搜索结果
- `total`: (`number`) 搜索结果总量
- `data`: (`Object[]`) 搜索结果(仅当前页数)
- `illustId`: (`string` -> `number`) 作品Id
- `illustTitle`: (`string`) 作品标题
- `id`: (`string` -> `number`) 与`illustId`一致, 猜测是以兼容旧版本为目录而存在
- `title`: (`string`) 与`illustTitle`一致, 猜测是以兼容旧版本为目录而存在
- `illustType`: (`number`) 作品类型
- `0`: 插画作品
- `1`: 漫画作品
- `2`: 动图作品
- `xRestrict`: (`number`) 作品是否为限制级, 基本准确, 少部分不一定(看Pixiv审核怎么理解了)
- `0`: 非限制级内容(即非R18作品)
- `1`: 限制级内容(即R18作品)
- `restrict`: (`number`) 作品限制级(意义不明, 可能是兼容性问题?)?
- `sl`: (`number`) 不明?
- `url`: (`string`) 作品预览图链接, 需要`Referer`请求头
- `description`: (`string`) 作品说明
- `tags`: (`string[]`) 作品标签数组
- `userId`: (`string` -> `number`) 作者用户Id
- `userName`: (`string`) 作者用户名
- `width`: (`number`) 作品长度
- `height`: (`number`) 作品高度
- `pageCount`: (`number`) 作品页数
- `isBookmarkable`: (`boolean`) 不明?
- `bookmarkData`: (`Unknown`) 不明?
- `alt`: (`string`) 简略介绍信息(在图片加载失败时可提供给`img`标签使用)
- `isAdContainer`: (`boolean`) 不明?
- `titleCaptionTranslation`: (`Object`) 不明?
- `workTitle`: (`Unknown`) 不明?
- `workCaption`: (`Unknown`) 不明?
- `createDate`: (`string`) 作品创建时间(或者是完成时间?)
- `updateDate`: (`string`) 作品上传时间
- `profileImageUrl`: (`string`) 作者用户头像图片链接

View File

@ -0,0 +1,129 @@
## Pixiv首页数据接口 ##
### 说明 ###
> 注意: 该接口涉及用户账户隐私, 不要尝试对该接口返回数据做不安全云端存储, 或未经用户允许的发送出去.
该接口用于获取Pixiv推荐给账号的首页作品信息每次调用都会有不同结果。
### 接口地址 ###
```
GET https://www.pixiv.net/ajax/top/{type}?mode={mode}&lang={lang}
```
### 参数 ###
- `type`: 首页类型
- `illust`: 插画
- `manga`: 漫画
- `novel`: 小说
- `mode`: 内容类型
- `all`: 不限类型
- `r18`: 成人内容
- `lang`: 语言(只写几个)
- `zh`: 中文
- 是否需要登录: `是`
- 是否为Pixiv接口标准返回格式: `是`
- 是否需要Referer请求头: `是`
### 请求示例 ###
```
GET https://www.pixiv.net/ajax/top/illust?mode=all&lang=zh
```
### 返回数据 ###
#### 数据示例 ####
```
(内容过长, 略)
```
#### 字段说明 ####
- `page`: 网页相关内容
- `tags`: (`Object[]`) 热门标签
- `tag`: (`String`) 标签名
- `count`: (`Number`) 作品数量?
- `lev`: (`Number`) 不明?
- `ids`: (`Number[]`) 作品数组
- `follow`: (`Number[]`) 已关注作者的作品推荐
- `mypixiv`: (`?[]`) 不明
- `recommend`: (`String[]` -> `Number[]`) 推荐作品的Id?
- `recommendByTag`: (`Object[]`) 标签的推荐作品
- `tag`: (`String`) 标签名
- `ids`: ((`String[]` -> `Number[]`) 作品Id数组
- `ranking`: (`Object`) 排行榜前100名数据
- `date`: (`String`) 排行榜日期(`yyyyMMdd`)
- `items`: (`Object[]`) 排行榜简略数据
- `rank`: (`String` -> `Number`) 排行榜名次
- `id`: (`String` -> `Number`) 作品Id
- `pixivision`: (`Object[]`) Pixiv的推荐文章
- `id`: (`String` -> `Number`) 文章Id
- `title`: (`String`) 文章标题
- `thumbnailUrl`: (`String`) 文章封面图的Url地址
- `abtestId`: (`String` -> `Number`) pixivision文章地址的参数`utm_content`的值
- `recommendUser`: (`Object[]`) 推荐用户及其作品
- `id`: (`Number`) 用户Id
- `illustIds`: (`String[]` -> `Number[]`) 插画作品Id
- `novelIds`: (`String[]` -> `Number[]`) 小说作品Id
- `contestOngoing`: (`Object[]`) 进行中的比赛活动(没找到更多信息了, 先这么说着)
- (待完善)
- `contestResult`: (`Object[]`) 比赛结果信息
- `slug`: (`String`) 比赛代号?
- `type`: (`String`) 比赛作品类型?
- `name`: (`String`) 比赛名
- `url`: (`String`) 结果公布链接
- `iconUrl`: (`String`) 图标链接(不明意义的图标)
- `editorRecommend`: (`Object[]`) 编辑推荐(小说), 后续可能会删除, 这个应该是配合活动出的数据
- (待完善)
- `boothFollowItemIds`: (`String[]` -> `Number[]`) 已关注用户的最新商品
- `sketchLiveFollowIds`: (`?[]`) 不明?
- `sketchLivePopularIds`: (`String[]` -> `Number[]`) 不明?
- `myFavoriteTags`: (`?[]`) 关注的标签
- `newPost`: (`String[]` -> `Number[]`) 不明?
- `thumbnails`: (`Object`) 已关注用户的作品
- `illust`: (`Object[]`) 插画作品
- `illustId`: (`String` -> `Number`) 作品Id(或者准确来讲是 插画Id?)
- `illustTitle`: (`String`) 插画标题
- `id`: (`String` -> `Number`) 作品Id
- `title`: (`String`) 作品标题
- `illustType`: (`Number`) 作品类型
- `xRestrict`: (`Number`) 不明?
- `restrict`: (`Number`) 不明?
- `sl`: (`Number`) 不明?
- `url`: (`String`) 作品在主页的封面图Url
- `description`: (`String`) 作品说明?
- `tags`: (`String[]`) 标签原始名数组(不带翻译的原始名称)
- `userId`: (`String` -> `Number`) 用户Id
- `userName`: (`String`) 用户名
- `width`: (`Number`) 作品宽度
- `height`: (`Number`) 作品高度
- `pageCount`: (`Number`) 作品页数
- `isBookmarkable`: (`Boolean`) 不明?
- `bookmarkData`: (`?`) 不明?
- `alt`: (`String`) 简略信息
- `isAdContainer`: (`Boolean`) 广告标识?
- `titleCaptionTranslation`: (`Object`) 不明?
- (待完善)
- `urls`: (`Object`) 其他封面图尺寸的链接
- (略)
- `seriesId`: (`?`) 不明?
- `seriesTitle`: (`?`) 不明?
- `profileImageUrl`: (`String`) 用户头像图链接
- `users`: (`Object[]`) 用户信?
- `userId`: (`String` -> `Number`) 用户Id
- `name`: (`String`) 用户名
- `image`: (`String`) 用户头像图链接
- `imageBig`: (`String`) 用户大尺寸头像图链接
- `premium`: (`Boolean`) 是否为Pixiv高级会员
- `isFollowed`: (`Boolean`) 是否关注
- `isMypixiv`: (`Boolean`) 不明?
- `isBlocking`: (`Boolean`) 是否为黑名单?
- `background`: (`?`) 不明?
- `partial`: (`Number`) 不明?
- `tagTranslation`: 标签翻译名
- `key=${标签原始名}`: (`Object`) 标签翻译信息
- `${语言代码}`: (`String`) 对应语言代码的翻译, 不一定有
- `boothItems`: (`Object[]`) 商品信息
- (待完善)
- `sketchLives`: (`Object[]`) 不明?
- (待完善)
- `zoneConfig`: (`Object`) 不明?
- (待完善)

11
pom.xml
View File

@ -6,7 +6,7 @@
<groupId>net.lamgc</groupId>
<artifactId>ContentGrabbingJi</artifactId>
<version>2.5.0-20200505.1-SNAPSHOT</version>
<version>2.5.2-20200609.1-SNAPSHOT</version>
<repositories>
<repository>
@ -19,7 +19,7 @@
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<maven.compiler.encoding>UTF-8</maven.compiler.encoding>
<mirai.CoreVersion>0.39.4</mirai.CoreVersion>
<mirai.CoreVersion>1.0-RC2-1</mirai.CoreVersion>
<mirai.JaptVersion>1.1.1</mirai.JaptVersion>
<kotlin.version>1.3.71</kotlin.version>
<ktor.version>1.3.2</ktor.version>
@ -86,7 +86,7 @@
<dependency>
<groupId>net.lamgc</groupId>
<artifactId>java-utils</artifactId>
<version>1.2.0_20200505.1-SNAPSHOT</version>
<version>1.2.0_20200517.1-SNAPSHOT</version>
</dependency>
<dependency>
@ -179,6 +179,11 @@
<artifactId>gifencoder</artifactId>
<version>0.10.1</version>
</dependency>
<dependency>
<groupId>org.jline</groupId>
<artifactId>jline</artifactId>
<version>3.15.0</version>
</dependency>
</dependencies>
</project>

View File

@ -1,24 +0,0 @@
搜索标签信息https://www.pixiv.net/ajax/search/tags/标签名
搜索接口:
https://www.pixiv.net/ajax/search/{Type}/搜索内容
Type = illustrations(插画) / top(顶部?) / manga(漫画) / novels(小说)
word=搜索内容 [参数可能不是必须的]
s_mode=s_tag(标签-部分一致) / s_tag_full(标签-完全一致) / s_tc(标题、说明文字)
type= all(插画、漫画、动图_动态插图) / illust_and_ugoira(插画、动图) / illust(插画) / manga(漫画) / ugoira(动图)
p=页数 [超出页数的情况下将获取不到数据(即"body.illust.data"是空数组)]
order=date(按旧排序) / date_d(按新排序) / Unknown(按热门度排序, 需要会员)
mode= all(全部) / safe(全年龄) / r18(咳咳)
可选参数:
wlt=最小宽度像素
wgt=最高宽度像素
hlt=最小高度像素
hgt=最高高度像素
ratio=0.5(横图) / -0.5(纵图) / 0(正方形) [可能不能改变参数, 三个值是固定的]
tool=使用工具, 不是很重要晚些再加
scd=开始时间(yyyy-MM-dd)
ecd=结束时间(yyyy-MM-dd)
最小收藏数 = 收藏数限定参数为会员功能, 无法获取

View File

@ -7,13 +7,15 @@ import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.boot.ApplicationBoot;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.framework.cli.ConsoleMain;
import net.lamgc.cgj.bot.framework.coolq.CQConfig;
import net.lamgc.cgj.bot.framework.mirai.MiraiMain;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivSearchBuilder;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.plps.PixivLoginProxyServer;
import net.lamgc.utils.base.ArgumentsProperties;
import net.lamgc.utils.base.runner.Argument;
import net.lamgc.utils.base.runner.ArgumentsRunner;
import net.lamgc.utils.base.runner.Command;
@ -29,7 +31,6 @@ import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import java.io.*;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import java.text.ParseException;
import java.text.SimpleDateFormat;
@ -42,110 +43,63 @@ import java.util.zip.ZipOutputStream;
@SpringBootApplication
public class Main {
private final static Logger log = LoggerFactory.getLogger("Main");
private final static Logger log = LoggerFactory.getLogger(Main.class);
private final static File storeDir = new File("store/");
public static CookieStore cookieStore;
private static CookieStore cookieStore;
public static HttpHost proxy;
private static HttpHost proxy;
public static void main(String[] args) throws IOException, ClassNotFoundException {
log.trace("ContentGrabbingJi 正在启动...");
log.debug("Args: {}, LogsPath: {}", Arrays.toString(args), System.getProperty("cgj.logsPath"));
log.debug("运行目录: {}", System.getProperty("user.dir"));
ArgumentsProperties argsProp = new ArgumentsProperties(args);
ApplicationBoot.initialApplication(args);
log.debug("botDataDir: {}", System.getProperty("cgj.botDataDir"));
if(!getSettingToSysProp(argsProp, "proxy", null)) {
getEnvSettingToSysProp("CGJ_PROXY", "proxy", null);
}
String proxyAddress = System.getProperty("cgj.proxy");
if(!Strings.isNullOrEmpty(proxyAddress)) {
URL proxyUrl = new URL(proxyAddress);
proxy = new HttpHost(proxyUrl.getHost(), proxyUrl.getPort());
log.info("已启用Http协议代理{}", proxy.toHostString());
} else {
proxy = null;
}
if(!storeDir.exists() && !storeDir.mkdirs()) {
log.error("创建文件夹失败!");
}
if(!getSettingToSysProp(argsProp, "botDataDir", "./") &&
!getEnvSettingToSysProp("CGJ_BOT_DATA_DIR", "botDataDir", "./")) {
log.warn("未设置botDataDir, 当前运行目录将作为酷Q机器人所在目录.");
}
if(!getSettingToSysProp(argsProp, "redisAddress", "127.0.0.1") &&
!getEnvSettingToSysProp("CGJ_REDIS_URI", "redisAddress", "127.0.0.1")) {
log.warn("未设置RedisAddress, 将使用默认值连接Redis服务器(127.0.0.1:6379)");
}
File cookieStoreFile = new File(System.getProperty("cgj.botDataDir"), "cookies.store");
proxy = BotGlobal.getGlobal().getProxy();
File cookieStoreFile = new File(BotGlobal.getGlobal().getDataStoreDir(), "cookies.store");
if(!cookieStoreFile.exists()) {
log.warn("未找到cookies.store文件, 是否启动PixivLoginProxyServer? (yes/no)");
Scanner scanner = new Scanner(System.in);
if(scanner.nextLine().trim().equalsIgnoreCase("yes")) {
startPixivLoginProxyServer();
} else {
System.exit(1);
return;
try(Scanner scanner = new Scanner(System.in)) {
if(scanner.nextLine().trim().equalsIgnoreCase("yes")) {
startPixivLoginProxyServer();
} else {
System.exit(1);
return;
}
}
}
ObjectInputStream ois = new ObjectInputStream(new FileInputStream(cookieStoreFile));
cookieStore = (CookieStore) ois.readObject();
BotGlobal.getGlobal().setCookieStore(cookieStore);
ois.close();
log.info("已载入CookieStore");
log.debug("传入参数: {}", Arrays.toString(args));
ArgumentsRunner.run(Main.class, args);
}
/**
* 从ArgumentsProperties获取设置项到System Properties
* @param prop ArgumentsProperties对象
* @param key 设置项key
* @param defaultValue 默认值
* @return 如果成功从ArgumentsProperties获得设置项, 返回true, 如未找到(使用了defaultValue或null), 返回false;
*/
private static boolean getSettingToSysProp(ArgumentsProperties prop, String key, String defaultValue) {
if(prop.containsKey(key)) {
log.info("{}: {}", key, prop.getValue(key));
System.setProperty("cgj." + key, prop.getValue(key));
return true;
} else {
if(defaultValue != null) {
System.setProperty("cgj." + key, defaultValue);
}
return false;
}
}
private static boolean getEnvSettingToSysProp(String envKey, String sysPropKey, String defaultValue) {
String env = System.getenv(envKey);
if(env != null) {
System.setProperty("cgj." + sysPropKey, env);
return true;
} else if(defaultValue != null) {
System.setProperty("cgj." + sysPropKey, defaultValue);
}
return false;
System.exit(0);
}
@Command
public static void botMode(@Argument(name = "args", force = false) String argsStr) {
new MiraiMain().init();
MiraiMain main = new MiraiMain();
main.init();
main.close();
}
@Command
public static void consoleMode() throws IOException {
ConsoleMain.start();
}
@Command
public static void pluginMode(@Argument(name = "args", force = false) String argsStr) {
if(!System.getProperty("cgj.botDataDir").endsWith("\\") && !System.getProperty("cgj.botDataDir").endsWith("/")) {
System.setProperty("cgj.botDataDir", System.getProperty("cgj.botDataDir") + "/");
}
log.info("酷Q机器人根目录: {}", System.getProperty("cgj.botDataDir"));
log.info("酷Q机器人根目录: {}", BotGlobal.getGlobal().getDataStoreDir().getPath());
CQConfig.init();
Pattern pattern = Pattern.compile("/\\s*(\".+?\"|[^:\\s])+((\\s*:\\s*(\".+?\"|[^\\s])+)|)|(\".+?\"|[^\"\\s])+");
Matcher matcher = pattern.matcher(Strings.nullToEmpty(argsStr));
@ -161,7 +115,7 @@ public class Main {
@Command
public static void collectionDownload() throws IOException {
PixivDownload pixivDownload = new PixivDownload(Objects.requireNonNull(cookieStore), proxy);
File outputFile = new File(storeDir, "collection.zip");
File outputFile = new File(getStoreDir(), "collection.zip");
if(!outputFile.exists() && !outputFile.createNewFile()) {
log.error("文件创建失败: " + outputFile.getAbsolutePath());
}
@ -188,10 +142,10 @@ public class Main {
PixivDownload pixivDownload = new PixivDownload(Objects.requireNonNull(cookieStore), proxy);
String date = new SimpleDateFormat("yyyyMMdd").format(new Date());
int id = 1;
File outputFile = new File(storeDir, "recommends-" + date + "-" + id + ".zip");
File outputFile = new File(getStoreDir(), "recommends-" + date + "-" + id + ".zip");
while(outputFile.exists()) {
id++;
outputFile = new File(storeDir, "recommends-" + date + "-" + id + ".zip");
outputFile = new File(getStoreDir(), "recommends-" + date + "-" + id + ".zip");
}
if(!outputFile.createNewFile()) {
@ -255,10 +209,10 @@ public class Main {
}
int id = 1;
File outputFile = new File(storeDir, "ranking" + rankingMode.modeParam + "-" + date + "-" + id + ".zip");
File outputFile = new File(getStoreDir(), "ranking" + rankingMode.modeParam + "-" + date + "-" + id + ".zip");
while(outputFile.exists()) {
id++;
outputFile = new File(storeDir, "ranking" + rankingMode.modeParam + "-" + date + "-" + id + ".zip");
outputFile = new File(getStoreDir(), "ranking" + rankingMode.modeParam + "-" + date + "-" + id + ".zip");
}
if(!outputFile.createNewFile()) {
@ -397,8 +351,8 @@ public class Main {
private static void saveCookieStoreToFile() throws IOException {
log.info("正在保存CookieStore...");
File outputFile = new File(System.getProperty("cgj.botDataDir"), "cookies.store");
if(!outputFile.exists() && !outputFile.delete() && !outputFile.createNewFile()){
File outputFile = new File(BotGlobal.getGlobal().getDataStoreDir(), "cookies.store");
if(!outputFile.exists() && !outputFile.createNewFile()){
log.error("保存CookieStore失败.");
return;
}
@ -423,25 +377,34 @@ public class Main {
proxyServerStartThread.setName("LoginProxyServerThread");
proxyServerStartThread.start();
//System.console().readLine();
Scanner scanner = new Scanner(System.in);
log.info("登录完成后, 使用\"done\"命令结束登录过程.");
while(true) {
if (scanner.nextLine().equalsIgnoreCase("done")) {
log.info("关闭PLPS服务器...");
proxyServer.close();
cookieStore = proxyServer.getCookieStore();
try {
log.info("正在保存CookieStore...");
saveCookieStoreToFile();
log.info("CookieStore保存完成.");
} catch (IOException e) {
log.error("CookieStore保存时发生异常, 本次CookieStore仅可在本次运行使用.", e);
try(Scanner scanner = new Scanner(System.in)) {
while(true) {
if (scanner.nextLine().equalsIgnoreCase("done")) {
log.info("关闭PLPS服务器...");
proxyServer.close();
cookieStore = proxyServer.getCookieStore();
try {
log.info("正在保存CookieStore...");
saveCookieStoreToFile();
log.info("CookieStore保存完成.");
} catch (IOException e) {
log.error("CookieStore保存时发生异常, 本次CookieStore仅可在本次运行使用.", e);
}
break;
} else {
log.warn("要结束登录过程, 请使用\"done\"命令.");
}
break;
} else {
log.warn("要结束登录过程, 请使用\"done\"命令.");
}
}
}
private static File getStoreDir() {
if(!storeDir.exists() && !storeDir.mkdirs()) {
log.error("创建文件夹失败!");
}
return storeDir;
}
}

View File

@ -1,57 +0,0 @@
package net.lamgc.cgj.bot;
import net.lz1998.cq.robot.CoolQ;
import org.apache.http.client.methods.HttpGet;
import java.util.Random;
import java.util.Timer;
import java.util.TimerTask;
public class AutoArtworksSender {
private final CoolQ CQ;
private final ReceiveType receiveType;
private final long targetReceiveId;
private Timer timer = new Timer();
private TimerTask task = new TimerTask() {
@Override
public void run() {
HttpGet request = new HttpGet();
// https://api.imjad.cn/pixiv/v2/?type=tags
}
};
public AutoArtworksSender(CoolQ cq, ReceiveType receiveType, long receiveId) {
this.CQ = cq;
this.receiveType = receiveType;
this.targetReceiveId = receiveId;
}
public void reset(long time) {
if(time <= 0) {
timer.schedule(task, new Random().nextInt(10 * 60 * 60 * 1000) + 7200000L); //2H ~ 12H
} else {
timer.schedule(task, time);
}
}
public void sendMessage(String message, boolean auto_escape) {
switch (receiveType) {
case GROUP:
CQ.sendGroupMsg(targetReceiveId, message, auto_escape);
break;
case Discuss:
CQ.sendDiscussMsg(targetReceiveId, message, auto_escape);
break;
case PRIVATE:
CQ.sendPrivateMsg(targetReceiveId, message, auto_escape);
break;
}
}
public enum ReceiveType {
PRIVATE, GROUP, Discuss
}
}

View File

@ -9,7 +9,7 @@ import java.util.Objects;
*/
public abstract class AutoSender {
private MessageSender messageSender;
private final MessageSender messageSender;
/**
* 构造一个自动发送器

View File

@ -5,6 +5,7 @@ import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonObject;
import com.google.gson.reflect.TypeToken;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lamgc.cgj.pixiv.PixivDownload;
@ -20,11 +21,9 @@ import java.util.*;
public class BotAdminCommandProcess {
private final static Logger log = LoggerFactory.getLogger(BotAdminCommandProcess.class.getSimpleName());
private final static Logger log = LoggerFactory.getLogger(BotAdminCommandProcess.class);
private final static File globalPropFile = new File(System.getProperty("cgj.botDataDir"), "global.properties");
private final static File pushListFile = new File(System.getProperty("cgj.botDataDir"), "pushList.json");
private final static File pushListFile = new File(BotGlobal.getGlobal().getDataStoreDir(), "pushList.json");
private final static Hashtable<Long, JsonObject> pushInfoMap = new Hashtable<>();
@ -39,59 +38,47 @@ public class BotAdminCommandProcess {
}
@Command
public static String setGlobalProperty(@Argument(name = "key") String key, @Argument(name = "value") String value, @Argument(name = "save", force = false) boolean saveNow) {
String lastValue = BotCommandProcess.globalProp.getProperty(key);
BotCommandProcess.globalProp.setProperty(key, Strings.nullToEmpty(value));
if(saveNow) {
saveGlobalProperties();
public static String setProperty(
@Argument(name = "group", force = false) long groupId,
@Argument(name = "key") String key,
@Argument(name = "value") String value
) {
if(Strings.isNullOrEmpty(key)) {
return "未选择配置项key.";
}
return "全局配置项 " + key + " 现已设置为: " + value + " (设置前的值: " + lastValue + ")";
String lastValue = SettingProperties.setProperty(groupId, key, value.equals("null") ? null : value);
return (groupId <= 0 ? "已更改全局配置 " : "已更改群组 " + groupId + " 配置 ") +
key + " 的值: '" + value + "' (原配置值: '" + lastValue + "')";
}
@Command
public static String getGlobalProperty(@Argument(name = "key") String key) {
return "全局配置项 " + key + " 当前值: " + BotCommandProcess.globalProp.getProperty(key, "(Empty)");
public static String getProperty(
@Argument(name = "group", force = false) long groupId,
@Argument(name = "key") String key
) {
if(Strings.isNullOrEmpty(key)) {
return "未选择配置项key.";
}
return (groupId <= 0 ? "全局配置 " : "群组 " + groupId + " 配置 ") +
key + " 设定值: '" + SettingProperties.getProperty(groupId, key, "(empty)") + "'";
}
@Command
public static String saveGlobalProperties() {
log.info("正在保存全局配置文件...");
try {
if(!globalPropFile.exists()) {
if(!globalPropFile.createNewFile()) {
log.error("全局配置项文件保存失败!({})", "文件创建失败");
return "全局配置项文件保存失败!";
}
}
BotCommandProcess.globalProp.store(new FileOutputStream(globalPropFile), "");
log.info("全局配置文件保存成功!");
return "保存全局配置文件 - 操作已完成.";
} catch (IOException e) {
log.error("全局配置项文件保存失败!", e);
return "全局配置项文件保存失败!";
}
public static String saveProperties() {
log.info("正在保存配置文件...");
SettingProperties.saveProperties();
log.info("配置文件保存操作已完成.");
return "保存配置 - 操作已完成.";
}
@Command
public static String loadGlobalProperties(@Argument(name = "reload", force = false) boolean reload) {
Properties cache = new Properties();
if(!globalPropFile.exists()) {
return "未找到全局配置文件, 无法重载";
}
try(Reader reader = new BufferedReader(new FileReader(globalPropFile))) {
cache.load(reader);
} catch (IOException e) {
log.error("重载全局配置文件时发生异常", e);
return "加载全局配置文件时发生错误!";
}
public static String loadProperties(@Argument(name = "reload", force = false) boolean reload) {
if(reload) {
BotCommandProcess.globalProp.clear();
SettingProperties.clearProperties();
}
BotCommandProcess.globalProp.putAll(cache);
return "全局配置文件重载完成.";
SettingProperties.loadProperties();
return "操作已完成.";
}
@Command
@ -125,6 +112,30 @@ public class BotAdminCommandProcess {
@Argument(name = "type", force = false, defaultValue = "ILLUST") String rankingContentType,
@Argument(name = "original", force = false, defaultValue = "false") boolean original
) {
if(minTime <= 0 || floatTime <= 0) {
return "时间不能为0或负数";
} else if(rankingStart <= 0 || rankingStop - rankingStart <= 0) {
return "排行榜范围选取错误!";
}
PixivURL.RankingContentType type;
PixivURL.RankingMode mode;
try {
type = PixivURL.RankingContentType.valueOf("TYPE_" + rankingContentType.toUpperCase());
} catch(IllegalArgumentException e) {
return "无效的排行榜类型参数!";
}
try {
mode = PixivURL.RankingMode.valueOf("MODE_" + rankingMode.toUpperCase());
} catch(IllegalArgumentException e) {
return "无效的排行榜模式参数!";
}
if(!type.isSupportedMode(mode)) {
return "不兼容的排行榜模式与类型!";
}
long group = groupId <= 0 ? fromGroup : groupId;
JsonObject setting = new JsonObject();
setting.addProperty(RANKING_SETTING_TIME_MIN, minTime);
@ -141,7 +152,8 @@ public class BotAdminCommandProcess {
removePushGroup(fromGroup, groupId);
}
log.info("正在增加Timer...(Setting: {})", setting);
log.info("群组 {} 新推送配置: {}", group, setting);
log.info("正在增加Timer...");
pushInfoMap.put(group, setting);
addPushTimer(group, setting);
return "已在 " + group + " 开启定时推送功能。";
@ -171,8 +183,8 @@ public class BotAdminCommandProcess {
@Command
public static String savePushList() {
try {
if(!pushListFile.exists()) {
pushListFile.createNewFile();
if(!pushListFile.exists() && !pushListFile.createNewFile()) {
throw new IOException("文件夹创建失败!(Path: " + pushListFile.getPath() + ")");
}
} catch (IOException e) {
log.error("PushList.json文件创建失败", e);
@ -259,6 +271,7 @@ public class BotAdminCommandProcess {
AutoSender sender = new RandomRankingArtworksSender(
MessageSenderBuilder.getMessageSender(MessageSource.Group, id),
id,
rankingStart,
rankingEnd,
rankingMode, rankingContentType,

View File

@ -1,100 +1,47 @@
package net.lamgc.cgj.bot;
import com.google.common.base.Strings;
import com.google.common.base.Throwables;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import com.google.gson.*;
import io.netty.handler.codec.http.HttpHeaderNames;
import net.lamgc.cgj.Main;
import net.lamgc.cgj.bot.cache.*;
import net.lamgc.cgj.bot.event.BotEventHandler;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.cache.CacheStore;
import net.lamgc.cgj.bot.cache.CacheStoreCentral;
import net.lamgc.cgj.bot.cache.JsonRedisCacheStore;
import net.lamgc.cgj.bot.event.BufferMessageEvent;
import net.lamgc.cgj.bot.sort.PreLoadDataComparator;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivDownload.PageQuality;
import net.lamgc.cgj.pixiv.PixivSearchBuilder;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.cgj.util.URLs;
import net.lamgc.utils.base.runner.Argument;
import net.lamgc.utils.base.runner.Command;
import net.lamgc.utils.event.EventExecutor;
import net.lz1998.cq.utils.CQCode;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpHead;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
@SuppressWarnings({"SynchronizationOnLocalVariableOrMethodParameter", "SameParameterValue"})
@SuppressWarnings({"SameParameterValue"})
public class BotCommandProcess {
private final static PixivDownload pixivDownload = new PixivDownload(Main.cookieStore, Main.proxy);
private final static Logger log = LoggerFactory.getLogger(BotCommandProcess.class.getSimpleName());
private final static File imageStoreDir = new File(System.getProperty("cgj.botDataDir"), "data/image/cgj/");
public final static Properties globalProp = new Properties();
private final static Gson gson = new GsonBuilder()
.serializeNulls()
.create();
private final static Hashtable<String, File> imageCache = new Hashtable<>();
private final static CacheStore<JsonElement> illustInfoCache = new JsonRedisCacheStore(BotEventHandler.redisServer, "illustInfo", gson);
private final static CacheStore<JsonElement> illustPreLoadDataCache = new HotDataCacheStore<>(
new JsonRedisCacheStore(BotEventHandler.redisServer, "illustPreLoadData", gson),
new LocalHashCacheStore<>(), 3600000, 900000);
private final static CacheStore<JsonElement> searchBodyCache = new JsonRedisCacheStore(BotEventHandler.redisServer, "searchBody", gson);
private final static CacheStore<List<JsonObject>> rankingCache = new JsonObjectRedisListCacheStore(BotEventHandler.redisServer, "ranking", gson);
private final static CacheStore<List<String>> pagesCache = new StringListRedisCacheStore(BotEventHandler.redisServer, "imagePages");
public final static CacheStore<JsonElement> reportStore = new JsonRedisCacheStore(BotEventHandler.redisServer, "report", gson);
private final static Logger log = LoggerFactory.getLogger(BotCommandProcess.class);
/**
* 图片异步缓存执行器
* 作品报告存储 - 不过期
*/
private final static EventExecutor imageCacheExecutor = new EventExecutor(new ThreadPoolExecutor(
Runtime.getRuntime().availableProcessors() >= 2 ? 2 : 1,
(int) Math.ceil(Runtime.getRuntime().availableProcessors() / 2F),
5L, TimeUnit.SECONDS,
new LinkedBlockingQueue<>(128),
new ThreadFactoryBuilder()
.setNameFormat("imageCacheThread-%d")
.build(),
new ThreadPoolExecutor.DiscardOldestPolicy()
));
public final static CacheStore<JsonElement> reportStore =
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(),
"report", BotGlobal.getGlobal().getGson());
private final static RankingUpdateTimer updateTimer = new RankingUpdateTimer();
public static void initialize() {
log.info("正在初始化...");
File globalPropFile = new File(System.getProperty("cgj.botDataDir"), "global.properties");
if(globalPropFile.exists() && globalPropFile.isFile()) {
log.info("正在加载全局配置文件...");
try {
globalProp.load(new FileInputStream(globalPropFile));
log.info("全局配置文件加载完成.");
} catch (IOException e) {
log.error("加载全局配置文件时发生异常", e);
}
} else {
log.info("未找到全局配置文件,跳过加载.");
}
try {
imageCacheExecutor.addHandler(new ImageCacheHandler());
} catch (IllegalAccessException e) {
e.printStackTrace();
}
SettingProperties.loadProperties();
updateTimer.schedule(null);
log.info("初始化完成.");
@ -112,74 +59,86 @@ public class BotCommandProcess {
@Command(defaultCommand = true)
public static String help() {
StringBuilder helpStrBuilder = new StringBuilder();
helpStrBuilder.append("CGJ Bot使用指南").append("\n");
helpStrBuilder.append("使用方法:.cgj <命令> [参数...]").append("\n");
helpStrBuilder.append("例如查询作品信息功能:").append("\n");
helpStrBuilder.append(".cgj artwork -id 80846159").append("\n");
helpStrBuilder.append("目前可用的命令:").append("\n");
helpStrBuilder.append("\t").append("ranking - 获取今天或指定日期排行榜的前10名作品").append("\n");
helpStrBuilder.append("\t\t").append("-date - 指定查询日期(年-月-日)").append("\n");
helpStrBuilder.append("\t\t").append("-type - 排行榜类型(illust/插画, ugoira/动图, manga/漫画)").append("\n");
helpStrBuilder.append("\t").append("search - 搜索指定关键词并显示前10个作品").append("\n");
helpStrBuilder.append("\t\t").append("-content - 搜索内容").append("\n");
helpStrBuilder.append("\t").append("link - 获取作品的Pixiv页面").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品id").append("\n");
helpStrBuilder.append("\t").append("info - 获取Pixiv作品信息").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品id").append("\n");
helpStrBuilder.append("\t").append("image - 获取指定作品的图片").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品id").append("\n");
helpStrBuilder.append("\t\t").append("-quality - 图片质量(original/原图 regular/预览图)").append("\n");
helpStrBuilder.append("\t\t").append("-page - 页数").append("\n");
helpStrBuilder.append("\t").append("report - 报告不当作品").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品Id").append("\n");
helpStrBuilder.append("\t\t").append("-msg - 报告原因").append("\n");
return helpStrBuilder.toString();
return "CGJ Bot使用指南" + "\n" +
"使用方法:.cgj <命令> [参数...]" + "\n" +
"例如查询作品信息功能:" + "\n" +
".cgj info -id 80846159" + "\n" +
"目前可用的命令:" + "\n" +
"\t" + "ranking - 获取今天或指定日期排行榜的前10名作品" + "\n" +
"\t\t" + "-date - 指定查询日期(年-月-日)" + "\n" +
"\t\t" + "-type - 排行榜类型(illust/插画, ugoira/动图, manga/漫画)" + "\n" +
"\t" + "search - 搜索指定关键词并显示前10个作品" + "\n" +
"\t\t" + "-content - 搜索内容" + "\n" +
"\t" + "link - 获取作品的Pixiv页面" + "\n" +
"\t\t" + "-id - 作品id" + "\n" +
"\t" + "info - 获取Pixiv作品信息" + "\n" +
"\t\t" + "-id - 作品id" + "\n" +
"\t" + "image - 获取指定作品的图片" + "\n" +
"\t\t" + "-id - 作品id" + "\n" +
"\t\t" + "-quality - 图片质量(original/原图 regular/预览图)" + "\n" +
"\t\t" + "-page - 页数" + "\n" +
"\t" + "report - 报告不当作品" + "\n" +
"\t\t" + "-id - 作品Id" + "\n" +
"\t\t" + "-msg - 报告原因" + "\n";
}
/**
* 作品信息查询
* @param fromGroup 来源群(系统提供)
* @param illustId 作品Id
* @return 返回作品信息
*/
@Command(commandName = "info")
public static String artworkInfo(@Argument(name = "id") int illustId) {
public static String artworkInfo(@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "id") int illustId) {
if(illustId <= 0) {
return "错误的作品id";
return "这个作品Id是错误的!";
}
try {
if(isNoSafe(illustId, globalProp, false) || isReported(illustId)) {
if(isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false) || isReported(illustId)) {
return "阅览禁止:该作品已被封印!!";
}
JsonObject illustPreLoadData = getIllustPreLoadData(illustId, false);
StringBuilder builder = new StringBuilder("---------------- 作品信息 ----------------\n");
builder.append("作品Id: ").append(illustId).append("\n");
builder.append("作品标题:").append(illustPreLoadData.get("illustTitle").getAsString()).append("\n");
builder.append("作者(作者Id)").append(illustPreLoadData.get("userName").getAsString())
.append("(").append(illustPreLoadData.get("userId").getAsInt()).append(")\n");
builder.append("点赞数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.LIKE.attrName).getAsInt()).append("\n");
builder.append("收藏数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.BOOKMARK.attrName).getAsInt()).append("\n");
builder.append("围观数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.VIEW.attrName).getAsInt()).append("\n");
builder.append("评论数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.COMMENT.attrName).getAsInt()).append("\n");
builder.append("页数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.PAGE.attrName).getAsInt()).append("\n");
builder.append("---------------- 作品图片 ----------------\n");
builder.append(getImageById(illustId, PixivDownload.PageQuality.REGULAR, 1)).append("\n");
builder.append("使用 \".cgj image -id ")
.append(illustId)
.append("\" 获取原图。\n如有不当作品使用\".cgj report -id ")
.append(illustId).append("\"向色图姬反馈。");
return builder.toString();
JsonObject illustPreLoadData = CacheStoreCentral.getIllustPreLoadData(illustId, false);
// 在 Java 6 开始, 编译器会将用'+'进行的字符串拼接将自动转换成StringBuilder拼接
return "色图姬帮你了解了这个作品的信息!\n" + "---------------- 作品信息 ----------------" +
"\n作品Id: " + illustId +
"\n作品标题" + illustPreLoadData.get("illustTitle").getAsString() +
"\n作者(作者Id)" + illustPreLoadData.get("userName").getAsString() +
"(" + illustPreLoadData.get("userId").getAsInt() + ")" +
"\n点赞数:" + illustPreLoadData.get(PreLoadDataComparator.Attribute.LIKE.attrName).getAsInt() +
"\n收藏数" + illustPreLoadData.get(PreLoadDataComparator.Attribute.BOOKMARK.attrName).getAsInt() +
"\n围观数:" + illustPreLoadData.get(PreLoadDataComparator.Attribute.VIEW.attrName).getAsInt() +
"\n评论数" + illustPreLoadData.get(PreLoadDataComparator.Attribute.COMMENT.attrName).getAsInt() +
"\n页数" + illustPreLoadData.get(PreLoadDataComparator.Attribute.PAGE.attrName).getAsInt() + "" +
"\n作品链接" + artworksLink(fromGroup, illustId) + "\n" +
"---------------- 作品图片 ----------------\n" +
CacheStoreCentral.getImageById(fromGroup, illustId, PageQuality.REGULAR, 1) + "\n" +
"使用 \".cgj image -id " +
illustId +
"\" 获取原图。\n如有不当作品可使用\".cgj report -id " +
illustId + "\"向色图姬反馈。";
} catch (IOException e) {
e.printStackTrace();
}
return "尚未支持";
}
/**
* 排行榜命令
* @param fromGroup 来源群(系统提供)
* @param queryTime 查询时间, 格式: 年-月-日
* @param force 是否强制查询, 当主动提供的时间不在查询范围时, 是否强制查询, 仅系统可用
* @param contentMode 内容模式
* @param contentType 排行榜类型
* @return 返回排行榜信息
*/
@Command
public static String ranking(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(force = false, name = "date") Date queryTime,
@Argument(force = false, name = "force") boolean force,
@Argument(force = false, name = "mode", defaultValue = "DAILY") String contentMode,
@Argument(force = false, name = "type", defaultValue = "ILLUST") String contentType
) {
@ -196,7 +155,7 @@ public class BotCommandProcess {
}
queryDate = gregorianCalendar.getTime();
} else {
if(new Date().before(queryDate)) {
if(new Date().before(queryDate) && !force) {
log.warn("查询的日期过早, 无法查询排行榜.");
return "查询日期过早, 暂未更新指定日期的排行榜!";
}
@ -205,7 +164,8 @@ public class BotCommandProcess {
PixivURL.RankingMode mode;
try {
String rankingModeValue = contentMode.toUpperCase();
mode = PixivURL.RankingMode.valueOf(rankingModeValue.startsWith("MODE_") ? rankingModeValue : "MODE_" + rankingModeValue);
mode = PixivURL.RankingMode.valueOf(rankingModeValue.startsWith("MODE_") ?
rankingModeValue : "MODE_" + rankingModeValue);
} catch (IllegalArgumentException e) {
log.warn("无效的RankingMode值: {}", contentMode);
return "参数无效, 请查看帮助信息";
@ -214,7 +174,8 @@ public class BotCommandProcess {
PixivURL.RankingContentType type;
try {
String contentTypeValue = contentType.toUpperCase();
type = PixivURL.RankingContentType.valueOf(contentTypeValue.startsWith("TYPE_") ? contentTypeValue : "TYPE_" + contentTypeValue);
type = PixivURL.RankingContentType.valueOf(
contentTypeValue.startsWith("TYPE_") ? contentTypeValue : "TYPE_" + contentTypeValue);
} catch (IllegalArgumentException e) {
log.warn("无效的RankingContentType值: {}", contentType);
return "参数无效, 请查看帮助信息";
@ -226,13 +187,15 @@ public class BotCommandProcess {
return "不支持的内容类型或模式!";
}
StringBuilder resultBuilder = new StringBuilder(mode.name() + " - 以下是 ").append(new SimpleDateFormat("yyyy-MM-dd").format(queryDate)).append(" 的Pixiv插画排名榜前十名\n");
StringBuilder resultBuilder = new StringBuilder(mode.name() + " - 以下是 ")
.append(new SimpleDateFormat("yyyy-MM-dd").format(queryDate)).append(" 的Pixiv插画排名榜前十名\n");
try {
int index = 0;
int itemLimit = 10;
String itemLimitPropertyKey = "ranking.ItemCountLimit";
String itemLimitPropertyKey = "ranking.itemCountLimit";
try {
itemLimit = Integer.parseInt(globalProp.getProperty(itemLimitPropertyKey, "10"));
itemLimit = Integer.parseInt(SettingProperties
.getProperty(fromGroup, itemLimitPropertyKey, "10"));
} catch(NumberFormatException e) {
log.warn("配置项 {} 的参数值格式有误!", itemLimitPropertyKey);
}
@ -240,12 +203,14 @@ public class BotCommandProcess {
int imageLimit = 3;
String imageLimitPropertyKey = "ranking.imageCountLimit";
try {
imageLimit = Integer.parseInt(globalProp.getProperty(imageLimitPropertyKey, "3"));
imageLimit = Integer.parseInt(
SettingProperties.getProperty(fromGroup, imageLimitPropertyKey, "3"));
} catch(NumberFormatException e) {
log.warn("配置项 {} 的参数值格式有误!", imageLimitPropertyKey);
}
List<JsonObject> rankingInfoList = getRankingInfoByCache(type, mode, queryDate, 1, Math.max(0, itemLimit), false);
List<JsonObject> rankingInfoList = CacheStoreCentral
.getRankingInfoByCache(type, mode, queryDate, 1, Math.max(0, itemLimit), false);
if(rankingInfoList.isEmpty()) {
return "无法查询排行榜,可能排行榜尚未更新。";
}
@ -259,26 +224,72 @@ public class BotCommandProcess {
String authorName = rankInfo.get("user_name").getAsString();
String title = rankInfo.get("title").getAsString();
resultBuilder.append(rank).append(". (id: ").append(illustId).append(") ").append(title)
.append("(Author: ").append(authorName).append(",").append(authorId).append(") ").append(pagesCount).append("p.\n");
.append("(Author: ").append(authorName).append(",").append(authorId).append(") ")
.append(pagesCount).append("p.\n");
if (index <= imageLimit) {
resultBuilder.append(getImageById(illustId, PixivDownload.PageQuality.REGULAR, 1)).append("\n");
resultBuilder
.append(CacheStoreCentral
.getImageById(fromGroup, illustId, PixivDownload.PageQuality.REGULAR, 1))
.append("\n");
}
}
} catch (IOException e) {
log.error("消息处理异常", e);
return "排名榜获取失败!详情请查看机器人控制台。";
}
return resultBuilder.append("如查询当前时间获取到昨天时间,则今日排名榜未更新。\n如有不当作品,可使用\".cgj report -id 作品id\"向色图姬反馈。").toString();
return resultBuilder.append("如查询当前时间获取到昨天时间,则今日排名榜未更新。\n" +
"如有不当作品,可使用\".cgj report -id 作品id\"向色图姬反馈。").toString();
}
/**
* 查询指定作者的作品(尚未完成)
* @return 返回作者信息和部分作品
*/
@Command(commandName = "userArt")
public static String userArtworks() {
return "功能未完成";
}
/**
* 随机获取一副作品
*/
@Command(commandName = "random")
public static String randomImage(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(force = false, name = "mode", defaultValue = "DAILY") String contentMode,
@Argument(force = false, name = "type", defaultValue = "ILLUST") String contentType) {
PixivURL.RankingMode mode;
try {
String rankingModeValue = contentMode.toUpperCase();
mode = PixivURL.RankingMode.valueOf(rankingModeValue.startsWith("MODE_") ?
rankingModeValue : "MODE_" + rankingModeValue);
} catch (IllegalArgumentException e) {
log.warn("无效的RankingMode值: {}", contentMode);
return "参数无效, 请查看帮助信息";
}
PixivURL.RankingContentType type;
try {
String contentTypeValue = contentType.toUpperCase();
type = PixivURL.RankingContentType.valueOf(
contentTypeValue.startsWith("TYPE_") ? contentTypeValue : "TYPE_" + contentTypeValue);
} catch (IllegalArgumentException e) {
log.warn("无效的RankingContentType值: {}", contentType);
return "参数无效, 请查看帮助信息";
}
BufferMessageEvent event = new BufferMessageEvent();
RandomRankingArtworksSender artworksSender =
new RandomRankingArtworksSender(event, fromGroup, 1, 200, mode, type,
PageQuality.ORIGINAL);
artworksSender.send();
return event.getBufferMessage();
}
/**
* 搜索命令
* @param fromGroup 来源群(系统提供)
* @param content 搜索内容
* @param type 搜索类型
* @param area 搜索区域
@ -290,107 +301,33 @@ public class BotCommandProcess {
* @throws IOException 当搜索发生异常时抛出
*/
@Command
public static String search(@Argument(name = "content") String content,
@Argument(name = "type", force = false) String type,
@Argument(name = "area", force = false) String area,
@Argument(name = "in", force = false) String includeKeywords,
@Argument(name = "ex", force = false) String excludeKeywords,
@Argument(name = "contentOption", force = false) String contentOption,
@Argument(name = "page", force = false, defaultValue = "1") int pagesIndex
public static String search(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "content") String content,
@Argument(name = "type", force = false) String type,
@Argument(name = "area", force = false) String area,
@Argument(name = "in", force = false) String includeKeywords,
@Argument(name = "ex", force = false) String excludeKeywords,
@Argument(name = "option", force = false) String contentOption,
@Argument(name = "page", force = false, defaultValue = "1") int pagesIndex
) throws IOException {
log.info("正在执行搜索...");
PixivSearchBuilder searchBuilder = new PixivSearchBuilder(Strings.isNullOrEmpty(content) ? "" : content);
if (type != null) {
try {
searchBuilder.setSearchType(PixivSearchBuilder.SearchType.valueOf(type.toUpperCase()));
} catch (IllegalArgumentException e) {
log.warn("不支持的SearchType: {}", type);
}
}
if (area != null) {
try {
searchBuilder.setSearchArea(PixivSearchBuilder.SearchArea.valueOf(area));
} catch (IllegalArgumentException e) {
log.warn("不支持的SearchArea: {}", area);
}
}
if (contentOption != null) {
try {
searchBuilder.setSearchContentOption(PixivSearchBuilder.SearchContentOption.valueOf(contentOption));
} catch (IllegalArgumentException e) {
log.warn("不支持的SearchContentOption: {}", contentOption);
}
}
if (!Strings.isNullOrEmpty(includeKeywords)) {
for (String keyword : includeKeywords.split(";")) {
searchBuilder.removeExcludeKeyword(keyword.trim());
searchBuilder.addIncludeKeyword(keyword.trim());
log.debug("已添加关键字: {}", keyword);
}
}
if (!Strings.isNullOrEmpty(excludeKeywords)) {
for (String keyword : excludeKeywords.split(";")) {
searchBuilder.removeIncludeKeyword(keyword.trim());
searchBuilder.addExcludeKeyword(keyword.trim());
log.debug("已添加排除关键字: {}", keyword);
}
}
log.info("正在搜索作品, 条件: {}", searchBuilder.getSearchCondition());
String requestUrl = searchBuilder.buildURL().intern();
log.debug("RequestUrl: {}", requestUrl);
JsonObject resultBody = null;
if(!searchBodyCache.exists(requestUrl)) {
synchronized (requestUrl) {
if (!searchBodyCache.exists(requestUrl)) {
log.debug("searchBody缓存失效, 正在更新...");
JsonObject jsonObject;
HttpGet httpGetRequest = pixivDownload.createHttpGetRequest(requestUrl);
HttpResponse response = pixivDownload.getHttpClient().execute(httpGetRequest);
String responseBody = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
log.debug("ResponseBody: {}", responseBody);
jsonObject = gson.fromJson(responseBody, JsonObject.class);
if (jsonObject.get("error").getAsBoolean()) {
log.error("接口请求错误, 错误信息: {}", jsonObject.get("message").getAsString());
return "处理命令时发生错误!";
}
long expire = 7200 * 1000;
String propValue = globalProp.getProperty("cache.searchBody.expire", "7200000");
try {
expire = Long.parseLong(propValue);
} catch (Exception e) {
log.warn("全局配置项 \"{}\" 值非法, 已使用默认值: {}", propValue, expire);
}
resultBody = jsonObject.getAsJsonObject().getAsJsonObject("body");
searchBodyCache.update(requestUrl, jsonObject, expire);
log.debug("searchBody缓存已更新(有效时间: {})", expire);
} else {
log.debug("搜索缓存命中.");
}
}
} else {
log.debug("搜索缓存命中.");
}
if(Objects.isNull(resultBody)) {
resultBody = searchBodyCache.getCache(requestUrl).getAsJsonObject().getAsJsonObject("body");
}
JsonObject resultBody = CacheStoreCentral
.getSearchBody(content, type, area, includeKeywords, excludeKeywords, contentOption);
StringBuilder result = new StringBuilder("内容 " + content + " 的搜索结果:\n");
log.debug("正在处理信息...");
int limit = 8;
try {
limit = Integer.parseInt(globalProp.getProperty("search.ItemCountLimit", "8"));
limit = Integer.parseInt(SettingProperties.
getProperty(fromGroup, "search.itemCountLimit", "8"));
} catch (Exception e) {
log.warn("参数转换异常!将使用默认值(" + limit + ")", e);
}
int totalCount = 0;
for (PixivSearchBuilder.SearchArea searchArea : PixivSearchBuilder.SearchArea.values()) {
if (!resultBody.has(searchArea.jsonKey) || resultBody.getAsJsonObject(searchArea.jsonKey).getAsJsonArray("data").size() == 0) {
if (!resultBody.has(searchArea.jsonKey) ||
resultBody.getAsJsonObject(searchArea.jsonKey).getAsJsonArray("data").size() == 0) {
log.debug("返回数据不包含 {}", searchArea.jsonKey);
continue;
}
@ -414,7 +351,8 @@ public class BotCommandProcess {
StringBuilder builder = new StringBuilder("[");
illustObj.get("tags").getAsJsonArray().forEach(el -> builder.append(el.getAsString()).append(", "));
builder.replace(builder.length() - 2, builder.length(), "]");
log.debug("{} ({} / {})\n\t作品id: {}, \n\t作者名(作者id): {} ({}), \n\t作品标题: {}, \n\t作品Tags: {}, \n\t页数: {}, \n\t作品链接: {}",
log.debug("{} ({} / {})\n\t作品id: {}, \n\t作者名(作者id): {} ({}), \n\t" +
"作品标题: {}, \n\t作品Tags: {}, \n\t页数: {}页, \n\t作品链接: {}",
searchArea.name(),
count,
illustsList.size(),
@ -427,10 +365,9 @@ public class BotCommandProcess {
PixivURL.getPixivRefererLink(illustId)
);
//pageCount
String imageMsg = getImageById(illustId, PixivDownload.PageQuality.REGULAR, 1);
if (isNoSafe(illustId, globalProp, true)) {
String imageMsg =
CacheStoreCentral.getImageById(fromGroup, illustId, PixivDownload.PageQuality.REGULAR, 1);
if (isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) {
log.warn("作品Id {} 为R-18作品, 跳过.", illustId);
continue;
} else if(isReported(illustId)) {
@ -438,25 +375,57 @@ public class BotCommandProcess {
continue;
}
result.append(searchArea.name()).append(" (").append(count).append(" / ").append(limit).append(")\n\t作品id: ").append(illustId)
JsonObject illustPreLoadData = CacheStoreCentral.getIllustPreLoadData(illustId, false);
result.append(searchArea.name()).append(" (").append(count).append(" / ")
.append(limit).append(")\n\t作品id: ").append(illustId)
.append(", \n\t作者名: ").append(illustObj.get("userName").getAsString())
.append("\n\t作品标题: ").append(illustObj.get("illustTitle").getAsString())
.append("\n\t作品页数: ").append(illustObj.get("pageCount").getAsInt())
.append("\n\t作品页数: ").append(illustObj.get("pageCount").getAsInt()).append("")
.append("\n\t点赞数")
.append(illustPreLoadData.get(PreLoadDataComparator.Attribute.LIKE.attrName).getAsInt())
.append("\n\t收藏数")
.append(illustPreLoadData.get(PreLoadDataComparator.Attribute.BOOKMARK.attrName).getAsInt())
.append("\n\t围观数")
.append(illustPreLoadData.get(PreLoadDataComparator.Attribute.VIEW.attrName).getAsInt())
.append("\n\t评论数")
.append(illustPreLoadData.get(PreLoadDataComparator.Attribute.COMMENT.attrName).getAsInt())
.append("\n").append(imageMsg).append("\n");
count++;
totalCount++;
}
if (count > limit) {
break;
}
}
return Strings.nullToEmpty(result.toString()) + "预览图片并非原图,使用“.cgj image -id 作品id”获取原图\n如有不当作品可使用\".cgj report -id 作品id\"向色图姬反馈。";
return totalCount <= 0 ?
"搜索完成,未找到相关作品。" :
Strings.nullToEmpty(result.toString()) + "预览图片并非原图,使用“.cgj image -id 作品id”获取原图\n" +
"如有不当作品,可使用\".cgj report -id 作品id\"向色图姬反馈。";
}
/**
* 获取作品页面的下载链接
* @param illustId 作品Id
* @param quality 画质类型
* @return 返回作品所有页面在Pixiv的下载链接(有防盗链, 考虑要不要设置镜像站)
*/
@Command(commandName = "pages")
public static String getPagesList(@Argument(name = "id") int illustId, @Argument(name = "quality", force = false) PixivDownload.PageQuality quality) {
public static String getPagesList(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "id") int illustId,
@Argument(name = "quality", force = false) PixivDownload.PageQuality quality) {
try {
List<String> pagesList = PixivDownload.getIllustAllPageDownload(pixivDownload.getHttpClient(), pixivDownload.getCookieStore(), illustId, quality);
StringBuilder builder = new StringBuilder("作品ID ").append(illustId).append(" 共有").append(pagesList.size()).append("页:").append("\n");
if(isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) {
log.warn("来源群 {} 查询的作品Id {} 为R18作品, 根据配置设定, 屏蔽该作品.", fromGroup, illustId);
return "该作品已被封印!";
}
List<String> pagesList =
PixivDownload.getIllustAllPageDownload(
BotGlobal.getGlobal().getPixivDownload().getHttpClient(),
BotGlobal.getGlobal().getPixivDownload().getCookieStore(),
illustId, quality);
StringBuilder builder = new StringBuilder("作品ID ").append(illustId)
.append(" 共有").append(pagesList.size()).append("页:").append("\n");
int index = 0;
for (String link : pagesList) {
builder.append("Page ").append(++index).append(": ").append(link).append("\n");
@ -468,10 +437,17 @@ public class BotCommandProcess {
}
}
/**
* 获取作品链接
* @param fromGroup 来源群(系统提供)
* @param illustId 作品Id
* @return 返回作品在Pixiv的链接
*/
@Command(commandName = "link")
public static String artworksLink(@Argument(name = "id") int illustId) {
public static String artworksLink(@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "id") int illustId) {
try {
if (isNoSafe(illustId, globalProp, false)) {
if (isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) {
log.warn("作品Id {} 已被屏蔽.", illustId);
return "由于相关设置,该作品已被屏蔽!";
} else if(isReported(illustId)) {
@ -485,112 +461,10 @@ public class BotCommandProcess {
return PixivURL.getPixivRefererLink(illustId);
}
/**
* 通过illustId获取作品图片
* @param illustId 作品Id
* @param quality 图片质量
* @param pageIndex 指定页面索引, 从1开始
* @return 如果成功, 返回BotCode, 否则返回错误信息.
*/
@Command(commandName = "image")
public static String getImageById(@Argument(name = "id") int illustId,
@Argument(name = "quality", force = false) PixivDownload.PageQuality quality,
@Argument(name = "page", force = false, defaultValue = "1") int pageIndex) {
log.debug("IllustId: {}, Quality: {}, PageIndex: {}", illustId, quality.name(), pageIndex);
List<String> pagesList;
try {
pagesList = getIllustPages(illustId, quality, false);
} catch (IOException e) {
log.error("获取下载链接列表时发生异常", e);
return "发生网络异常,无法获取图片!";
}
if(log.isDebugEnabled()) {
StringBuilder logBuilder = new StringBuilder("作品Id {} 所有页面下载链接: \n");
AtomicInteger index = new AtomicInteger();
pagesList.forEach(item -> logBuilder.append(index.incrementAndGet()).append(". ").append(item).append("\n"));
log.debug(logBuilder.toString());
}
if (pagesList.size() < pageIndex || pageIndex <= 0) {
log.warn("指定的页数超出了总页数({} / {})", pageIndex, pagesList.size());
return "指定的页数超出了范围(总共 " + pagesList.size() + " 页)";
}
try {
if (isNoSafe(illustId, globalProp, false)) {
log.warn("作品 {} 存在R-18内容且设置\"image.allowR18\"为false将屏蔽该作品不发送.", illustId);
return "(根据设置,该作品已被屏蔽!)";
} else if(isReported(illustId)) {
log.warn("作品Id {} 被报告, 正在等待审核, 跳过该作品.", illustId);
return "(该作品已被封印)";
}
} catch (IOException e) {
log.warn("作品信息无法获取!", e);
return "发生网络异常,无法获取图片!";
}
String downloadLink = pagesList.get(pageIndex - 1);
String fileName = URLs.getResourceName(Strings.nullToEmpty(downloadLink));
File imageFile = new File(getImageStoreDir(), downloadLink.substring(downloadLink.lastIndexOf("/") + 1));
log.debug("FileName: {}, DownloadLink: {}", fileName, downloadLink);
if(!imageCache.containsKey(fileName)) {
if(imageFile.exists()) {
HttpHead headRequest = new HttpHead(downloadLink);
headRequest.addHeader("Referer", PixivURL.getPixivRefererLink(illustId));
HttpResponse headResponse;
try {
headResponse = pixivDownload.getHttpClient().execute(headRequest);
} catch (IOException e) {
log.error("获取图片大小失败!", e);
return "图片获取失败!";
}
String contentLengthStr = headResponse.getFirstHeader(HttpHeaderNames.CONTENT_LENGTH.toString()).getValue();
log.debug("图片大小: {}B", contentLengthStr);
if (imageFile.length() == Long.parseLong(contentLengthStr)) {
imageCache.put(URLs.getResourceName(downloadLink), imageFile);
log.debug("作品Id {} 第 {} 页缓存已补充.", illustId, pageIndex);
return getImageToBotCode(imageFile, false).toString();
}
}
ImageCacheObject taskObject = new ImageCacheObject(imageCache, illustId, downloadLink, imageFile);
try {
imageCacheExecutor.executorSync(taskObject);
} catch (InterruptedException e) {
log.error("等待图片下载时发生中断", e);
return "图片获取失败!";
}
} else {
log.debug("图片 {} 缓存命中.", fileName);
}
return getImageToBotCode(imageCache.get(fileName), false).toString();
}
/**
* 通过文件获取图片的BotCode代码
* @param targetFile 图片文件
* @param updateCache 是否刷新缓存(只是让机器人重新上传, 如果上传接口有重复检测的话是无法处理的)
* @return 返回设定好参数的BotCode
*/
private static BotCode getImageToBotCode(File targetFile, boolean updateCache) {
String fileName = targetFile.getName();
BotCode code = BotCode.parse(CQCode.image(getImageStoreDir().getName() + "/" + fileName));
code.addParameter("absolutePath", targetFile.getAbsolutePath());
code.addParameter("imageName", fileName.substring(0, fileName.lastIndexOf(".")));
code.addParameter("updateCache", updateCache ? "true" : "false");
return code;
}
static void clearCache() {
log.warn("正在清除所有缓存...");
imageCache.clear();
illustInfoCache.clear();
illustPreLoadDataCache.clear();
pagesCache.clear();
searchBodyCache.clear();
File imageStoreDir = new File(System.getProperty("cgj.botDataDir") + "data/image/cgj/");
CacheStoreCentral.clearCache();
File imageStoreDir = new File(BotGlobal.getGlobal().getDataStoreDir(), "data/image/cgj/");
File[] listFiles = imageStoreDir.listFiles();
if (listFiles == null) {
log.debug("图片缓存目录为空或内部文件获取失败!");
@ -603,14 +477,30 @@ public class BotCommandProcess {
log.warn("缓存删除完成.");
}
@Command(commandName = "image")
public static String getImageById(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "id") int illustId,
@Argument(name = "quality", force = false) PixivDownload.PageQuality quality,
@Argument(name = "page", force = false, defaultValue = "1") int pageIndex
) {
return CacheStoreCentral.getImageById(fromGroup, illustId, quality, pageIndex);
}
/**
* 举报某一作品
* @param fromGroup 来源群(系统提供)
* @param illustId 需要举报的作品id
* @param reason 举报原因
* @return 返回提示信息
*/
@Command
public static String report(@Argument(name = "$fromGroup") long fromGroup, @Argument(name = "$fromQQ") long fromQQ, @Argument(name = "id") int illustId, @Argument(name = "msg", force = false) String reason) {
public static String report(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "$fromQQ") long fromQQ,
@Argument(name = "id") int illustId,
@Argument(name = "msg", force = false) String reason
) {
log.warn("收到作品反馈(IllustId: {}, 原因: {})", illustId, reason);
JsonObject reportJson = new JsonObject();
reportJson.addProperty("illustId", illustId);
@ -631,13 +521,6 @@ public class BotCommandProcess {
return reportStore.exists(String.valueOf(illustId));
}
/*
下一目标:
添加定时发图
定时发图支持设置关注标签
标签....标签支持搜索吧
*/
/**
* 检查指定作品是否为r18
* @param illustId 作品Id
@ -645,160 +528,25 @@ public class BotCommandProcess {
* @param returnRaw 是否返回原始值
* @return 如果为true, 则不为全年龄
* @throws IOException 获取数据时发生异常时抛出
* @throws NoSuchElementException 当作品不存在时抛出
*/
public static boolean isNoSafe(int illustId, Properties settingProp, boolean returnRaw) throws IOException {
boolean rawValue = getIllustInfo(illustId, false).getAsJsonArray("tags").contains(new JsonPrimitive("R-18"));
return returnRaw || settingProp == null ? rawValue : rawValue && !settingProp.getProperty("image.allowR18", "false").equalsIgnoreCase("true");
}
private static JsonObject getIllustInfo(int illustId, boolean flushCache) throws IOException {
String illustIdStr = buildSyncKey(Integer.toString(illustId));
JsonObject illustInfoObj = null;
if (!illustInfoCache.exists(illustIdStr) || flushCache) {
synchronized (illustIdStr) {
if (!illustInfoCache.exists(illustIdStr) || flushCache) {
illustInfoObj = pixivDownload.getIllustInfoByIllustId(illustId);
illustInfoCache.update(illustIdStr, illustInfoObj, null);
public static boolean isNoSafe(int illustId, Properties settingProp, boolean returnRaw)
throws IOException, NoSuchElementException {
JsonObject illustInfo = CacheStoreCentral.getIllustInfo(illustId, false);
JsonArray tags = illustInfo.getAsJsonArray("tags");
boolean rawValue = illustInfo.get("xRestrict").getAsInt() != 0;
if(!rawValue) {
for(JsonElement tag : tags) {
boolean current = tag.getAsString().matches("R-*18") || tag.getAsString().contains("R18");
if (current) {
rawValue = true;
break;
}
}
}
if(Objects.isNull(illustInfoObj)) {
illustInfoObj = illustInfoCache.getCache(illustIdStr).getAsJsonObject();
log.debug("作品Id {} IllustInfo缓存命中.", illustId);
}
return illustInfoObj;
return returnRaw || settingProp == null ? rawValue :
rawValue && !settingProp.getProperty("image.allowR18", "false")
.equalsIgnoreCase("true");
}
/**
* 获取作品预加载数据.
* 可以获取作品的一些与用户相关的信息
* @param illustId 作品Id
* @param flushCache 是否刷新缓存
* @return 成功返回JsonObject对象
* @throws IOException 当Http请求处理发生异常时抛出
*/
public static JsonObject getIllustPreLoadData(int illustId, boolean flushCache) throws IOException {
String illustIdStr = buildSyncKey(Integer.toString(illustId));
JsonObject result = null;
if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) {
synchronized (illustIdStr) {
if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) {
log.debug("IllustId {} 缓存失效, 正在更新...", illustId);
JsonObject preLoadDataObj = pixivDownload.getIllustPreLoadDataById(illustId)
.getAsJsonObject("illust")
.getAsJsonObject(Integer.toString(illustId));
long expire = 7200 * 1000;
String propValue = globalProp.getProperty("cache.illustPreLoadData.expire", "7200000");
log.debug("PreLoadData有效时间设定: {}", propValue);
try {
expire = Long.parseLong(propValue);
} catch (Exception e) {
log.warn("全局配置项 \"{}\" 值非法, 已使用默认值: {}", propValue, expire);
}
result = preLoadDataObj;
illustPreLoadDataCache.update(illustIdStr, preLoadDataObj, expire);
log.debug("作品Id {} preLoadData缓存已更新(有效时间: {})", illustId, expire);
}
}
}
if(Objects.isNull(result)) {
result = illustPreLoadDataCache.getCache(illustIdStr).getAsJsonObject();
log.debug("作品Id {} PreLoadData缓存命中.", illustId);
}
return result;
}
public static List<String> getIllustPages(int illustId, PixivDownload.PageQuality quality, boolean flushCache) throws IOException {
String pagesSign = buildSyncKey(Integer.toString(illustId), ".", quality.name());
List<String> result = null;
if (!pagesCache.exists(pagesSign) || flushCache) {
synchronized (pagesSign) {
if (!pagesCache.exists(pagesSign) || flushCache) {
List<String> linkList = PixivDownload.getIllustAllPageDownload(pixivDownload.getHttpClient(), pixivDownload.getCookieStore(), illustId, quality);
result = linkList;
pagesCache.update(pagesSign, linkList, null);
}
}
}
if(Objects.isNull(result)) {
result = pagesCache.getCache(pagesSign);
log.debug("作品Id {} Pages缓存命中.", illustId);
}
return result;
}
private static File getImageStoreDir() {
if(!imageStoreDir.exists() && !imageStoreDir.mkdirs()) {
log.warn("酷Q图片缓存目录失效(Path: {} )", imageStoreDir.getAbsolutePath());
throw new RuntimeException(new IOException("文件夹创建失败!"));
}
return imageStoreDir;
}
private final static Random expireTimeFloatRandom = new Random();
/**
* 获取排行榜
* @param contentType 排行榜类型
* @param mode 排行榜模式
* @param queryDate 查询时间
* @param start 开始排名, 从1开始
* @param range 取范围
* @param flushCache 是否强制刷新缓存
* @return 成功返回有值List, 失败且无异常返回空
* @throws IOException 获取异常时抛出
*/
public static List<JsonObject> getRankingInfoByCache(PixivURL.RankingContentType contentType, PixivURL.RankingMode mode, Date queryDate, int start, int range, boolean flushCache) throws IOException {
if(!contentType.isSupportedMode(mode)) {
log.warn("试图获取不支持的排行榜类型已拒绝.(ContentType: {}, RankingMode: {})", contentType.name(), mode.name());
if(log.isDebugEnabled()) {
try {
Thread.dumpStack();
} catch(Exception e) {
log.debug("本次非法请求的堆栈信息如下: \n{}", Throwables.getStackTraceAsString(e));
}
}
return new ArrayList<>(0);
}
String date = new SimpleDateFormat("yyyyMMdd").format(queryDate);
String requestSign = buildSyncKey(contentType.name(), ".", mode.name(), ".", date);
List<JsonObject> result = null;
if(!rankingCache.exists(requestSign) || flushCache) {
synchronized(requestSign) {
if(!rankingCache.exists(requestSign) || flushCache) {
log.debug("Ranking缓存失效, 正在更新...(RequestSign: {})", requestSign);
List<JsonObject> rankingResult = pixivDownload.getRanking(contentType, mode, queryDate, 1, 500);
long expireTime = 0;
if(rankingResult.size() == 0) {
expireTime = 5400000 + expireTimeFloatRandom.nextInt(1800000);
log.warn("数据获取失败, 将设置浮动有效时间以准备下次更新. (ExpireTime: {}ms)", expireTime);
}
result = new ArrayList<>(rankingResult).subList(start - 1, start + range - 1);
rankingCache.update(requestSign, rankingResult, expireTime);
log.debug("Ranking缓存更新完成.(RequestSign: {})", requestSign);
}
}
}
if (Objects.isNull(result)) {
result = rankingCache.getCache(requestSign, start - 1, range);
log.debug("RequestSign [{}] 缓存命中.", requestSign);
}
log.debug("Result-Length: {}", result.size());
return PixivDownload.getRanking(result, start - 1, range);
}
private static String buildSyncKey(String... keys) {
StringBuilder sb = new StringBuilder();
for (String string : keys) {
sb.append(string);
}
return sb.toString().intern();
}
}

View File

@ -26,13 +26,19 @@ public enum MessageEventExecutionDebugger {
try {
rotation = Integer.parseInt(properties.getProperty("debug.pm.rotation", "5"));
} catch(NumberFormatException ignored) {}
} catch(NumberFormatException e) {
log.warn("配置项 {} 值无效, 将使用默认值.({})", "debug.pm.rotation", rotation);
}
try {
number = Integer.parseInt(properties.getProperty("debug.pm.number", "50"));
} catch(NumberFormatException ignored) {}
} catch(NumberFormatException e) {
log.warn("配置项 {} 值无效, 将使用默认值.({})", "debug.pm.number", number);
}
try {
interval = Integer.parseInt(properties.getProperty("debug.pm.interval", "2500"));
} catch(NumberFormatException ignored) {}
} catch(NumberFormatException e) {
log.warn("配置项 {} 值无效, 将使用默认值.({})", "debug.pm.interval", interval);
}
boolean interrupted = false;
Thread currentThread = Thread.currentThread();
@ -52,8 +58,6 @@ public enum MessageEventExecutionDebugger {
}
});
public final static String debuggerLoggerNameTemp = "MessageEventExecuteDebugger-{debugger}";
public final MessageExecuteDebugger debugger;
MessageEventExecutionDebugger(MessageExecuteDebugger debugger) {
@ -61,7 +65,7 @@ public enum MessageEventExecutionDebugger {
}
public static Logger getDebuggerLogger(MessageEventExecutionDebugger debugger) {
return LoggerFactory.getLogger(debuggerLoggerNameTemp.replace("{debugger}", debugger.name()));
return LoggerFactory.getLogger(MessageEventExecutionDebugger.class.getName() + "." + debugger.name());
}
@FunctionalInterface

View File

@ -5,8 +5,6 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.lang.reflect.Field;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.concurrent.atomic.AtomicBoolean;
@ -15,8 +13,8 @@ import java.util.concurrent.atomic.AtomicBoolean;
*/
public class RandomIntervalSendTimer extends TimerTask {
private final static Timer timer = new Timer("Thread-RIST");
private final static Logger log = LoggerFactory.getLogger("RandomIntervalSendTimer");
private final static Timer timer = new Timer("Thread-RIST", true);
private final static Logger log = LoggerFactory.getLogger(RandomIntervalSendTimer.class);
private final static Map<Long, RandomIntervalSendTimer> timerMap = new HashMap<>();
private final long timerId;
@ -24,8 +22,10 @@ public class RandomIntervalSendTimer extends TimerTask {
private final AutoSender sender;
private final long time;
private final int floatTime;
private AtomicBoolean loop = new AtomicBoolean();
private final AtomicBoolean loop = new AtomicBoolean();
private final AtomicBoolean start = new AtomicBoolean();
private final String hashId = Integer.toHexString(this.hashCode());
/**
* 创建一个随机延迟发送器
@ -92,18 +92,16 @@ public class RandomIntervalSendTimer extends TimerTask {
start(this.loop.get());
}
private final static DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-ss HH:mm:ss");
/**
* 启动定时器
* @param loop 是否循环, 如果为true, 则任务完成后会自动调用start方法继续循环, 直到被调用{@code #}或总定时器被销毁;
*/
public void start(boolean loop) {
this.loop.set(loop);
long nextDelay = time + timeRandom.nextInt(floatTime);
long nextDelay = time + (floatTime <= 0 ? 0 : timeRandom.nextInt(floatTime));
Date nextDate = new Date();
nextDate.setTime(nextDate.getTime() + nextDelay);
log.info("定时器 {} 下一延迟: {}ms ({})", Integer.toHexString(this.hashCode()), nextDelay, dateFormat.format(nextDate));
log.info("定时器 {} 下一延迟: {}ms ({})", hashId, nextDelay, nextDate);
if(start.get()) {
try {
Field state = this.getClass().getSuperclass().getDeclaredField("state");

View File

@ -1,13 +1,13 @@
package net.lamgc.cgj.bot;
import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.cache.CacheStoreCentral;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivURL;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.util.Calendar;
import java.util.Date;
import java.util.List;
@ -19,6 +19,7 @@ import java.util.Random;
public class RandomRankingArtworksSender extends AutoSender {
private final Logger log;
private final long groupId;
private final int rankingStart;
private final int rankingStop;
private final PixivURL.RankingMode mode;
@ -35,11 +36,40 @@ public class RandomRankingArtworksSender extends AutoSender {
* @param quality 图片质量, 详见{@link PixivDownload.PageQuality}
* @throws IndexOutOfBoundsException 当 rankingStart > rankingStop时抛出
*/
public RandomRankingArtworksSender(MessageSender messageSender, int rankingStart, int rankingStop, PixivURL.RankingMode mode, PixivURL.RankingContentType contentType, PixivDownload.PageQuality quality) {
public RandomRankingArtworksSender(
MessageSender messageSender,
int rankingStart,
int rankingStop,
PixivURL.RankingMode mode,
PixivURL.RankingContentType contentType,
PixivDownload.PageQuality quality) {
this(messageSender, 0, rankingStart, rankingStop, mode, contentType, quality);
}
/**
* 构造一个推荐作品发送器
* @param messageSender 消息发送器
* @param groupId 群组Id, 如果发送目标为群组, 则可设置群组Id, 以使用群组配置.
* @param rankingStart 排行榜开始范围(从1开始, 名次)如传入0或负数则为默认值默认为1
* @param rankingStop 排名榜结束范围(包括该名次)如传入0或负数则为默认值默认为150
* @param mode 排行榜模式
* @param contentType 排行榜内容类型
* @param quality 图片质量, 详见{@link PixivDownload.PageQuality}
* @throws IndexOutOfBoundsException 当 rankingStart > rankingStop时抛出
*/
public RandomRankingArtworksSender(
MessageSender messageSender,
long groupId,
int rankingStart,
int rankingStop,
PixivURL.RankingMode mode,
PixivURL.RankingContentType contentType,
PixivDownload.PageQuality quality) {
super(messageSender);
this.groupId = groupId;
this.mode = mode;
this.contentType = contentType;
log = LoggerFactory.getLogger("RecommendArtworksSender@" + Integer.toHexString(this.hashCode()));
log = LoggerFactory.getLogger(this.toString());
this.rankingStart = rankingStart > 0 ? rankingStart : 1;
this.rankingStop = rankingStop > 0 ? rankingStop : 150;
if(this.rankingStart > this.rankingStop) {
@ -62,7 +92,7 @@ public class RandomRankingArtworksSender extends AutoSender {
int selectRanking = rankingStart + new Random().nextInt(rankingStop - rankingStart + 1);
try {
List<JsonObject> rankingList = BotCommandProcess.getRankingInfoByCache(
List<JsonObject> rankingList = CacheStoreCentral.getRankingInfoByCache(
contentType,
mode,
queryDate,
@ -77,7 +107,8 @@ public class RandomRankingArtworksSender extends AutoSender {
JsonObject rankingInfo = rankingList.get(0);
int illustId = rankingInfo.get("illust_id").getAsInt();
if(BotCommandProcess.isNoSafe(illustId, BotCommandProcess.globalProp, false)) {
if(BotCommandProcess.isNoSafe(illustId,
SettingProperties.getProperties(groupId), false)) {
log.warn("作品为r18作品, 取消本次发送.");
return;
} else if(BotCommandProcess.isReported(illustId)) {
@ -85,15 +116,14 @@ public class RandomRankingArtworksSender extends AutoSender {
return;
}
StringBuilder message = new StringBuilder();
message.append("#美图推送 - 今日排行榜 第 ").append(rankingInfo.get("rank").getAsInt()).append("\n");
message.append("标题").append(rankingInfo.get("title").getAsString()).append("(").append(illustId).append(")\n");
message.append("作者:").append(rankingInfo.get("user_name").getAsString()).append("\n");
message.append(BotCommandProcess.getImageById(illustId, quality, 1));
message.append("\n如有不当作品可使用\".cgj report -id ").append(illustId).append("\"向色图姬反馈。");
getMessageSender().sendMessage(message.toString());
} catch (IOException e) {
e.printStackTrace();
String message = "#美图推送 - 今日排行榜 第 " + rankingInfo.get("rank").getAsInt() + "\n" +
"标题:" + rankingInfo.get("title").getAsString() + "(" + illustId + ")\n" +
"作者" + rankingInfo.get("user_name").getAsString() + "\n" +
CacheStoreCentral.getImageById(0, illustId, quality, 1) +
"\n如有不当作品可使用\".cgj report -id " + illustId + "\"向色图姬反馈。";
getMessageSender().sendMessage(message);
} catch (Exception e) {
log.error("发送随机作品时发生异常", e);
}
}
}

View File

@ -6,6 +6,7 @@ import net.lamgc.cgj.pixiv.PixivURL;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.text.SimpleDateFormat;
import java.time.LocalDate;
import java.util.Calendar;
import java.util.Date;
@ -26,21 +27,24 @@ public class RankingUpdateTimer {
Calendar cal = Calendar.getInstance();
cal.setTime(firstRunDate == null ? new Date() : firstRunDate);
LocalDate currentLocalDate = LocalDate.now();
if(cal.get(Calendar.DAY_OF_YEAR) <= currentLocalDate.getDayOfYear() && cal.get(Calendar.HOUR_OF_DAY) >= 12) {
if(cal.get(Calendar.DAY_OF_YEAR) < currentLocalDate.getDayOfYear() || (
cal.get(Calendar.DAY_OF_YEAR) == currentLocalDate.getDayOfYear() &&
(cal.get(Calendar.HOUR_OF_DAY) * 60 + cal.get(Calendar.MINUTE) >= 690))) {
cal.set(Calendar.DAY_OF_YEAR, currentLocalDate.getDayOfYear() + 1);
}
cal.set(Calendar.HOUR_OF_DAY, 12);
cal.set(Calendar.MINUTE, 0);
cal.set(Calendar.HOUR_OF_DAY, 11);
cal.set(Calendar.MINUTE, 30);
cal.set(Calendar.SECOND, 0);
cal.set(Calendar.MILLISECOND, 0);
log.warn("已设置排行榜定时更新, 首次运行时间: {}", cal.getTime());
long delay = cal.getTime().getTime() - System.currentTimeMillis();
log.warn("已设置排行榜定时更新, 首次运行时间: {} ({}min)", cal.getTime(), delay / 1000 / 60);
timer.schedule(new TimerTask() {
@Override
public void run() {
now(null);
}
}, cal.getTime(), 86400000); // 1 Day
}, delay, 86400000); // 1 Day
}
public void now(Date queryDate) {
@ -51,13 +55,14 @@ public class RankingUpdateTimer {
LocalDate currentLocalDate = LocalDate.now();
if(calendar.get(Calendar.DAY_OF_YEAR) == currentLocalDate.getDayOfYear() ||
calendar.get(Calendar.DAY_OF_YEAR) == currentLocalDate.getDayOfYear() - 1) {
if(calendar.get(Calendar.HOUR_OF_DAY) < 12) {
if(calendar.get(Calendar.HOUR_OF_DAY) < 11) {
calendar.add(Calendar.DAY_OF_YEAR, -2);
} else {
calendar.add(Calendar.DAY_OF_YEAR, -1);
}
}
String dateStr = new SimpleDateFormat("yyyy-MM-dd").format(calendar.getTime());
log.info("正在获取 {} 期排行榜数据...", calendar.getTime());
for (PixivURL.RankingMode rankingMode : PixivURL.RankingMode.values()) {
for (PixivURL.RankingContentType contentType : PixivURL.RankingContentType.values()) {
@ -66,7 +71,7 @@ public class RankingUpdateTimer {
}
log.info("当前排行榜类型: {}.{}, 正在更新...", rankingMode.name(), contentType.name());
BotEventHandler.executeMessageEvent(new VirtualLoadMessageEvent(0,0,
".cgj ranking -type=" + contentType.name() + " -mode=" + rankingMode.name()));
".cgj ranking -type=" + contentType.name() + " -mode=" + rankingMode.name() + " -force -date " + dateStr));
log.info("排行榜 {}.{} 负载指令已投递.", rankingMode.name(), contentType.name());
}
}

View File

@ -0,0 +1,246 @@
package net.lamgc.cgj.bot;
import com.google.common.base.Throwables;
import net.lamgc.cgj.bot.boot.BotGlobal;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.*;
import java.nio.charset.StandardCharsets;
import java.util.*;
public final class SettingProperties {
private final static Logger log = LoggerFactory.getLogger(SettingProperties.class);
private final static File globalPropFile = new File(getPropertiesDir(), "global.properties");
private final static Properties globalProp = new Properties();
private final static Map<Long, Properties> groupPropMap = new HashMap<>();
private final static Set<Long> changeList = Collections.synchronizedSet(new HashSet<>());
/**
* 全局配置项
*/
public final static long GLOBAL = 0;
/**
* 清空所有Properties.
*/
public static void clearProperties() {
groupPropMap.clear();
globalProp.clear();
}
/**
* 加载配置文件
*/
public static void loadProperties() {
loadGlobalProperties();
File[] files = getPropertiesDir()
.listFiles((dir, fileName) -> fileName.startsWith("group.") && fileName.endsWith(".properties"));
if(files == null) {
log.error("检索群组配置文件失败, 可能是被拒绝访问.");
return;
}
for (File file : files) {
String name = file.getName();
long groupId;
try {
groupId = Long.parseLong(name.substring(name.indexOf("group.") + 6, name.lastIndexOf(".properties")));
if(groupId <= 0) {
log.warn("无效的群配置文件: {}", groupId);
continue;
}
} catch (NumberFormatException e) {
log.error("非法的配置文件名: {}", name);
continue;
}
if(!groupPropMap.containsKey(groupId)) {
groupPropMap.put(groupId, new Properties(globalProp));
}
loadGroupProperties(groupId, groupPropMap.get(groupId));
}
}
/**
* 保存配置项
*/
public static void saveProperties() {
log.info("正在保存所有配置...");
saveGlobalProperties();
for (Long groupId : groupPropMap.keySet()) {
if(!changeList.contains(groupId)) {
log.debug("群组 {} 配置无改动, 忽略保存.", groupId);
return;
}
log.debug("正在保存群组 {} 配置文件...", groupId);
saveGroupProperties(groupId);
}
log.info("配置保存完成.");
}
/**
* 保存指定群组的配置文件
* @param groupId 要保存配置的群组Id
*/
private static void saveGroupProperties(long groupId) {
try {
saveGroupProperties(groupId, getGroupProperties(groupId));
} catch (IOException e) {
log.error("群组 {} 配置保存失败\n{}", groupId, Throwables.getStackTraceAsString(e));
}
}
private static void saveGroupProperties(Long groupId, Properties properties) throws IOException {
File groupPropFile = new File(getPropertiesDir(), "group." + groupId + ".properties");
if(!groupPropFile.exists() && !groupPropFile.createNewFile()) {
log.error("群组 {} 配置文件创建失败!", groupId);
return;
}
saveProperties(properties, new FileOutputStream(groupPropFile));
}
private static void loadGlobalProperties() {
if(globalPropFile.exists() && globalPropFile.isFile()) {
log.info("正在加载全局配置文件...");
try (Reader reader = new InputStreamReader(new FileInputStream(globalPropFile), StandardCharsets.UTF_8)) {
globalProp.load(reader);
log.info("全局配置文件加载完成.");
} catch (IOException e) {
log.error("加载全局配置文件时发生异常", e);
}
} else {
log.info("未找到全局配置文件,跳过加载.");
}
}
/**
* 保存全局配置项
*/
private static void saveGlobalProperties() {
try {
if(!globalPropFile.exists() && !globalPropFile.createNewFile()) {
log.error("创建全局配置文件失败.");
return;
}
saveProperties(globalProp, new FileOutputStream(globalPropFile));
} catch (IOException e) {
log.error("全局配置文件保存时发生异常", e);
}
}
private static void loadGroupProperties(long groupId, Properties properties) {
File propFile = new File(getPropertiesDir(), "group." + groupId + ".properties");
Properties groupProp = Objects.requireNonNull(properties);
if(!propFile.exists() || !propFile.isFile()) {
log.warn("群组 {} 配置文件不存在, 或不是一个文件.({})", groupId, propFile.getAbsolutePath());
return;
}
try (Reader reader = new InputStreamReader(new FileInputStream(propFile), StandardCharsets.UTF_8)) {
groupProp.load(reader);
} catch (IOException e) {
log.error("读取群组 {} 群配置文件时发生异常:\n{}", groupId, Throwables.getStackTraceAsString(e));
}
}
private static void saveProperties(Properties properties, OutputStream stream) throws IOException {
properties.store(new OutputStreamWriter(stream, StandardCharsets.UTF_8), null);
}
/**
* 获取配置文件目录
* @return 返回目录File对象.
*/
private static File getPropertiesDir() {
File propDir = new File(BotGlobal.getGlobal().getDataStoreDir(), "/setting/");
if(!propDir.exists() && !propDir.mkdirs()) {
log.warn("Setting文件夹创建失败!");
}
return propDir;
}
public static String getProperty(long groupId, String key) {
return getProperty(groupId, key, null);
}
public static String getProperty(long groupId, String key, String defaultValue) {
if(groupId <= 0) {
return globalProp.getProperty(key, defaultValue);
} else {
Properties properties = groupPropMap.get(groupId);
return properties == null ? defaultValue : properties.getProperty(key, defaultValue);
}
}
/**
* 设置配置项
* @param groupId 群组Id, 如为0或负数则为全局配置
* @param key 配置项key名
* @param value 欲设置的新值, 如为null则删除该配置项
* @return 返回上一次设定值
*/
public static String setProperty(long groupId, String key, String value) {
Objects.requireNonNull(key);
Properties targetProperties;
if(groupId <= 0) {
targetProperties = globalProp;
} else {
changeList.add(groupId);
targetProperties = getGroupProperties(groupId);
}
String lastValue = targetProperties.getProperty(key);
if(value != null) {
targetProperties.setProperty(key, value);
} else {
targetProperties.remove(key);
}
return lastValue;
}
/**
* 获取GlobalProperties
* @return 全局Properties
*/
private static Properties getGlobalProperties() {
return globalProp;
}
/**
* 获取群组Properties
* @param groupId 群组Id
* @return 如果存在, 返回Properties, 不存在返回null.
* @throws IllegalArgumentException 当群组Id 小于或等于0 时抛出.
*/
private static Properties getGroupProperties(long groupId) {
if (groupId <= 0) {
throw new IllegalArgumentException("Group number cannot be 0 or negative: " + groupId);
}
if(!groupPropMap.containsKey(groupId)) {
groupPropMap.put(groupId, new Properties(globalProp));
}
return groupPropMap.get(groupId);
}
/**
* 获取群组 Properties, 如果指定群组没有 Properties, 则使用GlobalProperties.
* @param groupId 指定群组Id
* @return 如果群组存在所属Properties, 则返回群组Properties, 否则返回GlobalProperties.
*/
public static Properties getProperties(long groupId) {
if(groupId > 0 && groupPropMap.containsKey(groupId)) {
return groupPropMap.get(groupId);
}
return getGlobalProperties();
}
}

View File

@ -0,0 +1,46 @@
package net.lamgc.cgj.bot.boot;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.util.PropertiesUtils;
import net.lamgc.utils.base.ArgumentsProperties;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public final class ApplicationBoot {
private final static Logger log = LoggerFactory.getLogger(ApplicationBoot.class);
private ApplicationBoot() {}
/**
* 初始化应用.
* <p>该方法不会初始化机器人, 仅初始化应用所需的配置信息.</p>
*/
public static void initialApplication(String[] args) {
ArgumentsProperties argsProp = new ArgumentsProperties(args);
if(!PropertiesUtils.getSettingToSysProp(argsProp, "proxy", null)) {
PropertiesUtils.getEnvSettingToSysProp("CGJ_PROXY", "proxy", null);
}
if(!PropertiesUtils.getSettingToSysProp(argsProp, "botDataDir", "./") &&
!PropertiesUtils.getEnvSettingToSysProp("CGJ_BOT_DATA_DIR", "botDataDir", "./")) {
log.warn("未设置botDataDir, 当前运行目录将作为酷Q机器人所在目录.");
}
if(!PropertiesUtils.getSettingToSysProp(argsProp, "redisAddress", "127.0.0.1") &&
!PropertiesUtils.getEnvSettingToSysProp("CGJ_REDIS_URI", "redisAddress", "127.0.0.1")) {
log.warn("未设置RedisAddress, 将使用默认值连接Redis服务器(127.0.0.1:6379)");
}
// 初始化 BotGlobal
//noinspection ResultOfMethodCallIgnored 这里仅仅是加载BotGlobal而已
BotGlobal.getGlobal();
}
/**
* 初始化机器人.
* <p>本方法由框架调用.</p>
*/
public static void initialBot() {
BotEventHandler.initial();
}
}

View File

@ -0,0 +1,132 @@
package net.lamgc.cgj.bot.boot;
import com.google.common.base.Strings;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import net.lamgc.cgj.pixiv.PixivDownload;
import org.apache.http.HttpHost;
import org.apache.http.client.CookieStore;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.exceptions.JedisConnectionException;
import java.io.File;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URL;
import java.nio.file.Files;
public final class BotGlobal {
private final static Logger log = LoggerFactory.getLogger(BotGlobal.class);
private final static BotGlobal instance = new BotGlobal();
public static BotGlobal getGlobal() {
if(instance == null) {
throw new IllegalStateException("");
}
return instance;
}
private final URI redisUri;
/**
* 所有缓存共用的JedisPool
*/
private final JedisPool redisServer;
private final File dataStoreDir;
private final HttpHost proxy;
private CookieStore cookieStore;
private final Gson gson = new GsonBuilder()
.serializeNulls()
.create();
private PixivDownload pixivDownload;
private final File imageStoreDir;
private BotGlobal() {
this.redisUri = URI.create("redis://" + System.getProperty("cgj.redisAddress"));
this.redisServer = new JedisPool(
getRedisUri().getHost(),
getRedisUri().getPort() == -1 ? 6379 : getRedisUri().getPort());
try (Jedis jedis = this.redisServer.getResource()) {
log.warn("Redis连接状态(Ping): {}", jedis.ping().equalsIgnoreCase("pong"));
} catch(JedisConnectionException e) {
log.warn("Redis连接失败, 将会影响到后续功能运行.", e);
}
String dataStoreDirPath = System.getProperty("cgj.botDataDir");
this.dataStoreDir = new File((!dataStoreDirPath.endsWith("/") || !dataStoreDirPath.endsWith("\\")) ?
dataStoreDirPath + System.getProperty("file.separator") : dataStoreDirPath);
this.imageStoreDir = new File(getDataStoreDir(), "data/image/cgj/");
String proxyAddress = System.getProperty("cgj.proxy");
HttpHost temp = null;
if(!Strings.isNullOrEmpty(proxyAddress)) {
try {
URL proxyUrl = new URL(proxyAddress);
temp = new HttpHost(proxyUrl.getHost(), proxyUrl.getPort());
log.info("已启用代理:{}", temp.toHostString());
} catch (MalformedURLException e) {
log.error("Proxy地址解析失败, 代理将不会启用.", e);
}
}
this.proxy = temp;
}
public URI getRedisUri() {
return redisUri;
}
public File getDataStoreDir() {
if(!dataStoreDir.exists() && !dataStoreDir.mkdirs()) {
log.error("DataStoreDir 创建失败, 数据存储可能失效!");
}
return dataStoreDir;
}
public JedisPool getRedisServer() {
return redisServer;
}
public HttpHost getProxy() {
return proxy;
}
public void setCookieStore(CookieStore cookieStore) {
if(this.cookieStore != null) {
throw new IllegalStateException("CookieStore set");
}
this.cookieStore = cookieStore;
this.pixivDownload =
new PixivDownload(cookieStore, proxy);
}
public Gson getGson() {
return gson;
}
public PixivDownload getPixivDownload() {
return pixivDownload;
}
public File getImageStoreDir() {
if(!imageStoreDir.exists() && !Files.isSymbolicLink(imageStoreDir.toPath())) {
if(!imageStoreDir.mkdirs()) {
log.warn("酷Q图片缓存目录失效(Path: {} )", imageStoreDir.getAbsolutePath());
throw new RuntimeException(new IOException("文件夹创建失败!"));
}
}
return imageStoreDir;
}
}

View File

@ -0,0 +1,52 @@
package net.lamgc.cgj.bot.cache;
import com.google.common.base.Throwables;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Set;
import java.util.Timer;
import java.util.TimerTask;
import java.util.concurrent.CopyOnWriteArraySet;
public class AutoCleanTimer extends TimerTask {
private final static Set<Cleanable> cleanSet = new CopyOnWriteArraySet<>();
private final static Timer cleanTimer = new Timer("Thread-AutoClean", true);
private final static Logger log = LoggerFactory.getLogger(AutoCleanTimer.class);
static {
cleanTimer.schedule(new AutoCleanTimer(), 100L);
}
/**
* 增加需要定时执行清理的缓存库
* @param store 已实现Cleanable的对象
*/
public static void add(Cleanable store) {
cleanSet.add(store);
}
/**
* 移除已添加的缓存库
* @param store 需要从AutoCleanTimer移除的对象
*/
public static void remove(Cleanable store) {
cleanSet.remove(store);
}
private AutoCleanTimer() {}
@Override
public void run() {
cleanSet.forEach(cleanable -> {
try {
cleanable.clean();
} catch (Exception e) {
log.error("{} 执行清理动作时发生异常:\n{}", cleanable.toString(), Throwables.getStackTraceAsString(e));
}
});
}
}

View File

@ -13,7 +13,7 @@ public interface CacheStore<T> {
* 更新或添加缓存项
* @param key 缓存键名
* @param value 缓存值
* @param expire 有效期, 单位为ms(毫秒), 如不过期传入0或赋值
* @param expire 有效期, 单位为ms(毫秒), 如不过期传入0或负数
*/
void update(String key, T value, long expire);

View File

@ -0,0 +1,632 @@
package net.lamgc.cgj.bot.cache;
import com.google.common.base.Strings;
import com.google.common.base.Throwables;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.BotCode;
import net.lamgc.cgj.bot.BotCommandProcess;
import net.lamgc.cgj.bot.SettingProperties;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivSearchBuilder;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.cgj.util.URLs;
import net.lamgc.utils.encrypt.MessageDigestUtils;
import net.lz1998.cq.utils.CQCode;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.util.EntityUtils;
import org.apache.tomcat.util.http.fileupload.IOUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.*;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.concurrent.atomic.AtomicInteger;
@SuppressWarnings("SynchronizationOnLocalVariableOrMethodParameter")
public final class CacheStoreCentral {
private CacheStoreCentral() {}
private final static Logger log = LoggerFactory.getLogger(CacheStoreCentral.class);
private final static Hashtable<String, File> imageCache = new Hashtable<>();
private final static JsonRedisCacheStore imageChecksumCache =
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(),
"imageChecksum", BotGlobal.getGlobal().getGson());
/**
* 作品信息缓存 - 不过期
*/
private final static CacheStore<JsonElement> illustInfoCache =
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(),
"illustInfo", BotGlobal.getGlobal().getGson());
/**
* 作品信息预加载数据 - 有效期 2 小时, 本地缓存有效期1 ± 0.25
*/
private final static CacheStore<JsonElement> illustPreLoadDataCache =
CacheStoreUtils.hashLocalHotDataStore(
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(),
"illustPreLoadData", BotGlobal.getGlobal().getGson()),
3600000, 900000);
/**
* 搜索内容缓存, 有效期 2 小时
*/
private final static CacheStore<JsonElement> searchBodyCache =
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(),
"searchBody", BotGlobal.getGlobal().getGson());
/**
* 排行榜缓存, 不过期
*/
private final static CacheStore<List<JsonObject>> rankingCache =
new JsonObjectRedisListCacheStore(BotGlobal.getGlobal().getRedisServer(),
"ranking", BotGlobal.getGlobal().getGson());
/**
* 作品页面下载链接缓存 - 不过期
*/
private final static CacheStore<List<String>> pagesCache =
new StringListRedisCacheStore(BotGlobal.getGlobal().getRedisServer(), "imagePages");
/**
* 清空所有缓存
*/
public static void clearCache() {
imageCache.clear();
illustInfoCache.clear();
illustPreLoadDataCache.clear();
searchBodyCache.clear();
rankingCache.clear();
pagesCache.clear();
}
/**
* 通过illustId获取作品图片
* @param fromGroup 来源群(系统提供)
* @param illustId 作品Id
* @param quality 图片质量
* @param pageIndex 指定页面索引, 从1开始
* @return 如果成功, 返回BotCode, 否则返回错误信息.
*/
public static String getImageById(long fromGroup, int illustId, PixivDownload.PageQuality quality, int pageIndex) {
log.debug("IllustId: {}, Quality: {}, PageIndex: {}", illustId, quality.name(), pageIndex);
if(pageIndex <= 0) {
log.warn("指定的页数不能小于或等于0: {}", pageIndex);
return "指定的页数不能小于或等于0";
}
try {
if (BotCommandProcess.isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) {
log.warn("作品 {} 存在R-18内容且设置\"image.allowR18\"为false将屏蔽该作品不发送.", illustId);
return "(根据设置,该作品已被屏蔽!)";
} else if(BotCommandProcess.isReported(illustId)) {
log.warn("作品Id {} 被报告, 正在等待审核, 跳过该作品.", illustId);
return "(该作品已被封印)";
}
} catch (IOException e) {
log.warn("作品信息无法获取!", e);
return "(发生网络异常,无法获取图片!)";
}
List<String> pagesList;
try {
pagesList = CacheStoreCentral.getIllustPages(illustId, quality, false);
} catch (IOException e) {
log.error("获取下载链接列表时发生异常", e);
return "发生网络异常,无法获取图片!";
}
if(log.isDebugEnabled()) {
StringBuilder logBuilder = new StringBuilder("作品Id " + illustId + " 所有页面下载链接: \n");
AtomicInteger index = new AtomicInteger();
pagesList.forEach(item ->
logBuilder.append(index.incrementAndGet()).append(". ").append(item).append("\n"));
log.debug(logBuilder.toString());
}
if (pagesList.size() < pageIndex) {
log.warn("指定的页数超出了总页数({} / {})", pageIndex, pagesList.size());
return "指定的页数超出了范围(总共 " + pagesList.size() + " 页)";
}
String downloadLink = pagesList.get(pageIndex - 1);
String fileName = URLs.getResourceName(Strings.nullToEmpty(downloadLink));
File imageFile = new File(BotGlobal.getGlobal().getImageStoreDir(),
downloadLink.substring(downloadLink.lastIndexOf("/") + 1));
log.debug("FileName: {}, DownloadLink: {}", fileName, downloadLink);
if(!imageCache.containsKey(fileName)) {
if(imageFile.exists() && imageFile.isFile()) {
ImageChecksum imageChecksum = getImageChecksum(illustId, pageIndex);
if(imageChecksum != null) {
try {
log.debug("正在检查作品Id {} 第 {} 页图片文件 {} ...", illustId, pageIndex, imageFile.getName());
if (ImageChecksum.checkFile(imageChecksum, Files.readAllBytes(imageFile.toPath()))) {
imageCache.put(URLs.getResourceName(downloadLink), imageFile);
log.debug("作品Id {} 第 {} 页缓存已补充.", illustId, pageIndex);
return getImageToBotCode(imageFile, false).toString();
} else {
log.warn("图片文件 {} 校验失败, 重新下载图片...", imageFile.getName());
}
} catch(IOException e) {
log.error("文件检验时读取失败, 重新下载文件...(file: {})", imageFile.getPath());
}
} else {
log.warn("图片存在但校验不存在, 重新下载图片...");
}
}
try {
Throwable throwable = ImageCacheStore.executeCacheRequest(
new ImageCacheObject(imageCache, illustId, pageIndex, downloadLink, imageFile));
if(throwable != null) {
throw throwable;
}
} catch (InterruptedException e) {
log.warn("图片缓存被中断", e);
return "(错误:图片获取超时)";
} catch (Throwable e) {
log.error("图片 {} 获取失败:\n{}", illustId + "p" + pageIndex, Throwables.getStackTraceAsString(e));
return "(错误: 图片获取出错)";
}
} else {
log.debug("图片 {} 缓存命中.", fileName);
}
return getImageToBotCode(imageCache.get(fileName), false).toString();
}
/**
* 通过文件获取图片的BotCode代码
* @param targetFile 图片文件
* @param updateCache 是否刷新缓存(只是让机器人重新上传, 如果上传接口有重复检测的话是无法处理的)
* @return 返回设定好参数的BotCode
*/
@SuppressWarnings("SameParameterValue")
private static BotCode getImageToBotCode(File targetFile, boolean updateCache) {
String fileName = Objects.requireNonNull(targetFile, "targetFile is null").getName();
BotCode code = BotCode.parse(
CQCode.image(BotGlobal.getGlobal().getImageStoreDir().getName() + "/" + fileName));
code.addParameter("absolutePath", targetFile.getAbsolutePath());
code.addParameter("imageName", fileName.substring(0, fileName.lastIndexOf(".")));
code.addParameter("updateCache", updateCache ? "true" : "false");
return code;
}
/**
* 获取作品信息
* @param illustId 作品Id
* @param flushCache 强制刷新缓存
* @return 返回作品信息
* @throws IOException 当Http请求发生异常时抛出
* @throws NoSuchElementException 当作品未找到时抛出
*/
public static JsonObject getIllustInfo(int illustId, boolean flushCache)
throws IOException, NoSuchElementException {
String illustIdStr = buildSyncKey(Integer.toString(illustId));
JsonObject illustInfoObj = null;
if (!illustInfoCache.exists(illustIdStr) || flushCache) {
synchronized (illustIdStr) {
if (!illustInfoCache.exists(illustIdStr) || flushCache) {
illustInfoObj = BotGlobal.getGlobal().getPixivDownload().getIllustInfoByIllustId(illustId);
illustInfoCache.update(illustIdStr, illustInfoObj, null);
}
}
}
if(Objects.isNull(illustInfoObj)) {
illustInfoObj = illustInfoCache.getCache(illustIdStr).getAsJsonObject();
log.debug("作品Id {} IllustInfo缓存命中.", illustId);
}
return illustInfoObj;
}
/**
* 获取作品预加载数据.
* 可以获取作品的一些与用户相关的信息
* @param illustId 作品Id
* @param flushCache 是否刷新缓存
* @return 成功返回JsonObject对象
* @throws IOException 当Http请求处理发生异常时抛出
*/
public static JsonObject getIllustPreLoadData(int illustId, boolean flushCache) throws IOException {
String illustIdStr = buildSyncKey(Integer.toString(illustId));
JsonObject result = null;
if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) {
synchronized (illustIdStr) {
if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) {
log.debug("IllustId {} 缓存失效, 正在更新...", illustId);
JsonObject preLoadDataObj = BotGlobal.getGlobal().getPixivDownload()
.getIllustPreLoadDataById(illustId)
.getAsJsonObject("illust")
.getAsJsonObject(Integer.toString(illustId));
long expire = 7200 * 1000;
String propValue = SettingProperties.
getProperty(SettingProperties.GLOBAL, "cache.illustPreLoadData.expire", "7200000");
log.debug("PreLoadData有效时间设定: {}", propValue);
try {
expire = Long.parseLong(propValue);
} catch (Exception e) {
log.warn("全局配置项 \"{}\" 值非法, 已使用默认值: {}", propValue, expire);
}
result = preLoadDataObj;
illustPreLoadDataCache.update(illustIdStr, preLoadDataObj, expire);
log.debug("作品Id {} preLoadData缓存已更新(有效时间: {})", illustId, expire);
}
}
}
if(Objects.isNull(result)) {
result = illustPreLoadDataCache.getCache(illustIdStr).getAsJsonObject();
log.debug("作品Id {} PreLoadData缓存命中.", illustId);
}
return result;
}
public static List<String> getIllustPages(int illustId, PixivDownload.PageQuality quality, boolean flushCache)
throws IOException {
String pagesSign = buildSyncKey(Integer.toString(illustId), ".", quality.name());
List<String> result = null;
if (!pagesCache.exists(pagesSign) || flushCache) {
synchronized (pagesSign) {
if (!pagesCache.exists(pagesSign) || flushCache) {
List<String> linkList = PixivDownload
.getIllustAllPageDownload(BotGlobal.getGlobal().getPixivDownload().getHttpClient(),
BotGlobal.getGlobal().getPixivDownload().getCookieStore(), illustId, quality);
result = linkList;
pagesCache.update(pagesSign, linkList, null);
}
}
}
if(Objects.isNull(result)) {
result = pagesCache.getCache(pagesSign);
log.debug("作品Id {} Pages缓存命中.", illustId);
}
return result;
}
private final static Random expireTimeFloatRandom = new Random();
/**
* 获取排行榜
* @param contentType 排行榜类型
* @param mode 排行榜模式
* @param queryDate 查询时间
* @param start 开始排名, 从1开始
* @param range 取范围
* @param flushCache 是否强制刷新缓存
* @return 成功返回有值List, 失败且无异常返回空
* @throws IOException 获取异常时抛出
*/
public static List<JsonObject> getRankingInfoByCache(PixivURL.RankingContentType contentType,
PixivURL.RankingMode mode,
Date queryDate, int start, int range, boolean flushCache)
throws IOException {
if(!contentType.isSupportedMode(mode)) {
log.warn("试图获取不支持的排行榜类型已拒绝.(ContentType: {}, RankingMode: {})", contentType.name(), mode.name());
if(log.isDebugEnabled()) {
try {
Thread.dumpStack();
} catch(Exception e) {
log.debug("本次非法请求的堆栈信息如下: \n{}", Throwables.getStackTraceAsString(e));
}
}
return new ArrayList<>(0);
}
String date = new SimpleDateFormat("yyyyMMdd").format(queryDate);
String requestSign = buildSyncKey(contentType.name(), ".", mode.name(), ".", date);
List<JsonObject> result = null;
if(!rankingCache.exists(requestSign) || flushCache) {
synchronized(requestSign) {
if(!rankingCache.exists(requestSign) || flushCache) {
log.debug("Ranking缓存失效, 正在更新...(RequestSign: {})", requestSign);
List<JsonObject> rankingResult = BotGlobal.getGlobal().getPixivDownload()
.getRanking(contentType, mode, queryDate, 1, 500);
long expireTime = 0;
if(rankingResult.size() == 0) {
expireTime = 5400000 + expireTimeFloatRandom.nextInt(1800000);
log.warn("数据获取失败, 将设置浮动有效时间以准备下次更新. (ExpireTime: {}ms)", expireTime);
}
result = new ArrayList<>(rankingResult).subList(start - 1, start + range - 1);
rankingCache.update(requestSign, rankingResult, expireTime);
log.debug("Ranking缓存更新完成.(RequestSign: {})", requestSign);
}
}
}
if (Objects.isNull(result)) {
result = rankingCache.getCache(requestSign, start - 1, range);
log.debug("RequestSign [{}] 缓存命中.", requestSign);
}
log.debug("Result-Length: {}", result.size());
return PixivDownload.getRanking(result, start - 1, range);
}
/**
* 获取搜索结果
* @param content 搜索内容
* @param type 类型
* @param area 范围
* @param includeKeywords 包含关键词
* @param excludeKeywords 排除关键词
* @param contentOption 内容类型
* @return 返回完整搜索结果
* @throws IOException 当请求发生异常, 或接口返回异常信息时抛出.
*/
public static JsonObject getSearchBody(
String content,
String type,
String area,
String includeKeywords,
String excludeKeywords,
String contentOption) throws IOException {
PixivSearchBuilder searchBuilder = new PixivSearchBuilder(Strings.isNullOrEmpty(content) ? "" : content);
if (type != null) {
try {
searchBuilder.setSearchType(PixivSearchBuilder.SearchType.valueOf(type.toUpperCase()));
} catch (IllegalArgumentException e) {
log.warn("不支持的SearchType: {}", type);
}
}
if (area != null) {
try {
searchBuilder.setSearchArea(PixivSearchBuilder.SearchArea.valueOf(area));
} catch (IllegalArgumentException e) {
log.warn("不支持的SearchArea: {}", area);
}
}
if (contentOption != null) {
try {
searchBuilder.setSearchContentOption(PixivSearchBuilder.SearchContentOption.valueOf(contentOption));
} catch (IllegalArgumentException e) {
log.warn("不支持的SearchContentOption: {}", contentOption);
}
}
if (!Strings.isNullOrEmpty(includeKeywords)) {
for (String keyword : includeKeywords.split(";")) {
searchBuilder.removeExcludeKeyword(keyword.trim());
searchBuilder.addIncludeKeyword(keyword.trim());
log.debug("已添加关键字: {}", keyword);
}
}
if (!Strings.isNullOrEmpty(excludeKeywords)) {
for (String keyword : excludeKeywords.split(";")) {
searchBuilder.removeIncludeKeyword(keyword.trim());
searchBuilder.addExcludeKeyword(keyword.trim());
log.debug("已添加排除关键字: {}", keyword);
}
}
log.info("正在搜索作品, 条件: {}", searchBuilder.getSearchCondition());
String requestUrl = searchBuilder.buildURL().intern();
log.debug("RequestUrl: {}", requestUrl);
JsonObject resultBody = null;
if(!searchBodyCache.exists(requestUrl)) {
synchronized (requestUrl) {
if (!searchBodyCache.exists(requestUrl)) {
log.debug("searchBody缓存失效, 正在更新...");
JsonObject jsonObject;
HttpGet httpGetRequest = BotGlobal.getGlobal().getPixivDownload().
createHttpGetRequest(requestUrl);
HttpResponse response = BotGlobal.getGlobal().getPixivDownload().
getHttpClient().execute(httpGetRequest);
String responseBody = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
log.debug("ResponseBody: {}", responseBody);
jsonObject = BotGlobal.getGlobal().getGson().fromJson(responseBody, JsonObject.class);
if (jsonObject.get("error").getAsBoolean()) {
log.error("接口请求错误, 错误信息: {}", jsonObject.get("message").getAsString());
throw new IOException("Interface Request Error: " + jsonObject.get("message").getAsString());
}
long expire = 7200 * 1000;
String propValue = SettingProperties
.getProperty(SettingProperties.GLOBAL, "cache.searchBody.expire", "7200000");
try {
expire = Long.parseLong(propValue);
} catch (Exception e) {
log.warn("全局配置项 \"{}\" 值非法, 已使用默认值: {}", propValue, expire);
}
resultBody = jsonObject.getAsJsonObject().getAsJsonObject("body");
searchBodyCache.update(requestUrl, jsonObject, expire);
log.debug("searchBody缓存已更新(有效时间: {})", expire);
} else {
log.debug("搜索缓存命中.");
}
}
} else {
log.debug("搜索缓存命中.");
}
if(Objects.isNull(resultBody)) {
resultBody = searchBodyCache.getCache(requestUrl).getAsJsonObject().getAsJsonObject("body");
}
return resultBody;
}
protected static ImageChecksum getImageChecksum(int illustId, int pageIndex) {
String cacheKey = illustId + ":" + pageIndex;
if(!imageChecksumCache.exists(cacheKey)) {
return null;
} else {
return ImageChecksum.fromJsonObject(imageChecksumCache.getCache(cacheKey).getAsJsonObject());
}
}
protected static void setImageChecksum(ImageChecksum checksum) {
String cacheKey = checksum.getIllustId() + ":" + checksum.getPage();
imageChecksumCache.update(cacheKey, ImageChecksum.toJsonObject(checksum), 0);
}
/**
* 合并String并存取到常量池, 以保证对象一致
* @param keys String对象
* @return 合并后, 如果常量池存在合并后的结果, 则返回常量池中的对象, 否则存入常量池后返回.
*/
private static String buildSyncKey(String... keys) {
StringBuilder sb = new StringBuilder();
for (String string : keys) {
sb.append(string);
}
return sb.toString().intern();
}
/**
* 图片检验信息
*/
public static class ImageChecksum implements Serializable {
private final static MessageDigestUtils.Algorithm ALGORITHM = MessageDigestUtils.Algorithm.SHA256;
private ImageChecksum() {}
private int illustId;
private int page;
private String fileName;
private long size;
private byte[] checksum;
public long getSize() {
return size;
}
public void setSize(long size) {
this.size = size;
}
public byte[] getChecksum() {
return checksum;
}
public void setChecksum(byte[] checksum) {
this.checksum = checksum;
}
public int getIllustId() {
return illustId;
}
public void setIllustId(int illustId) {
this.illustId = illustId;
}
public String getFileName() {
return fileName;
}
public void setFileName(String fileName) {
this.fileName = fileName;
}
public int getPage() {
return page;
}
public void setPage(int page) {
this.page = page;
}
public static ImageChecksum buildImageChecksumFromStream(
int illustId, int pageIndex,
String fileName, InputStream imageStream) throws IOException {
ImageChecksum checksum = new ImageChecksum();
checksum.setIllustId(illustId);
checksum.setPage(pageIndex);
checksum.setFileName(fileName);
ByteArrayOutputStream bufferStream = new ByteArrayOutputStream();
checksum.setSize(IOUtils.copyLarge(imageStream, bufferStream));
checksum.setChecksum(
MessageDigestUtils.encrypt(bufferStream.toByteArray(), ALGORITHM));
return checksum;
}
/**
* 将图片检验信息转换成JsonObject
* @param checksum 检验信息对象
* @return 转换后的JsonObject对象
*/
public static JsonObject toJsonObject(ImageChecksum checksum) {
JsonObject result = new JsonObject();
result.addProperty("illustId", checksum.getIllustId());
result.addProperty("page", checksum.getPage());
result.addProperty("fileName", checksum.getFileName());
result.addProperty("size", checksum.getSize());
result.addProperty("checksum", Base64.getEncoder().encodeToString(checksum.getChecksum()));
return result;
}
/**
* 从JsonObject转换到图片检验信息
* @param checksumObject JsonObject对象
* @return 转换后的图片检验信息对象
*/
public static ImageChecksum fromJsonObject(JsonObject checksumObject) {
ImageChecksum checksum = new ImageChecksum();
checksum.setIllustId(checksumObject.get("illustId").getAsInt());
checksum.setPage(checksumObject.get("page").getAsInt());
checksum.setFileName(checksumObject.get("fileName").getAsString());
checksum.setSize(checksumObject.get("size").getAsLong());
checksum.setChecksum(Base64.getDecoder().decode(checksumObject.get("checksum").getAsString()));
return checksum;
}
/**
* 比对图片文件是否完整.
* @param checksum 图片检验信息
* @param imageData 图片数据
* @return 如果检验成功, 则返回true
*/
public static boolean checkFile(ImageChecksum checksum, byte[] imageData) {
byte[] sha256Checksum = MessageDigestUtils.encrypt(imageData, ALGORITHM);
return checksum.getSize() == imageData.length &&
Arrays.equals(checksum.getChecksum(), sha256Checksum);
}
@Override
public String toString() {
return "ImageChecksum{" +
"illustId=" + illustId +
", page=" + page +
", fileName='" + fileName + '\'' +
", size=" + size +
", checksum=" + Base64.getEncoder().encodeToString(getChecksum()) +
'}';
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
ImageChecksum checksum1 = (ImageChecksum) o;
return illustId == checksum1.illustId &&
page == checksum1.page &&
size == checksum1.size &&
Objects.equals(fileName, checksum1.fileName) &&
Arrays.equals(checksum, checksum1.checksum);
}
@Override
public int hashCode() {
int result = Objects.hash(illustId, page, fileName, size);
result = 31 * result + Arrays.hashCode(checksum);
return result;
}
}
}

View File

@ -0,0 +1,35 @@
package net.lamgc.cgj.bot.cache;
public final class CacheStoreUtils {
private CacheStoreUtils() {}
/**
* 将 CacheStore 转换成 基于 {@link LocalHashCacheStore} 的 {@link HotDataCacheStore}
* <p>通过该方法转换, 会自动启用 自动清理</p>
* @param cacheStore 上游缓存库
* @param expireTime 热点缓存最小有效期
* @param floatRange 缓存浮动最大范围
* @param <T> 缓存库数据类型
* @return 返回 {@link HotDataCacheStore}
*/
public static <T> CacheStore<T> hashLocalHotDataStore(CacheStore<T> cacheStore, long expireTime, int floatRange) {
return hashLocalHotDataStore(cacheStore, expireTime, floatRange, true);
}
/**
* 将 CacheStore 转换成 基于 {@link LocalHashCacheStore} 的 {@link HotDataCacheStore}
* @param cacheStore 上游缓存库
* @param expireTime 热点缓存最小有效期
* @param floatRange 缓存浮动最大范围
* @param autoClean 是否启用自动清理
* @param <T> 缓存库数据类型
* @return 返回 {@link HotDataCacheStore}
*/
public static <T> CacheStore<T> hashLocalHotDataStore(CacheStore<T> cacheStore,
long expireTime, int floatRange, boolean autoClean) {
return new HotDataCacheStore<>(cacheStore, new LocalHashCacheStore<>(), expireTime, floatRange, autoClean);
}
}

View File

@ -0,0 +1,13 @@
package net.lamgc.cgj.bot.cache;
/**
* 可清理接口, 实现该接口代表该类具有清理动作.
*/
public interface Cleanable {
/**
* 该方法需要CacheStore完成对过期Entry的清除.
*/
void clean() throws Exception;
}

View File

@ -10,31 +10,38 @@ import java.util.*;
* @param <T> 存储类型
* @author LamGC
*/
public class HotDataCacheStore<T> implements CacheStore<T> {
public class HotDataCacheStore<T> implements CacheStore<T>, Cleanable {
private final CacheStore<T> parent;
private final CacheStore<T> current;
private final long expireTime;
private final int expireFloatRange;
private final Random random = new Random();
private final Logger log = LoggerFactory.getLogger(HotDataCacheStore.class.getSimpleName() + "@" + Integer.toHexString(this.hashCode()));
private final Logger log = LoggerFactory.getLogger(this.toString());
/**
* 构造热点缓存存储对象
* @param parent 上级缓存存储库
* @param current 热点缓存存储库, 最好使用本地缓存(例如 {@linkplain LocalHashCacheStore LocalHashCacheStore})
* @param expireTime 本地缓存库的缓存项过期时间, 单位毫秒;
* 该时间并不是所有缓存项的最终过期时间, 还需要根据expireFloatRange的设定随机设置, 公式:
* {@code expireTime + new Random().nextInt(expireFloatRange)}
* @param expireFloatRange 过期时间的浮动范围(单位毫秒), 用于防止短时间内大量缓存项失效导致的缓存雪崩
* 该时间并不是所有缓存项的最终过期时间, 还需要根据expireFloatRange的设定随机设置, 公式:
* {@code expireTime + new Random().nextInt(expireFloatRange)}
* @param expireFloatRange 过期时间的浮动范围(单位毫秒), 用于防止短时间内大量缓存项失效导致的缓存雪崩,
* 如设置为0或负数, 则不启用浮动范围.
* @param autoClean 是否交由{@link AutoCleanTimer}自动执行清理, 启用后, AutoCleanTimer会自动检查过期Key并进行删除.
*/
public HotDataCacheStore(CacheStore<T> parent, CacheStore<T> current, long expireTime, int expireFloatRange) {
public HotDataCacheStore(CacheStore<T> parent, CacheStore<T> current, long expireTime, int expireFloatRange, boolean autoClean) {
this.parent = parent;
this.current = current;
this.expireTime = expireTime;
this.expireFloatRange = expireFloatRange;
log.debug("HotDataCacheStore初始化完成. (Parent: {}, Current: {}, expireTime: {}, expireFloatRange: {})",
parent, current, expireTime, expireFloatRange);
if(autoClean) {
AutoCleanTimer.add(this);
}
log.trace("HotDataCacheStore初始化完成. " +
"(Parent: {}, Current: {}, expireTime: {}, expireFloatRange: {}, autoClean: {})",
parent, current, expireTime, expireFloatRange, autoClean);
}
@Override
@ -51,23 +58,24 @@ public class HotDataCacheStore<T> implements CacheStore<T> {
@Override
public T getCache(String key) {
if(!exists(key)) {
log.debug("查询缓存键名不存在, 直接返回null.");
log.trace("查询缓存键名不存在, 直接返回null.");
return null;
}
T result = current.getCache(key);
if(Objects.isNull(result)) {
log.debug("Current缓存库未命中, 查询Parent缓存库");
log.trace("Current缓存库未命中, 查询Parent缓存库");
T parentResult = parent.getCache(key);
if(Objects.isNull(parentResult)) {
log.debug("Parent缓存库未命中, 缓存不存在");
log.trace("Parent缓存库未命中, 缓存不存在");
return null;
}
log.debug("Parent缓存命中, 正在更新Current缓存库...");
current.update(key, parentResult, expireTime + random.nextInt(expireFloatRange));
log.debug("Current缓存库更新完成.");
log.trace("Parent缓存命中, 正在更新Current缓存库...");
current.update(key, parentResult,
expireTime + (expireFloatRange <= 0 ? 0 : random.nextInt(expireFloatRange)));
log.trace("Current缓存库更新完成.");
result = parentResult;
} else {
log.debug("Current缓存库缓存命中.");
log.trace("Current缓存库缓存命中.");
}
return result;
}
@ -114,11 +122,27 @@ public class HotDataCacheStore<T> implements CacheStore<T> {
@Override
public boolean supportedPersistence() {
// 由于Current的缓存数据会更新到Parent上,
// 所以只要任意一边支持持久化, 那么该缓存库就支持持久化
return current.supportedPersistence() || parent.supportedPersistence();
}
@Override
public boolean supportedList() {
return false;
// 只有两边都支持List, 该缓存库才会支持持久化
return current.supportedList() && parent.supportedList();
}
/**
* 检查并清理已过期的Entry.
* <p>该方法仅清理Current缓存库, 不会对上游缓存库造成影响.</p>
*/
@Override
public void clean() {
for(String key : this.current.keys()) {
if(current.exists(key)) {
current.remove(key);
}
}
}
}

View File

@ -1,6 +1,7 @@
package net.lamgc.cgj.bot.cache;
import net.lamgc.cgj.Main;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.cache.exception.HttpRequestException;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.cgj.util.URLs;
import net.lamgc.utils.event.EventHandler;
@ -12,25 +13,25 @@ import org.apache.http.impl.client.HttpClientBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.*;
import java.util.Collections;
import java.util.HashSet;
import java.util.Set;
public class ImageCacheHandler implements EventHandler {
private final static Logger log = LoggerFactory.getLogger("ImageCacheHandler");
private final static Logger log = LoggerFactory.getLogger(ImageCacheHandler.class);
private final static HttpClient httpClient = HttpClientBuilder.create().setProxy(Main.proxy).build();
private final static HttpClient httpClient = HttpClientBuilder.create()
.setProxy(BotGlobal.getGlobal().getProxy())
.build();
private final static Set<ImageCacheObject> cacheQueue = Collections.synchronizedSet(new HashSet<>());
@SuppressWarnings("unused")
public void getImageToCache(ImageCacheObject event) {
public void getImageToCache(ImageCacheObject event) throws Exception {
if(cacheQueue.contains(event)) {
log.debug("图片 {} 已存在相同缓存任务, 跳过.", event.getStoreFile().getName());
log.warn("图片 {} 已存在相同缓存任务, 跳过.", event.getStoreFile().getName());
return;
} else {
cacheQueue.add(event);
@ -43,11 +44,11 @@ public class ImageCacheHandler implements EventHandler {
try {
if(!storeFile.exists() && !storeFile.createNewFile()) {
log.error("无法创建文件(Path: {})", storeFile.getAbsolutePath());
return;
throw new IOException("Failed to create file");
}
} catch (IOException e) {
log.error("无法创建文件(Path: {})", storeFile.getAbsolutePath());
e.printStackTrace();
throw e;
}
HttpGet request = new HttpGet(event.getDownloadLink());
@ -57,19 +58,32 @@ public class ImageCacheHandler implements EventHandler {
response = httpClient.execute(request);
} catch (IOException e) {
log.error("Http请求时发生异常", e);
return;
throw e;
}
if(response.getStatusLine().getStatusCode() != 200) {
log.warn("Http请求异常{}", response.getStatusLine());
return;
HttpRequestException requestException = new HttpRequestException(response);
log.warn("Http请求异常{}", requestException.getStatusLine());
throw requestException;
}
log.debug("正在下载...(Content-Length: {}KB)", response.getEntity().getContentLength() / 1024);
try(FileOutputStream fos = new FileOutputStream(storeFile)) {
IOUtils.copy(response.getEntity().getContent(), fos);
ByteArrayOutputStream bufferOutputStream = new ByteArrayOutputStream();
try(FileOutputStream fileOutputStream = new FileOutputStream(storeFile)) {
IOUtils.copy(response.getEntity().getContent(), bufferOutputStream);
ByteArrayInputStream bufferInputStream = new ByteArrayInputStream(bufferOutputStream.toByteArray());
CacheStoreCentral.ImageChecksum imageChecksum = CacheStoreCentral.ImageChecksum
.buildImageChecksumFromStream(
event.getIllustId(),
event.getPageIndex(),
event.getStoreFile().getName(),
bufferInputStream
);
bufferInputStream.reset();
IOUtils.copy(bufferInputStream, fileOutputStream);
CacheStoreCentral.setImageChecksum(imageChecksum);
} catch (IOException e) {
log.error("下载图片时发生异常", e);
return;
throw e;
}
event.getImageCache().put(URLs.getResourceName(event.getDownloadLink()), storeFile);
} finally {

View File

@ -12,13 +12,16 @@ public class ImageCacheObject implements EventObject {
private final int illustId;
private final int pageIndex;
private final String downloadLink;
private final File storeFile;
public ImageCacheObject(Map<String, File> imageCache, int illustId, String downloadLink, File storeFile) {
public ImageCacheObject(Map<String, File> imageCache, int illustId, int pageIndex, String downloadLink, File storeFile) {
this.imageCache = imageCache;
this.illustId = illustId;
this.pageIndex = pageIndex;
this.downloadLink = downloadLink;
this.storeFile = storeFile;
}
@ -39,12 +42,17 @@ public class ImageCacheObject implements EventObject {
return illustId;
}
public int getPageIndex() {
return pageIndex;
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
ImageCacheObject that = (ImageCacheObject) o;
return illustId == that.illustId &&
pageIndex == that.pageIndex &&
Objects.equals(imageCache, that.imageCache) &&
Objects.equals(downloadLink, that.downloadLink) &&
Objects.equals(storeFile, that.storeFile);
@ -52,13 +60,15 @@ public class ImageCacheObject implements EventObject {
@Override
public int hashCode() {
return Objects.hash(imageCache, illustId, downloadLink, storeFile);
return Objects.hash(imageCache, illustId, pageIndex, downloadLink, storeFile);
}
@Override
public String toString() {
return "ImageCacheObject@" + Integer.toHexString(hashCode()) + "{" +
"illustId=" + illustId +
return "ImageCacheObject{" +
"imageCache=" + imageCache +
", illustId=" + illustId +
", pageIndex=" + pageIndex +
", downloadLink='" + downloadLink + '\'' +
", storeFile=" + storeFile +
'}';

View File

@ -0,0 +1,136 @@
package net.lamgc.cgj.bot.cache;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import net.lamgc.cgj.bot.cache.exception.HttpRequestException;
import java.util.Hashtable;
import java.util.Map;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicReference;
import java.util.concurrent.locks.ReentrantLock;
public final class ImageCacheStore {
private final static Logger log = LoggerFactory.getLogger(ImageCacheStore.class);
private final static Map<ImageCacheObject, Task> cacheMap = new Hashtable<>();
private final static ThreadPoolExecutor imageCacheExecutor = new ThreadPoolExecutor(
4, 6,
30L,
TimeUnit.SECONDS,
new LinkedBlockingQueue<>(100),
new ThreadFactoryBuilder()
.setNameFormat("ImageCacheThread-%d")
.build()
);
private final static ImageCacheHandler handler = new ImageCacheHandler();
static {
Thread shutdownThread = new Thread(imageCacheExecutor::shutdownNow);
shutdownThread.setName("Thread-ImageCacheShutdown");
Runtime.getRuntime().addShutdownHook(shutdownThread);
}
private ImageCacheStore() {}
/**
* 传递图片缓存任务, 并等待缓存完成.
* @param cacheObject 缓存任务组
*/
public static Throwable executeCacheRequest(ImageCacheObject cacheObject) throws InterruptedException {
Task task = getTaskState(cacheObject);
if(task.taskState.get() == TaskState.COMPLETE) {
return null;
}
boolean locked = false;
try {
if(task.taskState.get() == TaskState.COMPLETE) {
return null;
}
task.lock.lock();
locked = true;
// 双重检查
if(task.taskState.get() == TaskState.COMPLETE) {
return null;
}
// 置任务状态
task.taskState.set(TaskState.RUNNING);
Throwable throwable = null;
try {
throwable = imageCacheExecutor.submit(() -> {
try {
handler.getImageToCache(cacheObject);
} catch (Throwable e) {
return e;
}
return null;
}).get();
if(throwable == null) {
task.taskState.set(TaskState.COMPLETE);
} else {
task.taskState.set(TaskState.ERROR);
}
} catch (ExecutionException e) {
log.error("执行图片缓存任务时发生异常", e);
}
return throwable;
} finally {
if(locked) {
task.lock.unlock();
}
}
}
private static Task getTaskState(ImageCacheObject cacheObject) {
if(!cacheMap.containsKey(cacheObject)) {
cacheMap.put(cacheObject, new Task());
}
return cacheMap.get(cacheObject);
}
/**
* 获取错误信息
*/
public static String getErrorMessageFromThrowable(Throwable throwable, boolean onlyRequestException) {
if(throwable == null) {
return "";
} else if(!(throwable instanceof HttpRequestException)) {
if(onlyRequestException) {
return "";
}
return throwable.getMessage();
}
JsonObject result = new Gson()
.fromJson(((HttpRequestException) throwable).getContent(), JsonObject.class);
return result.get("msg").getAsString();
}
/**
* 任务状态
*/
private enum TaskState {
READY, RUNNING, COMPLETE, ERROR
}
private static class Task {
public final ReentrantLock lock = new ReentrantLock(true);
public final AtomicReference<TaskState> taskState = new AtomicReference<>(TaskState.READY);
}
}

View File

@ -12,19 +12,45 @@ import java.util.concurrent.atomic.AtomicReference;
* 基于Hashtable的本地缓存库
* @param <T> 缓存类型
*/
public class LocalHashCacheStore<T> implements CacheStore<T> {
public class LocalHashCacheStore<T> implements CacheStore<T>, Cleanable {
private final Hashtable<String, CacheObject<T>> cache;
/**
* 构造一个基于Hashtable的本地缓存库
* @see Hashtable
*/
public LocalHashCacheStore() {
this(0);
}
/**
* 构造一个基于Hashtable的本地缓存库
* @param initialCapacity 初始容量
* @see Hashtable
*/
public LocalHashCacheStore(int initialCapacity) {
this(initialCapacity, 0F);
}
/**
* 构造一个基于Hashtable的本地缓存库
* @param initialCapacity 初始容量
* @param loadFactor 重载因子
* @see Hashtable
*/
public LocalHashCacheStore(int initialCapacity, float loadFactor) {
this(initialCapacity, loadFactor, false);
}
/**
* 构造一个基于Hashtable的本地缓存库
* @param initialCapacity 初始容量
* @param loadFactor 重载因子
* @param autoClean 是否自动清理
* @see Hashtable
*/
public LocalHashCacheStore(int initialCapacity, float loadFactor, boolean autoClean) {
if(initialCapacity != 0) {
if(loadFactor <= 0F) {
cache = new Hashtable<>(initialCapacity);
@ -34,6 +60,10 @@ public class LocalHashCacheStore<T> implements CacheStore<T> {
} else {
cache = new Hashtable<>();
}
if(autoClean) {
AutoCleanTimer.add(this);
}
}
@Override
@ -118,6 +148,15 @@ public class LocalHashCacheStore<T> implements CacheStore<T> {
return false;
}
@Override
public void clean() throws Exception {
Date currentDate = new Date();
cache.forEach((key, value) -> {
if(value.isExpire(currentDate)) {
cache.remove(key);
}
});
}
public static class CacheObject<T> implements Comparable<CacheObject<T>> {

View File

@ -13,7 +13,7 @@ import java.util.Set;
import java.util.function.Consumer;
import java.util.function.Function;
public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
private final JedisPool jedisPool;
private final String keyPrefix;
@ -36,7 +36,7 @@ public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
if(jedisPool.isClosed()) {
throw new IllegalStateException("JedisPool is closed");
}
log = LoggerFactory.getLogger(this.getClass().getSimpleName() + "@" + Integer.toHexString(jedisPool.hashCode()));
log = LoggerFactory.getLogger(this.getClass().getName() + "@" + Integer.toHexString(jedisPool.hashCode()));
if(!Strings.isNullOrEmpty(keyPrefix)) {
this.keyPrefix = keyPrefix.endsWith(".") ? keyPrefix : keyPrefix + ".";
} else {

View File

@ -0,0 +1,44 @@
package net.lamgc.cgj.bot.cache.exception;
import java.io.IOException;
import java.util.Objects;
import org.apache.http.HttpResponse;
import org.apache.http.StatusLine;
import org.apache.http.util.EntityUtils;
public class HttpRequestException extends IOException {
private static final long serialVersionUID = -2229221075943552798L;
private final StatusLine statusLine;
private final String content;
public HttpRequestException(HttpResponse response) throws IOException {
this(response.getStatusLine(), EntityUtils.toString(response.getEntity()));
}
public HttpRequestException(StatusLine statusLine, String content) {
super("Http Response Error: " + Objects.requireNonNull(statusLine, "statusLine is null") +
", Response Content: " + (content == null ? "null" : '\'' + content + '\''));
this.statusLine = statusLine;
this.content = content;
}
/**
* 获取Http状态行
*/
public StatusLine getStatusLine() {
return statusLine;
}
/**
* 获取Response内容
* @return 如果没有返回, 则返回null
*/
public String getContent() {
return content;
}
}

View File

@ -6,6 +6,7 @@ import com.google.common.util.concurrent.ThreadFactoryBuilder;
import net.lamgc.cgj.bot.BotAdminCommandProcess;
import net.lamgc.cgj.bot.BotCommandProcess;
import net.lamgc.cgj.bot.MessageEventExecutionDebugger;
import net.lamgc.cgj.bot.SettingProperties;
import net.lamgc.cgj.util.DateParser;
import net.lamgc.cgj.util.PagesQualityParser;
import net.lamgc.cgj.util.TimeLimitThreadPoolExecutor;
@ -16,12 +17,11 @@ import net.lamgc.utils.base.runner.exception.NoSuchCommandException;
import net.lamgc.utils.base.runner.exception.ParameterNoFoundException;
import net.lamgc.utils.event.*;
import net.lamgc.utils.event.EventObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.JedisPool;
import java.lang.reflect.Method;
import java.net.URI;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.concurrent.LinkedBlockingQueue;
@ -38,19 +38,15 @@ public class BotEventHandler implements EventHandler {
private final ArgumentsRunner processRunner;
private final ArgumentsRunner adminRunner;
private final static Logger log = LoggerFactory.getLogger("BotEventHandler");
private final static Logger log = LoggerFactory.getLogger(BotEventHandler.class);
/**
* 所有缓存共用的JedisPool
*/
private final static URI redisServerUri = URI.create("redis://" + System.getProperty("cgj.redisAddress"));
public final static JedisPool redisServer = new JedisPool(redisServerUri.getHost(), redisServerUri.getPort() == -1 ? 6379 : redisServerUri.getPort());
private final static Map<Long, AtomicBoolean> muteStateMap = new Hashtable<>();
/**
* 消息事件执行器
*/
private final static EventExecutor executor = new EventExecutor(new TimeLimitThreadPoolExecutor(
60 * 1000,
0,
Math.max(Runtime.getRuntime().availableProcessors(), 4),
Math.max(Math.max(Runtime.getRuntime().availableProcessors() * 2, 4), 32),
30L,
@ -62,10 +58,10 @@ public class BotEventHandler implements EventHandler {
));
private static boolean initialled = false;
static {
initial();
}
/**
* 初始化BotEventHandler
*/
public synchronized static void initial() {
if(initialled) {
Logger logger = LoggerFactory.getLogger("BotEventHandler@<init>");
@ -88,25 +84,19 @@ public class BotEventHandler implements EventHandler {
});
try {
executor.addHandler(new BotEventHandler());
Thread shutdownThread = new Thread(() -> executor.shutdown(true));
shutdownThread.setName("Thread-EventHandlerShutdown");
Runtime.getRuntime().addShutdownHook(shutdownThread);
} catch (IllegalAccessException e) {
LoggerFactory.getLogger("BotEventHandler@Static").error("添加Handler时发生异常", e);
}
initialled = true;
}
private final static AtomicBoolean preLoaded = new AtomicBoolean();
/**
* 预加载
*/
public synchronized static void preLoad() {
if(preLoaded.get()) {
return;
}
try {
BotAdminCommandProcess.loadPushList();
} finally {
preLoaded.set(true);
} catch(Throwable e) {
log.error("加载推送列表失败", e);
}
initialled = true;
}
private BotEventHandler() {
@ -130,20 +120,38 @@ public class BotEventHandler implements EventHandler {
*/
@NotAccepted
public static void executeMessageEvent(MessageEvent event) {
String debuggerName;
try {
executeMessageEvent(event, false);
} catch (InterruptedException e) {
log.error("执行时发生异常", e);
throw new RuntimeException(e);
}
}
/**
* 投递消息事件
* @param event 事件对象
*/
@NotAccepted
public static void executeMessageEvent(MessageEvent event, boolean sync) throws InterruptedException {
String debuggerName = SettingProperties.getProperty(0, "debug.debugger");
if(!event.getMessage().startsWith(ADMIN_COMMAND_PREFIX) &&
!Strings.isNullOrEmpty(debuggerName = BotCommandProcess.globalProp.getProperty("debug.debugger"))) {
!Strings.isNullOrEmpty(debuggerName)) {
try {
MessageEventExecutionDebugger debugger = MessageEventExecutionDebugger.valueOf(debuggerName.toUpperCase());
debugger.debugger.accept(executor, event, BotCommandProcess.globalProp,
MessageEventExecutionDebugger.getDebuggerLogger(debugger));
debugger.debugger.accept(executor, event, SettingProperties.getProperties(SettingProperties.GLOBAL),
MessageEventExecutionDebugger.getDebuggerLogger(debugger));
} catch(IllegalArgumentException e) {
log.warn("未找到指定调试器: '{}'", debuggerName);
} catch (Exception e) {
log.error("事件调试处理时发生异常", e);
}
} else {
BotEventHandler.executor.executor(event);
if(sync) {
BotEventHandler.executor.executorSync(event);
} else {
BotEventHandler.executor.executor(event);
}
}
}
@ -155,7 +163,10 @@ public class BotEventHandler implements EventHandler {
public void processMessage(MessageEvent event) {
String msg = event.getMessage();
log.debug(event.toString());
if(!match(msg)) {
if(mismatch(msg)) {
return;
} else if(isMute(event.getFromGroup())) {
log.debug("机器人已被禁言, 忽略请求.");
return;
}
@ -195,7 +206,8 @@ public class BotEventHandler implements EventHandler {
Object result;
try {
if(msg.toLowerCase().startsWith(ADMIN_COMMAND_PREFIX)) {
if(!String.valueOf(event.getFromQQ()).equals(BotCommandProcess.globalProp.getProperty("admin.adminId"))) {
if(!String.valueOf(event.getFromQQ())
.equals(SettingProperties.getProperty(0, "admin.adminId"))) {
result = "你没有执行该命令的权限!";
} else {
result = adminRunner.run(args.length <= 1 ? new String[0] : Arrays.copyOfRange(args, 1, args.length));
@ -208,21 +220,31 @@ public class BotEventHandler implements EventHandler {
} catch(ParameterNoFoundException e) {
result = "命令缺少参数: " + e.getParameterName();
} catch(DeveloperRunnerException e) {
if (!(e.getCause() instanceof InterruptedException)) {
log.error("执行命令时发生异常", e);
result = "色图姬在执行命令时遇到了一个错误!";
} else {
Throwable cause = e.getCause();
if (cause instanceof InterruptedException) {
log.error("命令执行超时, 终止执行.");
result = "色图姬发现这个命令的处理时间太久了!所以打断了这个命令。";
} else if(cause instanceof NoSuchElementException && cause.getMessage().startsWith("No work found: ")) {
String message = cause.getMessage();
log.error("指定作品不存在.(Id: {})", message.substring(message.lastIndexOf(": ") + 2));
result = "色图姬找不到这个作品!";
} else {
log.error("执行命令时发生异常", e);
result = "色图姬在执行命令时遇到了一个错误!";
}
}
long processTime = System.currentTimeMillis() - time;
if(Objects.requireNonNull(result) instanceof String) {
if(!Objects.isNull(result) && result instanceof String && !isMute(event.getFromGroup())) {
try {
event.sendMessage((String) result);
int sendResult = event.sendMessage((String) result);
if(sendResult < 0) {
log.warn("消息发送失败, Sender {} 返回错误代码: {}", event.getClass().getName(), sendResult);
}
} catch (Exception e) {
log.error("发送消息时发生异常", e);
}
} else if(isMute(event.getFromGroup())) {
log.warn("命令反馈时机器人已被禁言, 跳过反馈.");
}
long totalTime = System.currentTimeMillis() - time;
log.info("命令反馈完成.(事件耗时: {}ms, P: {}%({}ms), R: {}%({}ms))", totalTime,
@ -235,8 +257,44 @@ public class BotEventHandler implements EventHandler {
* @param message 要检查的消息
* @return 如果为true则提交
*/
public static boolean match(String message) {
return message.startsWith(COMMAND_PREFIX) || message.startsWith(ADMIN_COMMAND_PREFIX);
public static boolean mismatch(String message) {
return !message.startsWith(COMMAND_PREFIX) && !message.startsWith(ADMIN_COMMAND_PREFIX);
}
private static boolean isMute(long groupId) {
Boolean mute = isMute(groupId, false);
return mute != null && mute;
}
/**
* 查询某群是否被禁言.
* @param groupId 群组Id
* @param rawValue 是否返回原始值(当没有该群状态, 且本参数为true时, 将返回null)
* @return 返回状态值, 如无该群禁言记录且rawValue = true, 则返回null
*/
public static Boolean isMute(long groupId, boolean rawValue) {
if(groupId <= 0) {
return false;
}
AtomicBoolean state = muteStateMap.get(groupId);
if(state == null && rawValue) {
return null;
}
return state != null && state.get();
}
/**
* 设置机器人禁言状态.
* <p>设置该项可防止因机器人在禁言期间反馈请求导致被封号.</p>
* @param mute 如果被禁言, 传入true
*/
public static void setMuteState(long groupId, boolean mute) {
if(!muteStateMap.containsKey(groupId)) {
muteStateMap.put(groupId, new AtomicBoolean(mute));
} else {
muteStateMap.get(groupId).set(mute);
}
log.warn("群组 {} 机器人禁言状态已变更: {}", groupId, mute ? "已禁言" : "已解除");
}
}

View File

@ -0,0 +1,70 @@
package net.lamgc.cgj.bot.event;
import java.util.Objects;
public class BufferMessageEvent extends MessageEvent {
private final StringBuffer buffer = new StringBuffer();
private final MessageEvent parent;
/**
* 以空消息空Id生成BufferMessageEvent
*/
public BufferMessageEvent() {
super(0, 0, "");
parent = null;
}
/**
* 提供消息内容构造BufferMessageEvent
* @param message 传入的消息内容
*/
public BufferMessageEvent(String message) {
super(0, 0, message);
parent = null;
}
/**
* 提供消息内容构和Id信息造BufferMessageEvent
* @param groupId 群组Id
* @param qqId 发送者Id
* @param message 传入的消息内容
*/
public BufferMessageEvent(int groupId, int qqId, String message) {
super(groupId, qqId, message);
parent = null;
}
/**
* 使用事件构造BufferMessageEvent
* @param parentEvent 父级消息事件对象
*/
public BufferMessageEvent(MessageEvent parentEvent) {
super(parentEvent.getFromGroup(), parentEvent.getFromQQ(), parentEvent.getMessage());
parent = parentEvent;
}
@Override
public int sendMessage(String message) {
buffer.append(message);
return 0;
}
/**
* 当提供了父级消息事件时, 本方法调用父级消息事件对象的{@code getImageUrl(String)}, 如果没有, 返回{@code null}
*/
@Override
public String getImageUrl(String image) {
return Objects.isNull(this.parent) ? null : this.parent.getImageUrl(image);
}
/**
* 获取缓冲区消息内容
* @return 消息内容
*/
public String getBufferMessage() {
return buffer.toString();
}
}

View File

@ -12,16 +12,17 @@ public abstract class MessageEvent implements EventObject, MessageSender {
public MessageEvent(long fromGroup, long fromQQ, String message) {
this.fromGroup = fromGroup;
this.fromQQ = fromQQ;
this.message = message;
this.message = message.trim();
}
/**
* 发送消息
* @param message 消息内容
* @return 成功返回MessageId, 如没有MessageId则返回0, 失败返回负数错误码
* @throws Exception 该方法根据不同实现, 可能会抛出不同异常, 详见实现所标识的文档内容.
*/
@Override
public abstract int sendMessage(final String message);
public abstract int sendMessage(final String message) throws Exception;
/**
* 获取图片下载地址.

View File

@ -0,0 +1,52 @@
package net.lamgc.cgj.bot.framework.cli;
import net.lamgc.cgj.bot.boot.ApplicationBoot;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import org.jline.reader.LineReader;
import org.jline.reader.LineReaderBuilder;
import org.jline.reader.impl.history.DefaultHistory;
import org.jline.terminal.TerminalBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
public class ConsoleMain {
private final static Logger log = LoggerFactory.getLogger(ConsoleMain.class);
public static void start() throws IOException {
ConsoleMessageSenderFactory senderFactory = new ConsoleMessageSenderFactory();
MessageSenderBuilder.setCurrentMessageSenderFactory(senderFactory);
ApplicationBoot.initialBot();
LineReader lineReader = LineReaderBuilder.builder()
.appName("CGJ")
.history(new DefaultHistory())
.terminal(TerminalBuilder.builder()
.dumb(true)
.build())
.build();
long qqId = Long.parseLong(lineReader.readLine("会话QQ: "));
long groupId = Long.parseLong(lineReader.readLine("会话群组号:"));
boolean isGroup = false;
do {
String input = lineReader.readLine("App " + qqId + (isGroup ? "@" + groupId : "$private") + " >");
if(input.equalsIgnoreCase("#exit")) {
System.out.println("退出应用...");
break;
} else if(input.equalsIgnoreCase("#setgroup")) {
isGroup = !isGroup;
System.out.println("System: 群模式状态已变更: " + isGroup);
continue;
}
try {
BotEventHandler.executeMessageEvent(new ConsoleMessageEvent(isGroup ? groupId : 0, qqId, input), true);
} catch (InterruptedException e) {
log.error("执行时发生中断", e);
}
} while(true);
}
}

View File

@ -0,0 +1,23 @@
package net.lamgc.cgj.bot.framework.cli;
import net.lamgc.cgj.bot.event.MessageEvent;
import java.util.Date;
public class ConsoleMessageEvent extends MessageEvent {
public ConsoleMessageEvent(long groupId, long qqId, String message) {
super(groupId, qqId, message);
}
@Override
public int sendMessage(String message) {
System.out.println(new Date() + " Bot: " + message);
return 0;
}
@Override
public String getImageUrl(String image) {
return null;
}
}

View File

@ -0,0 +1,13 @@
package net.lamgc.cgj.bot.framework.cli;
import net.lamgc.cgj.bot.message.MessageSender;
import java.util.Date;
public class ConsoleMessageSender implements MessageSender {
@Override
public synchronized int sendMessage(String message) {
System.out.println(new Date() + " Bot: " + message);
return 0;
}
}

View File

@ -0,0 +1,15 @@
package net.lamgc.cgj.bot.framework.cli;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSource;
public class ConsoleMessageSenderFactory implements MessageSenderFactory {
private final static ConsoleMessageSender sender = new ConsoleMessageSender();
@Override
public MessageSender createMessageSender(MessageSource source, long id) {
return sender;
}
}

View File

@ -1,5 +1,6 @@
package net.lamgc.cgj.bot.framework.coolq;
import net.lamgc.cgj.bot.boot.ApplicationBoot;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.framework.coolq.message.SpringCQMessageEvent;
import net.lamgc.utils.event.EventHandler;
@ -17,8 +18,9 @@ public class CQPluginMain extends CQPlugin implements EventHandler {
public CQPluginMain() {
// TODO(LamGC, 2020.04.21): SpringCQ无法适配MessageSenderBuilder
BotEventHandler.preLoad();
LoggerFactory.getLogger(this.toString())
// MessageSenderBuilder.setCurrentMessageSenderFactory(new SpringCQMessageSenderFactory());
ApplicationBoot.initialBot();
LoggerFactory.getLogger(CQPluginMain.class)
.info("BotEventHandler.COMMAND_PREFIX = {}", BotEventHandler.COMMAND_PREFIX);
}
@ -37,8 +39,14 @@ public class CQPluginMain extends CQPlugin implements EventHandler {
return processMessage(cq, event);
}
public int processMessage(CoolQ cq, CQMessageEvent event) {
if(!BotEventHandler.match(event.getMessage())) {
/**
* 处理消息
* @param cq CoolQ机器人对象
* @param event 消息事件
* @return 是否拦截消息
*/
private static int processMessage(CoolQ cq, CQMessageEvent event) {
if(BotEventHandler.mismatch(event.getMessage())) {
return MESSAGE_IGNORE;
}
BotEventHandler.executeMessageEvent(new SpringCQMessageEvent(cq, event));

View File

@ -36,7 +36,7 @@ public class SpringCQMessageEvent extends MessageEvent {
}
@Override
public int sendMessage(final String message) {
public int sendMessage(final String message) throws Exception {
return messageSender.sendMessage(message);
}

View File

@ -5,16 +5,14 @@ import net.lamgc.cgj.bot.message.MessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lz1998.cq.robot.CoolQ;
import java.util.Objects;
public class SpringCQMessageSenderFactory implements MessageSenderFactory {
private final CoolQ coolQ;
public SpringCQMessageSenderFactory(CoolQ coolQ) {
this.coolQ = coolQ;
}
private final static ThreadLocal<CoolQ> threadCoolQ = new ThreadLocal<>();
@Override
public MessageSender createMessageSender(MessageSource source, long id) {
return new SpringCQMessageSender(coolQ, source, id);
return new SpringCQMessageSender(
Objects.requireNonNull(threadCoolQ.get(), "CoolQ object is not included in ThreadLocal"), source, id);
}
}

View File

@ -1,14 +1,17 @@
package net.lamgc.cgj.bot.framework.mirai;
import net.lamgc.cgj.bot.boot.ApplicationBoot;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.framework.mirai.message.MiraiMessageEvent;
import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import net.lamgc.cgj.bot.framework.mirai.message.MiraiMessageSenderFactory;
import net.mamoe.mirai.Bot;
import net.mamoe.mirai.BotFactoryJvm;
import net.mamoe.mirai.event.events.BotMuteEvent;
import net.mamoe.mirai.event.events.BotUnmuteEvent;
import net.mamoe.mirai.japt.Events;
import net.mamoe.mirai.message.FriendMessage;
import net.mamoe.mirai.message.GroupMessage;
import net.mamoe.mirai.message.*;
import net.mamoe.mirai.utils.BotConfiguration;
import org.apache.commons.net.util.Base64;
import org.slf4j.Logger;
@ -19,7 +22,7 @@ import java.util.Properties;
public class MiraiMain implements Closeable {
private final Logger log = LoggerFactory.getLogger(this.toString());
private final Logger log = LoggerFactory.getLogger(MiraiMain.class);
private Bot bot;
@ -34,7 +37,7 @@ public class MiraiMain implements Closeable {
return;
}
File botPropFile = new File(System.getProperty("cgj.botDataDir"), "./bot.properties");
File botPropFile = new File(BotGlobal.getGlobal().getDataStoreDir(), "./bot.properties");
try (Reader reader = new BufferedReader(new FileReader(botPropFile))) {
botProperties.load(reader);
} catch (IOException e) {
@ -42,18 +45,47 @@ public class MiraiMain implements Closeable {
return;
}
bot = BotFactoryJvm.newBot(Long.parseLong(botProperties.getProperty("bot.qq", "0")), Base64.decodeBase64(botProperties.getProperty("bot.password", "")), new BotConfiguration());
Events.subscribeAlways(GroupMessage.class, (msg) -> BotEventHandler.executeMessageEvent(new MiraiMessageEvent(msg)));
Events.subscribeAlways(FriendMessage.class, (msg) -> BotEventHandler.executeMessageEvent(new MiraiMessageEvent(msg)));
BotConfiguration configuration = new BotConfiguration();
configuration.setProtocol(BotConfiguration.MiraiProtocol.ANDROID_PAD);
bot = BotFactoryJvm.newBot(Long.parseLong(botProperties.getProperty("bot.qq", "0")), Base64.decodeBase64(botProperties.getProperty("bot.password", "")), configuration);
Events.subscribeAlways(GroupMessageEvent.class, this::executeMessageEvent);
Events.subscribeAlways(FriendMessageEvent.class, this::executeMessageEvent);
Events.subscribeAlways(TempMessageEvent.class, this::executeMessageEvent);
Events.subscribeAlways(BotMuteEvent.class,
event -> BotEventHandler.setMuteState(event.getGroup().getId(), true));
Events.subscribeAlways(BotUnmuteEvent.class,
event -> BotEventHandler.setMuteState(event.getGroup().getId(), false));
bot.login();
MessageSenderBuilder.setCurrentMessageSenderFactory(new MiraiMessageSenderFactory(bot));
BotEventHandler.preLoad();
ApplicationBoot.initialBot();
bot.join();
}
public void close() {
/**
* 处理消息事件
* @param message 消息事件对象
*/
private void executeMessageEvent(MessageEvent message) {
if(message instanceof GroupMessageEvent) {
GroupMessageEvent GroupMessageEvent = (GroupMessageEvent) message;
if(BotEventHandler.isMute(GroupMessageEvent.getGroup().getId(), true) == null) {
BotEventHandler.setMuteState(GroupMessageEvent.getGroup().getId(),
((GroupMessageEvent) message).getGroup().getBotMuteRemaining() != 0);
}
}
BotEventHandler.executeMessageEvent(MiraiMessageEvent.covertEventObject(message));
}
/**
* 关闭机器人
*/
public synchronized void close() {
if(bot == null) {
return;
}
log.warn("正在关闭机器人...");
bot.close(null);
bot = null;
log.warn("机器人已关闭.");
}

View File

@ -1,33 +1,92 @@
package net.lamgc.cgj.bot.framework.mirai.message;
import net.lamgc.cgj.bot.event.MessageEvent;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.message.ContactMessage;
import net.mamoe.mirai.message.GroupMessage;
import net.mamoe.mirai.message.FriendMessageEvent;
import net.mamoe.mirai.message.GroupMessageEvent;
import net.mamoe.mirai.message.MessageEvent;
import net.mamoe.mirai.message.TempMessageEvent;
import net.mamoe.mirai.message.data.MessageChain;
import net.mamoe.mirai.message.data.MessageUtils;
import java.util.Objects;
public class MiraiMessageEvent extends MessageEvent {
public class MiraiMessageEvent extends net.lamgc.cgj.bot.event.MessageEvent {
private final ContactMessage messageObject;
private final MessageEvent messageObject;
private final MessageSender messageSender;
public MiraiMessageEvent(ContactMessage message) {
super(message instanceof GroupMessage ? ((GroupMessage) message).getGroup().getId() : 0,
message.getSender().getId(), message.getMessage().contentToString());
/**
* 通过Mirai的MessageEvent转换成应用支持的MessageEvent.
* @deprecated 请使用 {@link #covertEventObject(MessageEvent)}方法转换.
* @param message 消息对象
* @see #covertEventObject(MessageEvent)
*/
@Deprecated
public MiraiMessageEvent(MessageEvent message) {
super(message instanceof GroupMessageEvent ? ((GroupMessageEvent) message).getGroup().getId() : 0,
message.getSender().getId(), getMessageBodyWithoutSource(message.getMessage().toString()));
this.messageObject = Objects.requireNonNull(message);
if(message instanceof GroupMessage) {
messageSender = new MiraiMessageSender(((GroupMessage) message).getGroup(), MessageSource.Group);
if(message instanceof GroupMessageEvent) {
messageSender = new MiraiMessageSender(((GroupMessageEvent) message).getGroup(), MessageSource.Group);
} else {
messageSender = new MiraiMessageSender(message.getSender(), MessageSource.Private);
}
}
/**
* 通过解析好的信息构造MessageEvent
* @param messageObject 消息原始对象
* @param groupId 群组Id, 非群聊或无需使用群聊时, 该参数为0
* @param qqId 发送者Id, 不能为0
* @param message 原始消息内容对象, 由构造方法内部解析
*/
private MiraiMessageEvent(MessageEvent messageObject, long groupId, long qqId, MessageChain message) {
super(groupId, qqId, getMessageBodyWithoutSource(message.toString()));
this.messageObject = Objects.requireNonNull(messageObject, "messageObject is null");
if(groupId != 0) {
this.messageSender = new MiraiMessageSender(((GroupMessageEvent) messageObject).getGroup(), MessageSource.Group);
} else {
this.messageSender = new MiraiMessageSender(messageObject.getSender(), MessageSource.Group);
}
}
/**
* 将Mirai原始MessageEvent转换成应用支持的MessageEvent对象
* @param event 原始消息对象
* @return 原始消息对象所对应的应用MessageEvent对象.
* @throws IllegalArgumentException 当出现不支持的Mirai {@link MessageEvent}实现时将抛出异常.
* @see MessageEvent 原始消息对象
* @see net.lamgc.cgj.bot.event.MessageEvent 应用消息对象
*/
public static MiraiMessageEvent covertEventObject(MessageEvent event) throws IllegalArgumentException {
if(event instanceof GroupMessageEvent) {
return new MiraiMessageEvent(event,
((GroupMessageEvent) event).getGroup().getId(), event.getSender().getId(), event.getMessage());
} else if(event instanceof FriendMessageEvent) {
return new MiraiMessageEvent(event, 0, event.getSender().getId(), event.getMessage());
} else if(event instanceof TempMessageEvent) {
return new MiraiMessageEvent(event, 0, event.getSender().getId(), event.getMessage());
} else {
throw new IllegalArgumentException("Unsupported event type: " + event.toString());
}
}
/**
* 将ContactMessage获得的消息内容删除 Mirai:source 并返回.
* <p>该做法比较保守, 防止Mirai:source位置出现变动.</p>
* @param message ContactMessage的消息内容;
* @return 返回删除了Mirai:source的消息
*/
private static String getMessageBodyWithoutSource(String message) {
StringBuilder builder = new StringBuilder(message);
int startIndex = builder.indexOf("[mirai:source:");
int endIndex = builder.indexOf("]", startIndex) + 1;
return builder.delete(startIndex, endIndex).toString();
}
@Override
public int sendMessage(final String message) {
public int sendMessage(final String message) throws Exception {
return messageSender.sendMessage(message);
}

View File

@ -2,19 +2,16 @@ package net.lamgc.cgj.bot.framework.mirai.message;
import com.google.common.base.Strings;
import net.lamgc.cgj.bot.BotCode;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.cache.CacheStore;
import net.lamgc.cgj.bot.cache.HotDataCacheStore;
import net.lamgc.cgj.bot.cache.LocalHashCacheStore;
import net.lamgc.cgj.bot.cache.StringRedisCacheStore;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.Bot;
import net.mamoe.mirai.contact.Contact;
import net.mamoe.mirai.message.data.Image;
import net.mamoe.mirai.message.data.Message;
import net.mamoe.mirai.message.data.MessageChain;
import net.mamoe.mirai.message.data.MessageUtils;
import net.mamoe.mirai.message.data.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@ -29,11 +26,11 @@ public class MiraiMessageSender implements MessageSender {
private final Contact member;
private final MessageSource source;
private final static Logger log = LoggerFactory.getLogger("MiraiMessageSender");
private final static Logger log = LoggerFactory.getLogger(MiraiMessageSender.class);
private final static CacheStore<String> imageIdCache = new HotDataCacheStore<>(
new StringRedisCacheStore(BotEventHandler.redisServer, "mirai.imageId"),
new StringRedisCacheStore(BotGlobal.getGlobal().getRedisServer(), "mirai.imageId"),
new LocalHashCacheStore<>(),
5400000, 1800000);
5400000, 1800000, true);
/**
* 使用id构造发送器
@ -60,7 +57,7 @@ public class MiraiMessageSender implements MessageSender {
public int sendMessage(final String message) {
log.debug("处理前的消息内容:\n{}", message);
Message msgBody = processMessage(Objects.requireNonNull(message));
log.debug("处理后的消息内容(可能出现乱序的情况, 但实际上顺序是没问题的):\n{}", msgBody);
log.debug("处理后的消息内容(可能出现乱序的情况, 但实际上顺序是没问题的):\n{}", msgBody.contentToString());
member.sendMessage(msgBody);
return 0;
}
@ -80,7 +77,7 @@ public class MiraiMessageSender implements MessageSender {
.replaceAll("&38", "&")
.split("\\|");
MessageChain messages = MessageUtils.newChain().plus("");
MessageChain messages = MessageUtils.newChain();
int codeIndex = 0;
for(String text : texts) {
if(text.equals("{BotCode}")) {
@ -117,6 +114,15 @@ public class MiraiMessageSender implements MessageSender {
} else {
return img;
}
case "face":
if(!code.containsParameter("id")) {
return MessageUtils.newChain("(无效的表情Id)");
}
int faceId = Integer.parseInt(code.getParameter("id"));
if(faceId <= 0) {
return MessageUtils.newChain("(无效的表情Id)");
}
return new Face(faceId);
default:
log.warn("解析到不支持的BotCode: {}", code);
return MessageUtils.newChain("(不支持的BotCode)");
@ -128,6 +134,7 @@ public class MiraiMessageSender implements MessageSender {
* @param code 图片BotCode
* @return Image对象
*/
@SuppressWarnings("SynchronizationOnLocalVariableOrMethodParameter")
public Image uploadImage(BotCode code) {
log.debug("传入BotCode信息:\n{}", code);
String absolutePath = code.getParameter("absolutePath");

View File

@ -5,6 +5,8 @@ import net.lamgc.cgj.bot.message.MessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.Bot;
import java.util.Objects;
public class MiraiMessageSenderFactory implements MessageSenderFactory {
private final Bot bot;
@ -14,15 +16,11 @@ public class MiraiMessageSenderFactory implements MessageSenderFactory {
}
@Override
public MessageSender createMessageSender(MessageSource source, long id) throws Exception {
switch(source) {
case Group:
case Discuss:
return new MiraiMessageSender(bot.getGroup(id), source);
case Private:
return new MiraiMessageSender(bot.getFriend(id), source);
default:
throw new NoSuchFieldException(source.toString());
public MessageSender createMessageSender(MessageSource source, long id) {
Objects.requireNonNull(source);
if(id <= 0) {
throw new IllegalArgumentException("id cannot be 0 or negative: " + id);
}
return new MiraiMessageSender(bot, source, id);
}
}

View File

@ -9,7 +9,8 @@ public interface MessageSender {
* 返回0则发送器不支持消息Id,
* 返回非0正整数则为消息Id,
* 返回负数则为错误.
* @throws Exception 该方法根据不同实现, 可能会抛出不同异常, 详见实现所标识的文档内容.
*/
int sendMessage(final String message);
int sendMessage(final String message) throws Exception;
}

View File

@ -2,7 +2,7 @@ package net.lamgc.cgj.bot.sort;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.BotCommandProcess;
import net.lamgc.cgj.bot.cache.CacheStoreCentral;
import java.io.IOException;
import java.util.Comparator;
@ -20,6 +20,15 @@ public class PreLoadDataComparator implements Comparator<JsonElement> {
@Override
public int compare(JsonElement o1, JsonElement o2) {
if(!o1.isJsonObject() || !o2.isJsonObject()) {
if(o1.isJsonObject()) {
return 1;
} else if(o2.isJsonObject()) {
return -1;
} else {
return 0;
}
}
if(!o1.getAsJsonObject().has("illustId") || !o2.getAsJsonObject().has("illustId")) {
if(o1.getAsJsonObject().has("illustId")) {
return 1;
@ -30,9 +39,13 @@ public class PreLoadDataComparator implements Comparator<JsonElement> {
}
}
try {
JsonObject illustPreLoadData1 = BotCommandProcess.getIllustPreLoadData(o1.getAsJsonObject().get("illustId").getAsInt(), false);
JsonObject illustPreLoadData2 = BotCommandProcess.getIllustPreLoadData(o2.getAsJsonObject().get("illustId").getAsInt(), false);
return Integer.compare(illustPreLoadData2.get(attribute.attrName).getAsInt(), illustPreLoadData1.get(attribute.attrName).getAsInt());
JsonObject illustPreLoadData1 =
CacheStoreCentral.getIllustPreLoadData(o1.getAsJsonObject().get("illustId").getAsInt(), false);
JsonObject illustPreLoadData2 =
CacheStoreCentral.getIllustPreLoadData(o2.getAsJsonObject().get("illustId").getAsInt(), false);
return Integer.compare(
illustPreLoadData2.get(attribute.attrName).getAsInt(),
illustPreLoadData1.get(attribute.attrName).getAsInt());
} catch (IOException e) {
e.printStackTrace();
return 0;

View File

@ -18,7 +18,7 @@ public interface AdultContentDetector {
* @param pageIndex 指定页数, 设为0或负数则视为单页面作品
* @return 如果为true则为成人作品, 该方法将由检测器决定如何为成人作品.
*/
boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex) throws Exception;
boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex) throws Exception;
/**
* 检查某一作品是否为成人内容
@ -26,8 +26,8 @@ public interface AdultContentDetector {
* @param isUgoira 是否为动图
* @param pageIndex 指定页数, 设为0或负数则视为单页面作品
* @param threshold 指数阀值, 当等于或大于该阀值时返回true.
* @return 如果为true则为成人作品, 该方法将由 threshold 参数决定是否为成人作品.
* @return 如果为true则为成人作品, 该方法将由 threshold 参数决定是否为成人作品(如果超过阈值, 则为true).
*/
boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex, double threshold) throws Exception;
boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex, double threshold) throws Exception;
}

View File

@ -33,7 +33,7 @@ import java.util.function.BiConsumer;
public class PixivDownload {
private final static Logger log = LoggerFactory.getLogger("PixivDownload");
private final static Logger log = LoggerFactory.getLogger(PixivDownload.class);
private final HttpClient httpClient;
@ -542,8 +542,9 @@ public class PixivDownload {
* }
* </pre>
* @throws IOException 当请求发生异常, 或接口返回错误信息时抛出.
* @throws NoSuchElementException 当该作品不存在时抛出异常
*/
public JsonObject getIllustInfoByIllustId(int illustId) throws IOException {
public JsonObject getIllustInfoByIllustId(int illustId) throws IOException, NoSuchElementException {
HttpGet request = createHttpGetRequest(PixivURL.getPixivIllustInfoAPI(illustId));
HttpResponse response = httpClient.execute(request);
String responseStr = EntityUtils.toString(response.getEntity());
@ -558,7 +559,7 @@ public class PixivDownload {
if(illustsArray.size() == 1) {
return illustsArray.get(0).getAsJsonObject();
} else {
return null;
throw new NoSuchElementException("No work found: " + illustId);
}
}

View File

@ -27,21 +27,30 @@ public class PixivURL {
/**
* P站搜索请求url
* @deprecated 该接口已被替换, 请使用{@link PixivSearchBuilder}构造搜索Url
* @see PixivSearchBuilder
*/
@Deprecated
private static final String PIXIV_SEARCH_URL = "https://www.pixiv.net/search.php";
/**
* P站搜索用户url
* 需要替换的参数:
* {nick} - 用户昵称、部分名称
* @deprecated 该接口已被替换, 请使用{@link PixivSearchBuilder}构造搜索Url
* @see PixivSearchBuilder
*/
@Deprecated
public static final String PIXIV_SEARCH_USER_URL = PIXIV_SEARCH_URL + "?s_mode=s_usr&nick={nick}";
/**
* P站搜索插画url
* 需要替换的参数:
* {word} - 插画相关文本
* @deprecated 该接口已被替换, 请使用{@link PixivSearchBuilder}构造搜索Url
* @see PixivSearchBuilder
*/
@Deprecated
public static final String PIXIV_SEARCH_TAG_URL = PIXIV_SEARCH_URL + "?s_mode=s_tag&word={word}";
/**
@ -164,6 +173,7 @@ public class PixivURL {
/**
* 查询用户收藏.<br/>
* 该URL返回HTML页面需要进行解析.<br/>
* <p>注意: 该接口需要登陆</p>
* 需要替换的文本:<br/>
* {pageIndex} - 页数, 超出了则结果为空<br/>
*/
@ -336,6 +346,7 @@ public class PixivURL {
* @param mode 要检查的RankingMode项
* @return 如果支持返回true
*/
@SuppressWarnings("BooleanMethodIsAlwaysInverted")
public boolean isSupportedMode(RankingMode mode) {
return Arrays.binarySearch(supportedMode, mode) >= 0;
}

View File

@ -30,7 +30,7 @@ import java.util.zip.ZipInputStream;
*/
public final class PixivUgoiraBuilder {
private final Logger log = LoggerFactory.getLogger(PixivUgoiraBuilder.class.getSimpleName() + "@" + Integer.toHexString(this.hashCode()));
private final Logger log = LoggerFactory.getLogger(this.toString());
private final HttpClient httpClient;
private final JsonObject ugoiraMeta;
@ -82,7 +82,33 @@ public final class PixivUgoiraBuilder {
log.debug("IllustId: {}, UgoiraMeta: {}", this.illustId, this.ugoiraMeta);
}
/**
* 获取动图元数据
* @return 动图元数据, 返回的对象不影响Builder中的meta对象
*/
public JsonObject getUgoiraMeta() {
return this.ugoiraMeta.deepCopy();
}
/**
* 构建动图
* @param original 是否为原图画质
* @return 返回动图数据输入流
* @throws IOException 当获取数据发生异常时抛出
*/
public InputStream buildUgoira(boolean original) throws IOException {
ByteArrayOutputStream bufferOutput = new ByteArrayOutputStream();
buildUgoira(bufferOutput, original);
return new ByteArrayInputStream(bufferOutput.toByteArray());
}
/**
* 构建动图
* @param outputStream 动图输出流
* @param original 是否为原图画质
* @throws IOException 当获取数据发生异常时抛出
*/
public void buildUgoira(OutputStream outputStream, boolean original) throws IOException {
getUgoiraImageSize();
log.debug("动图尺寸信息: Height: {}, Width: {}", height, width);
@ -95,7 +121,6 @@ public final class PixivUgoiraBuilder {
HttpResponse response = httpClient.execute(request);
log.trace("请求已发送, 正在处理响应...");
ZipInputStream zipInputStream = new ZipInputStream(new BufferedInputStream(response.getEntity().getContent(), 64 * 1024));
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
ZipEntry entry;
ByteArrayOutputStream cacheOutputStream = new ByteArrayOutputStream(512);
HashMap<String, InputStream> frameMap = new HashMap<>(frames.size());
@ -140,7 +165,6 @@ public final class PixivUgoiraBuilder {
}
});
encoder.finishEncoding();
return new ByteArrayInputStream(outputStream.toByteArray());
}
/**

View File

@ -27,7 +27,7 @@ import java.util.List;
*/
public class PixivAccessProxyServer {
private final Logger log = LoggerFactory.getLogger(this.getClass().getSimpleName());
private final Logger log = LoggerFactory.getLogger(PixivAccessProxyServer.class);
private final HttpProxyServer proxyServer;
@ -99,7 +99,7 @@ public class PixivAccessProxyServer {
}*/
log.info("Response Cookie: " + value);
BasicClientCookie cookie = parseRawCookie(value);
cookieStore.addCookie(null);
cookieStore.addCookie(cookie);
});
httpResponse.headers().remove(HttpHeaderNames.SET_COOKIE);
super.afterResponse(clientChannel, proxyChannel, httpResponse, pipeline);

View File

@ -1,7 +1,6 @@
package net.lamgc.cgj.util;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.utils.base.runner.StringParameterParser;
public class PagesQualityParser implements StringParameterParser<PixivDownload.PageQuality> {

View File

@ -0,0 +1,51 @@
package net.lamgc.cgj.util;
import net.lamgc.utils.base.ArgumentsProperties;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public final class PropertiesUtils {
private final static Logger log = LoggerFactory.getLogger(PropertiesUtils.class);
private PropertiesUtils() {}
/**
* 从ArgumentsProperties获取设置项到System Properties
* @param prop ArgumentsProperties对象
* @param key 设置项key
* @param defaultValue 默认值
* @return 如果成功从ArgumentsProperties获得设置项, 返回true, 如未找到(使用了defaultValue或null), 返回false;
*/
public static boolean getSettingToSysProp(ArgumentsProperties prop, String key, String defaultValue) {
if(prop.containsKey(key)) {
log.info("{}: {}", key, prop.getValue(key));
System.setProperty("cgj." + key, prop.getValue(key));
return true;
} else {
if(defaultValue != null) {
System.setProperty("cgj." + key, defaultValue);
}
return false;
}
}
/**
* 将环境变量的值读取并存入System Properties.
* @param envKey 待获取的环境变量Key
* @param sysPropKey 要设置的System Properties Key值
* @param defaultValue 默认值, 可选
* @return 如果设置成功, 返回true, 否则返回false
*/
public static boolean getEnvSettingToSysProp(String envKey, String sysPropKey, String defaultValue) {
String env = System.getenv(envKey);
if(env != null) {
System.setProperty("cgj." + sysPropKey, env);
return true;
} else if(defaultValue != null) {
System.setProperty("cgj." + sysPropKey, defaultValue);
}
return false;
}
}

View File

@ -28,32 +28,29 @@ public class TimeLimitThreadPoolExecutor extends ThreadPoolExecutor {
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
setInitialTime(0, executeLimitTime);
setInitialTime(executeLimitTime);
timeoutCheckThread.start();
}
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory);
setInitialTime(0, executeLimitTime);
setInitialTime(executeLimitTime);
timeoutCheckThread.start();
}
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, RejectedExecutionHandler handler) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, handler);
setInitialTime(0, executeLimitTime);
setInitialTime(executeLimitTime);
timeoutCheckThread.start();
}
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory, RejectedExecutionHandler handler) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler);
setInitialTime(0, executeLimitTime);
setInitialTime(executeLimitTime);
timeoutCheckThread.start();
}
private void setInitialTime(long checkInterval, long executeLimitTime) {
if(checkInterval > 0) {
timeoutCheckInterval.set(checkInterval);
}
private void setInitialTime(long executeLimitTime) {
if(executeLimitTime > 0) {
executeTimeLimit.set(executeLimitTime);
}

View File

@ -11,7 +11,8 @@
<Console name="CONSOLE_STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="${pattern}" charset="${charset}"/>
<Filters>
<LevelRangeFilter minLevel="INFO" maxLevel="INFO" />
<ThresholdFilter level="warn" onMatch="DENY" onMismatch="NEUTRAL"/>
<ThresholdFilter level="info" onMatch="ACCEPT" onMismatch="DENY"/>
</Filters>
</Console>
<Console name="CONSOLE_STDERR" target="SYSTEM_ERR">
@ -31,7 +32,7 @@
<Loggers>
<Logger level="INFO" name="org.apache.http"/>
<Root level="TRACE">
<Root level="INFO">
<AppenderRef ref="CONSOLE_STDOUT"/>
<AppenderRef ref="CONSOLE_STDERR"/>
<AppenderRef ref="rollingFile"/>

View File

@ -31,9 +31,9 @@ public class PixivDownloadTest {
private static CookieStore cookieStore;
private final static Logger log = LoggerFactory.getLogger("PixivDownloadTest");
private final static Logger log = LoggerFactory.getLogger(PixivDownloadTest.class);
private static HttpHost proxy = new HttpHost("127.0.0.1", 1001);
private static final HttpHost proxy = new HttpHost("127.0.0.1", 1001);
@BeforeClass
public static void before() throws IOException, ClassNotFoundException {
@ -180,9 +180,8 @@ public class PixivDownloadTest {
log.info("正在调用方法...");
try {
pixivDownload.getRankingAsInputStream(null, null, queryDate, 5, 50, PixivDownload.PageQuality.ORIGINAL, (rank, link, rankInfo, inputStream) -> {
log.info("空操作");
});
pixivDownload.getRankingAsInputStream(null, null, queryDate, 5, 50,
PixivDownload.PageQuality.ORIGINAL, (rank, link, rankInfo, inputStream) -> log.info("空操作"));
} finally {
zos.finish();
zos.flush();

View File

@ -1,11 +1,12 @@
package net.lamgc.cgj.pixiv;
import org.apache.http.HttpHost;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.client.HttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.bouncycastle.util.io.Streams;
import org.junit.Ignore;
import org.junit.Test;
import org.slf4j.LoggerFactory;
import java.io.File;
import java.io.IOException;
@ -18,8 +19,9 @@ public class PixivUgoiraBuilderTest {
@Test
public void buildTest() throws IOException {
File outputFile = new File("./output2.gif");
CloseableHttpClient httpClient = HttpClientBuilder.create().setProxy(new HttpHost("127.0.0.1", 1001)).build();
PixivUgoiraBuilder builder = new PixivUgoiraBuilder(httpClient, 80766493);
HttpClient httpClient = HttpClientBuilder.create().setProxy(new HttpHost("127.0.0.1", 1080)).build();
PixivUgoiraBuilder builder = new PixivUgoiraBuilder(httpClient, 81163967);
LoggerFactory.getLogger(PixivUgoiraBuilderTest.class).info("UgoiraMeta: {}", builder.getUgoiraMeta());
InputStream inputStream = builder.buildUgoira(true);
Files.write(outputFile.toPath(), Streams.readAll(inputStream));
}