Compare commits

...

286 Commits

Author SHA1 Message Date
5637ef30d4 Merge branch 'master' of github.com:LamGC/ContentGrabbingJi into add-framework-interface 2020-07-15 14:40:38 +08:00
bf8de1ac1e [Fix] 修复潜在的时区错误问题;
[Fix] Dockerfile.sample 将时区配置从`Asia/Shanghai`改成`GMT+8`;
2020-07-15 14:09:47 +08:00
6fc7d8ad78 [Change] 调整Framework接口, 为框架提供特定数据存储目录;
[Change] Framework 适配FrameworkResources的更改, 将`getName`调整为`getIdentify`, 增加`getFrameworkName`方法用于获取框架名(可能会改为注解方式以防止更改);
[Change] FrameworkManager 适配Framework的更改, 增加`checkFramework`方法以对FrameworkName进行检查;
[Change] FrameworkManager, FrameworkResources 将FrameworkResources从FrameworkManager分离成单独的类;
[Change] ConsoleMain, MiraiMain, SpringCQApplication 适配相关更改;
2020-07-15 10:57:30 +08:00
a7c434da61 [Change] 调整Readme内容;
[Fix] README.md 补充管理员命令内容;
2020-07-15 10:35:31 +08:00
575dc0c7fb [Fix] Dockerfile.sample 修复了容器无法正常启动的问题; 2020-07-14 11:01:21 +08:00
56ef463c63 [Change] MessageEvent 调整toString输出中hashCode的格式; 2020-07-13 20:35:39 +08:00
4387da37f5 [Change] 调整群禁言管理职责;
[Delete] BotEventHandler 移除禁言状态管理相关功能;
[Add] GroupMuteManager, GroupMuteManagerTest 增加群禁言状态管理类;
[Change] MiraiMain 将禁言状态对接由使用BotEventHandler改为独立管理;
2020-07-13 09:47:36 +08:00
0fc3e3ab48 [Change] 调整应用内事件线程池参数;
[Change] BotEventHandler 调整线程池参数;
2020-07-13 09:47:31 +08:00
a606ec0423 [Fix] IssueTemplates 修复模板错误的问题; 2020-07-12 11:18:29 +08:00
210aa84ed5 [Update] IssueTemplates 增加功能模板并更新Bug模板; 2020-07-12 11:12:21 +08:00
6fbbe522db [Fix #22] PixivDownload 修复因InputStream意外关闭导致的异常误判; 2020-07-12 01:12:58 +08:00
6d55325fc7 [Fix] PixivUgoiraBuilder 修复ZipInputStream在读取完第一帧图像后被意外关闭的问题; 2020-07-10 09:53:41 +08:00
d1c7f6f973 [Version] 更新版本(2.5.2-20200630.2-SNAPSHOT -> 2.5.2-20200709.1-SNAPSHOT); 2020-07-09 22:45:36 +08:00
0727ef4f93 [Clear] CQPluginMain 整理代码; 2020-07-09 22:19:35 +08:00
73a1caaf46 [Fix #21] 修复了RandomIntervalSendTimer在高版本Java中因非法反射导致应用异常终止的问题; 2020-07-09 22:17:09 +08:00
4784f8773b [Change] Pixiv搜索推荐候选接口.md 更新文档内容; 2020-07-03 10:09:21 +08:00
9a7d16124a [Add] FrameworkManager 添加"frameworkSet"方法; 2020-07-03 10:05:20 +08:00
b754559187 Merge branch 'master' into add-framework-interface 2020-07-03 09:15:27 +08:00
f80b6e72e0 [Fix] CQPluginMain 通过延迟加载来修复ApplicationBoot初始化失败的问题; 2020-07-03 09:11:27 +08:00
05e933838e Merge branch 'update-SpringCQ' 2020-07-03 08:20:18 +08:00
a87735d9e0 [Change] Main, SpringCQApplication 移除仅与SpringCQ相关的Spring相关代码, 转移到SpringCQApplication;
[Change] FrameworkManager 调整"registerFramework"返回值, 调整"shutdownAllFramework"的过程;
2020-07-03 08:19:21 +08:00
6ec99dbf17 Merge branch 'update-SpringCQ' into add-framework-interface
# Conflicts:
#	src/main/java/net/lamgc/cgj/Main.java
#	src/main/java/net/lamgc/cgj/bot/framework/coolq/SpringCQApplication.java
2020-07-03 08:02:50 +08:00
1599a5325a [Change] Framework 调整"getName"方法的默认实现;
[Change] Main 调整pluginMode启动方式;
[Change] SpringCQApplication 实现Framework接口;
2020-07-03 08:01:31 +08:00
3045b571a8 [Change] Main, SpringCQApplication 移除仅与SpringCQ相关的Spring相关代码, 转移到SpringCQApplication;
[Delete] CQPluginMain 移除注释信息;
2020-07-03 07:52:48 +08:00
5c2b6b4ee5 Merge branch 'update-SpringCQ' into add-framework-interface 2020-07-03 07:48:16 +08:00
c2e8a07500 [Change] ConsoleMain, MiraiMain 尝试为平台框架实现Framework接口;
[Change] Framework 补充Javadoc, 增加"getName"方法的默认方法体;
[Change] FrameworkManager 调整向frameworkThreadGroup发起中断的时机;
2020-07-03 00:46:23 +08:00
1b937953c3 [Add] Framework, FrameworkManager 增加框架统一管理接口; 2020-07-03 00:27:54 +08:00
553212e556 [Change] SpringCQApplication, Main 将SpringCQ启动部分迁移到SpringCQApplication内; 2020-07-03 00:23:56 +08:00
5383de7450 Merge remote-tracking branch 'origin/master' 2020-07-02 23:33:36 +08:00
9964205dc2 [Add] Pixiv搜索推荐候选接口.md 添加可用于优化搜索的接口文档; 2020-07-02 23:21:28 +08:00
394c3940e4 Merge remote-tracking branch 'origin/master' 2020-07-02 14:03:59 +08:00
d3d6f151d4 [Delete] application.properties 移除Properties版Application配置文件;
[Delete] CQConfig 移除不兼容的类;
[Add] application.yml 添加yml格式的Application配置文件;
[Change] Main 移除CQConfig类引用;
[Update] net.lz1998:spring-cq 升级依赖项版本(4.14.0.6 -> 4.15.0.1);
[Change] SpringCQMessageSenderFactory, CQPluginMain 使用较为妥协的方法实现SenderFactory;
2020-07-02 14:02:01 +08:00
62e3affef1 [Change] LocalHashCacheStore 调整内部类CacheObject的访问权; 2020-07-02 13:46:46 +08:00
51efc95c1c [Update] net.mamoe:mirai-core, net.mamoe:mirai-core-qqandroid 更新依赖库版本(1.0.2 -> 1.0.4); 2020-07-02 13:41:34 +08:00
d5c3a438b0 [Change] Pixiv预加载数据.md 更改文档标题; 2020-07-02 09:25:01 +08:00
792bfcb1bd [Add] Pixiv预加载数据.md 增加新的Pixiv接口文档; 2020-07-02 09:10:01 +08:00
cbd7db3570 Merge branch 'master' of github.com:LamGC/ContentGrabbingJi 2020-06-30 22:43:22 +08:00
467e7065fa [Update] net.lamgc:java-utils 更新依赖项版本(1.2.0_20200517.1-SNAPSHOT -> 1.3.1); 2020-06-30 22:40:39 +08:00
d2240e56fd [Change] README.md 调整格式; 2020-06-30 10:40:34 +08:00
eef9f285ca [Update] README.md 更新一份更规范的Readme; 2020-06-30 08:58:08 +08:00
f63d52df6d [Version] 更新版本(2.5.2-20200630.1-SNAPSHOT -> 2.5.2-20200630.2-SNAPSHOT); 2020-06-30 08:05:18 +08:00
50a638e97d [Clear] CacheStoreCentral 清理测试代码; 2020-06-30 08:01:08 +08:00
968a616595 [Version] 更新版本(2.5.2-20200618.1-SNAPSHOT -> 2.5.2-20200630.1-SNAPSHOT); 2020-06-30 01:23:04 +08:00
6443ba68ab [Fix] AutoCleanTimer 修复自动清除定时器没有重复工作;
[Fix] LocalHashCacheStore 修复clean方法抛出'ConcurrentModificationException'异常的问题;
[Change] CacheStoreCentral 调整PreLoadData的热点缓存时间;
[Change] HotDataCacheStore 将最近获取的Key重置其过期时间, 调整clean过程;
2020-06-30 01:21:17 +08:00
cbe0a38f59 Merge branch 'master' of github.com:LamGC/ContentGrabbingJi 2020-06-29 23:33:41 +08:00
eadfacb7d0 [Clear] PixivSearchLinkBuilder 清理代码; 2020-06-23 13:53:54 +08:00
572189d906 [Change] Main 增加'buildPassword'命令用于构造加密后的密码, 移除对PixivLoginProxyServer的嵌入支持;
[Delete] pom.xml 移除依赖项'net.lamgc:PixivLoginProxyServer';
2020-06-23 10:02:47 +08:00
811694587d [Version] 更新版本(2.5.2-20200617.1-SNAPSHOT -> 2.5.2-20200618.1-SNAPSHOT); 2020-06-18 19:27:11 +08:00
419f2de055 [Add] BotEventHandler 设计在消息正在发送,被时限线程池中断时, 重新投递事件以重新发送消息; 2020-06-18 10:11:08 +08:00
368c78e171 [Add] CacheStoreCentral, BotCommandProcess Search命令增加对pageIndex参数使用(之前竟然是个摆设!?); 2020-06-18 10:05:34 +08:00
bccf47db6e [Change] MessageEvent toString方法增加hashCode输出; 2020-06-18 09:35:37 +08:00
cf351074cc [Delete] BufferMessageEvent 移除缓冲消息事件;
[Add] BufferedMessageSender 增加缓冲消息发送器, 用于替代BufferMessageEvent;
[Change] BotCommandProcess 将Random命令中所使用的BufferMessageEvent替换成BufferedMessageSender;
2020-06-18 09:35:25 +08:00
c4ce18d37a [Fix] log4j2.xml 修复HttpClient日志输出过多的问题;
[Change] PixivURL 将PixivURL设计为不可继承且不可实例化;
2020-06-18 09:01:16 +08:00
26fd18917d [Delete] PixivAccessProxyServer 移除弃用的类;
[Change] Locker 删除不必要的代码;
[Change] PixivDownload 删除多余的Javadoc内容;
[Change] TimeLimitThreadPoolExecutorTest 调整日志输出方向, 补充测试细节;
2020-06-17 19:49:36 +08:00
acbd990181 [Version] 更新版本(2.5.2-20200611.1-SNAPSHOT -> 2.5.2-20200617.1-SNAPSHOT); 2020-06-17 16:48:47 +08:00
6db9cda08a Merge branch 'optimize-memory-cache' 2020-06-17 16:47:08 +08:00
1c742bfb6f [Fix] TimeLimitThreadPoolExecutor 修复了超时提醒遭到时限线程池中断的问题, 调整线程池仅发起一次中断; 2020-06-17 16:14:32 +08:00
2f30fe1696 [Change] CacheStoreCentral 调整PreLoadData的本地缓存时间, 添加注意事项; 2020-06-17 10:51:06 +08:00
32db952e63 Merge branch 'master' into optimize-memory-cache 2020-06-15 17:42:51 +08:00
bd6b825704 [Fix] pom.xml 排除slf4j-log4j12依赖项以修复package后应用启动报Slf4j实现冲突的问题; 2020-06-15 17:42:31 +08:00
3943963505 [Fix] BotEventHandler 修复线程池最大线程数与最低线程数错误, 导致无法启动应用的问题; 2020-06-15 17:40:23 +08:00
08822f68eb [Change] Locker 增加清理日志;
[Add] LockerMapTest 增加单元测试;
2020-06-15 17:18:28 +08:00
2f647ee9fa [Add] LockerMap, Locker 增加Locker锁对象和LockerMap锁对象存储;
[Change] CacheStoreCentral 将synchronized所使用的的锁对象由String(常量池)转换成Locker<K>以尝试减少内存占用;
2020-06-15 16:22:37 +08:00
44a7f49510 [Change] BotGlobal 补充IllegalStateException错误信息; 2020-06-15 15:48:07 +08:00
85088e1b2c [Change] RandomIntervalSendTimer 调整日志输出内容; 2020-06-13 23:10:01 +08:00
8dfb858b9f [Change] RedisPoolCacheStore 优化内部方法实现; 2020-06-13 17:24:06 +08:00
0e76cebc31 [Fix] PreLoadDataAttributeComparator 修复获取PreLoadData抛出的异常信息未记录到日志的问题; 2020-06-13 16:51:15 +08:00
c64320ad78 [Change] HttpRequestException 统一接口异常, 调整类所在包路径; 2020-06-13 16:44:52 +08:00
91e065f657 [Change] ImageCacheStore 优化中断处理, 自动取消任务, 补充TaskState设置; 2020-06-13 12:54:17 +08:00
91f8b0070f [Change] PreLoadDataAttributeComparator 更改类名; 2020-06-13 00:28:52 +08:00
ad54dbfbf3 [Change] PreLoadDataComparator, PreLoadDataAttribute 将PreLoadData的属性Enum迁移到单独的类;
[Change] BotCommandProcess 适配更改;
2020-06-12 22:28:14 +08:00
ebb3dea99e [Change] ConsoleMessageEvent, ConsoleMessageSender 将消息传递给Sender发送, 设置消息前缀格式;
[Change] ConsoleMain 调整私聊模式下的前缀;
[Change] MessageSource 规范化命名;
2020-06-12 20:13:36 +08:00
abcd26f21b [Fix] BotEventHandler 修复事件处理线程非预期设置的问题;
[Change] BotEventHandler 设置线程超时时间;
[Fix] CacheStoreCentral 整理'InterruptedException'在'getImageById'的传递路径;
2020-06-12 20:00:37 +08:00
d9b08f8ad9 [Change] BotCommandProcess, CacheStoreCentral 调整个别日志的输出级别; 2020-06-12 19:28:59 +08:00
5e030c12b2 [Change] BotCommandProcess 将各命令中'page'参数更名为'p'以简化命令;
[Change] PixivSearchLinkBuilder 更改类名(PixivSearchBuilder -> PixivSearchLinkBuilder);
[Add] PixivURL 增加接口常量;
[Change] Main, CacheStoreCentral, PixivSearchLinkBuilderTest 适配PixivSearchLinkBuilder的更改;
2020-06-12 17:06:04 +08:00
62eabce8f6 [Clear] PixivDownload, RandomIntervalSendTimer 整理代码; 2020-06-12 10:31:16 +08:00
64bca3c8f7 [Change] RandomIntervalSendTimer 更改Timer线程名; 2020-06-12 10:20:56 +08:00
3b3f97e638 [Fix] CacheStoreCentral 修复Search命令中'option'参数区分大小写的问题;
[Fix] BotCommandProcess 修复Search命令中遇到不存在作品时会中断处理的问题;
[Change] BotCommandProcess 调整Ranking命令的'type'参数默认值(ILLUST -> ALL);
2020-06-12 10:12:29 +08:00
951824cbe2 [Version] 更新版本(2.5.2-20200610.4-SNAPSHOT -> 2.5.2-20200611.1-SNAPSHOT); 2020-06-11 16:47:35 +08:00
e104abedeb [Add] MiraiMain 增加原始消息事件日志信息; 2020-06-11 16:46:32 +08:00
c3967d214d [Change] Dockerfile.sample 提升镜像使用的Jdk版本(8-jre -> 14 jdk), 增加Arthas诊断工具, 优化构建指令顺序; 2020-06-11 10:57:40 +08:00
26e377a2c7 [Change] log4j2.xml 调整日志输出到文件的限制;
[Change] 将内容较长的日志设为TRACE级别, 以减少日志占用;
[CLear] 整理代码;
2020-06-11 09:49:19 +08:00
87f2535b48 [Change] BotCommandProcess 优化Tag过滤表达式;
[Change] ImageCacheHandler 调整日志输出级别;
2020-06-11 09:08:58 +08:00
97d06c4fc3 [Change] CacheStoreCentral 调整代码以为后续更改做准备;
[Change] BotCommandProcess, ImageCacheHandler, PreLoadDataComparator, RandomRankingArtworksSender 适配CacheStoreCentral的更改;
2020-06-10 20:21:23 +08:00
bfe25c2012 [Version] 更新版本(2.5.2-20200610.3-SNAPSHOT -> 2.5.2-20200610.4-SNAPSHOT); 2020-06-10 16:39:01 +08:00
036f3eaf4a [Delete] simple.properties 删除弃用的Properties模板文件; 2020-06-10 16:27:13 +08:00
f07c8d0b76 [Add] MiraiMain 增加Bot网络重连相关设置; 2020-06-10 16:26:20 +08:00
d0cb4417b1 [Version] 更新版本(2.5.2-20200610.2-SNAPSHOT -> 2.5.2-20200610.3-SNAPSHOT); 2020-06-10 15:47:30 +08:00
3f256c5a0a [Fix] log4j2.xml 修复日志配置错误的问题;
[Change] BotGlobal, Main 调整日志输出内容和级别;
2020-06-10 15:46:33 +08:00
b9180b9651 [Change] MiraiMain 显性设置DeviceInfo为'randomDeviceInfo'; 2020-06-10 15:26:26 +08:00
7ccb306ca1 Merge pull request #15 from LamGC/change-license-AGPLv3
更改开源许可证(LGPLv3 -> AGPLv3)
2020-06-10 14:57:05 +08:00
5dd13ce088 [Change] pom.xml 增加'licenses'; 2020-06-10 14:49:40 +08:00
e9b33938fa [Add] README.md 增加'LICENSE'部分; 2020-06-10 14:41:12 +08:00
cc040e6ec9 [Change] 更改开源许可证; 2020-06-10 10:44:53 +08:00
04724168aa [Version] 更新版本(2.5.2-20200610.1-SNAPSHOT -> 2.5.2-20200610.2-SNAPSHOT); 2020-06-10 10:20:20 +08:00
89ded96336 [Change] 更改部分组件所使用的的工具类; 2020-06-10 10:17:05 +08:00
af0346e952 [Delete] net.sourceforge.htmlunit:htmlunit 删除未使用的依赖项;
[Delete] com.github.rholder:guava-retrying 删除未使用的依赖项;
[Delete] mysql:mysql-connector-java 删除未使用的依赖项;
2020-06-10 09:39:10 +08:00
4ccf2fafbc [Version] 更新版本(2.5.2-20200609.2-SNAPSHOT -> 2.5.2-20200610.1-SNAPSHOT); 2020-06-10 09:21:27 +08:00
f1e58d72ac [Change] log4j2.xml 调整mirai日志在STANDARD_STD*的输出级别; 2020-06-10 09:20:35 +08:00
9242a1d474 [Change] MiraiToSlf4jLoggerAdapter 调整类命名(MiraiToSlf4jLogger -> MiraiToSlf4jLoggerAdapter);
[Change] MiraiMain 适配调整;
2020-06-10 09:13:12 +08:00
065d21c4e4 [Update] MiraiToSlf4jLogger 补充类Javadoc; 2020-06-10 09:10:17 +08:00
5eab94c429 [Change] Console* 调整包路径; 2020-06-10 09:09:54 +08:00
2dd62bb6c8 [Version] 更新版本(2.5.2-20200609.1-SNAPSHOT -> 2.5.2-20200609.2-SNAPSHOT); 2020-06-09 16:02:03 +08:00
21613fe3c0 [Update] net.mamoe:mirai-core 更新依赖项版本(1.0-RC2-1 -> 1.0.2);
[Update] net.mamoe:mirai-core-qqandroid 更新依赖项版本(1.0-RC2-1 -> 1.0.2);
2020-06-09 15:51:33 +08:00
ca56b2c9ba [Change] MiraiToSlf4jLogger, log4j2.xml, log4j2-test.xml, MiraiMain 将Mirai框架的日志接入Slf4j(Log4j2);
[Change] BotEventHandler 调整日志使用;
2020-06-09 15:35:43 +08:00
e0f773639f [Change] ConsoleMain 调整Terminal获取方式; 2020-06-09 14:16:26 +08:00
75aa78a3d7 [Version] 更新版本(2.5.2-20200606.1-SNAPSHOT -> 2.5.2-20200609.1-SNAPSHOT); 2020-06-09 10:12:02 +08:00
18a8ad95a1 [Change] PixivUgoiraBuilderTest 调整测试细节; 2020-06-09 10:07:54 +08:00
438d0a95d3 [Add] pom.xml 增加JLine库;
[Update] ConsoleMain 增强Cli使用体验;
[Add] BotEventHandler 增加同步执行命令的方法'executeMessageEvent(MessageEvent, boolean)';
[Change] Main 适配ConsoleMain的更改;
2020-06-09 09:50:16 +08:00
6789b5b7c5 [Change] CacheStoreCentral 调整'getImageById'方法中对'pageIndex'的参数值检查时机;
[Change] MiraiMessageSender 增加警告忽略注释;
2020-06-08 19:13:51 +08:00
ad289f952f [Fix] log4j2.xml 修复SYSTEM_OUT级别限定错误的问题; 2020-06-08 18:47:53 +08:00
3ae0e4cd8d [Change] HotDataCacheStore 调整日志输出级别(DEBUG -> TRACE); 2020-06-08 16:08:13 +08:00
5550c7aef1 [Clear] CacheStoreCentral 删除无用的运行器代码; 2020-06-08 15:57:38 +08:00
d4d3432c76 [Add] PixivUgoiraBuilder 增加方法'buildUgoira(OutputStream, boolean)'以提供输出流给Builder输出已构建的动图数据;
[Add] PixivUgoiraBuilder 增加方法'getUgoiraMeta():JsonObject'方法, 可通过该方法获取动图元数据;
2020-06-08 15:30:22 +08:00
d1aeda012e [Fix] BotCommandProcess 修复因'getImageById'迁移导致'image'命令不可用的问题;
[Clear] CacheStoreCentral 清除无用代码;
2020-06-08 09:37:06 +08:00
683a38bc17 [Add] BotGlobal BotGlobal在初始化时将检查Redis连通性;
[Change] RandomRankingArtworksSender 支持外部设置groupId, 以使用群组配置;
[Change] BotCommandProcess, BotAdminCommandProcess 适配RandomRankingArtworksSender更改;
2020-06-08 09:24:17 +08:00
188309509b [Change] BotCommandProcess 简化Search参数'ContentOption'的名称('contentOption' -> 'option'); 2020-06-06 19:27:55 +08:00
3915712337 [Delete] search.txt 删除旧文档; 2020-06-06 17:57:24 +08:00
1e88ba70dd [Version] 更新版本(2.5.2-20200604.3-SNAPSHOT -> 2.5.2-20200606.1-SNAPSHOT); 2020-06-06 17:54:09 +08:00
e6b2544998 [Update] 更新接口文档; 2020-06-06 17:53:37 +08:00
a2f6f1d140 [Clear] BotCommandProcess 整理代码; 2020-06-06 17:07:40 +08:00
f54ed35a09 [Change] SettingProperties 增加群号检查, 防止出现非法群号; 2020-06-06 15:42:35 +08:00
a426f80ec5 [Fix] BotCommandProcess 修复Search命令对作品限制的管理不受'image.allowR18'选项控制的问题; 2020-06-06 11:50:29 +08:00
c1a21d1065 [Change] ConsoleMain, ConsoleMessageEvent 支持启动时设置会话群组Id和QQId; 2020-06-06 10:52:12 +08:00
e570ddbb53 [Change] PreLoadDataComparator 增加对JsonElement是否为JsonObject的检查; 2020-06-06 10:51:24 +08:00
223d78dbd6 [Change #5] 优化'getImageById'从文件读入缓存时, 检查图片完整性的速度; 2020-06-05 17:03:41 +08:00
ef5651be47 [Fix] CacheStore 修正Javadoc中的错误描述; 2020-06-05 16:29:16 +08:00
9a8aac1960 [Fix] BotGlobal 修复BotGlobal初始化失败的问题;
[Fix] BotCommandProcess 修复Search命令未找到相关作品时提示语无法触发的问题;
[Change] BotCommandProcess 调整'isNoSafe(int, Properties, boolean)'方法中对于作品限制的判断方式;
2020-06-05 16:23:13 +08:00
4bbed5fd55 [Change] BotCommandProcess, CacheStoreCentral 将ImageFile缓存管理转移到CacheStoreCentral;
[Change] RandomRankingArtworksSender 适配修改;
[Change] BotCommandProcess, BotGlobal 将'imageStoreDir'由BotCommandProcess转移到BotGlobal;
2020-06-05 10:07:13 +08:00
bcc21149b9 [Change #10] 将缓存存取部分从BotCommandProcess分离;
[Change] 整理代码;
2020-06-05 09:53:30 +08:00
e93c322c02 [Change] BotGlobal, BotCommandProcess 调整PixivDownload的初始化过程;
[Change] BotCommandProcess 将'Search'所属缓存部分抽出到单独的方法('getSearchBody');
2020-06-04 20:14:57 +08:00
feb51b8534 [Change] BotGlobal, BotCommandProcess 将'Gson'和'PixivDownload'对象纳入BotGlobal; 2020-06-04 19:47:20 +08:00
273dbd45b0 [Version] 更新版本(2.5.2-20200604.2-SNAPSHOT -> 2.5.2-20200604.3-SNAPSHOT); 2020-06-04 16:16:53 +08:00
b39a82b936 [Fix] BotGlobal 修复BotGlobal初始化失败的问题; 2020-06-04 16:15:27 +08:00
4589624b9b [Version] 更新版本(2.5.2-20200604.1-SNAPSHOT -> 2.5.2-20200604.2-SNAPSHOT); 2020-06-04 15:58:56 +08:00
033fd0188b [Change] 调整Logger创建方式(getLogger(String) -> getLogger(Class)); 2020-06-04 15:58:10 +08:00
4318869846 [Change] 将'Proxy', 'CookieStore'转移到BotGlobal类; 2020-06-04 15:54:52 +08:00
749b89e668 [Version] 更新版本(2.5.2-20200522.1-SNAPSHOT -> 2.5.2-20200604.1-SNAPSHOT); 2020-06-04 11:26:26 +08:00
9ce18469e6 [Add] 增加控制台框架以进行本地测试, 或非聊天机器人使用; 2020-06-04 11:25:08 +08:00
e803d2161b [Add] PropertiesUtils 增加一个工具类; 2020-06-04 11:24:40 +08:00
3fbf80f233 Merge remote-tracking branch 'origin/master' 2020-06-04 11:20:22 +08:00
8b8ef7e744 [Change] Issue #12 调整框架启动机器人应用的方式, 增加一个用于机器人应用内部共享数据的类;
[Fix] BotEventHandler, ImageCacheStore 增加ShutdownHook用于关闭线程池, 解决线程池阻塞关闭过程的问题;
[Change] BotEventHandler 调整'match(String)'方法;
[Change] BotAdminCommandProcess 调整'savePushList()'方法对文件创建失败的行为;
2020-06-04 11:19:18 +08:00
7fbdf28ec8 [Update] Bug_Report.md 增加默认标签; 2020-06-04 10:50:37 +08:00
b29482927c [Add] Bug_Report.md 添加Bug反馈模板; 2020-06-04 10:47:28 +08:00
69da2b02ac [Fix #11] 修复在图片缓存失效的情况下, 'getImageToBotCode'依然会尝试从缓存获取图片File对象导致NPE;
[Change] BotCommandProcess 调整字符串拼接形式, 统一错误提示语的格式;
2020-06-04 09:29:53 +08:00
eb2de09859 [Change] RandomIntervalSendTimer 将随机间隔发送器的Timer设为daemon thread; 2020-06-03 19:37:45 +08:00
ef70ac77cb [Update] HotDataCacheStore, Cleanable 更新Javadoc; 2020-06-03 17:03:54 +08:00
b4e9fdab7d [Change] TimeLimitThreadPoolExecutor 整理代码, 清除无用参数;
[Update] Pixiv排行榜接口.md 更新文档内容, 补充返回数据信息;
2020-06-03 17:02:46 +08:00
5c5827123d [Update] net.lamgc:java-utils 更新依赖项版本(1.2.0_20200514.1-SNAPSHOT -> 1.2.0_20200517.1-SNAPSHOT), 以修复作品无效检测不生效的问题; 2020-06-03 16:51:26 +08:00
2bfb78304c [Change] HotDataCacheStore, RandomIntervalSendTimer 支持浮动时间参数为0; 2020-06-03 14:50:45 +08:00
637ea46b9a [Fix] MiraiMessageEvent 修复群消息事件错误设置私聊Sender的问题; 2020-06-01 21:02:46 +08:00
f5140a7a1e [Change] SpringCQMessageSenderFactory 尝试通过ThreadLocal获取当前处理的CoolQ对象, 目前EventExecutor暂不支持, 仅预留;
[Change] CQPluginMain 调整'processMessage'方法修饰符, 补充Javadoc;
2020-06-01 20:15:59 +08:00
0fafddc715 [Change] AdultContentDetector, PixivURL 整理代码, 补充Javadoc; 2020-06-01 16:12:33 +08:00
4c03a0f7d9 [Change] HotDataCacheStore 调整'supportedList'的条件, HotDataCacheStore将支持使用ListCacheStore;
[Change] AutoSender 将AutoSender的messageSender增加final属性;
2020-06-01 16:01:17 +08:00
ee02072b2d [Deprecated] PixivURL 将旧版Search接口设为弃用; 2020-06-01 15:58:54 +08:00
3f6c87da3c [Add] MiraiMain 增加对临时消息的支持;
[Change] MiraiMessageEvent 调整消息对象转换的方式, 通过静态方法转换Mirai的MessageEvent;
2020-06-01 15:55:31 +08:00
d45cd8aab5 [Update] 更新文档, 增加排行榜接口文档, 待补充返回数据; 2020-05-28 16:20:16 +08:00
598d6ef205 [Fix] README.md 修复格式错误;
[Update] README.md 补充参数细节;
2020-05-27 19:30:04 +08:00
2190ef1b59 [Change] README.md 调整Readme内容以减少对用户选择带来的影响; 2020-05-27 19:27:29 +08:00
fcc522c057 Merge branch 'master' of github.com:LamGC/ContentGrabbingJi 2020-05-26 14:45:01 +08:00
5a81604d94 [Change] BotCommandProcess 调整'色图'命令处理方法的命名; 2020-05-26 14:31:12 +08:00
376306e404 [Update] README.md 更新文档说明, 提供详细命令说明和注意事项; 2020-05-26 10:37:40 +08:00
0fe69253ed [Change] .gitignore 将Idea项目文件由指定文件名改为匹配后缀名; 2020-05-26 10:28:46 +08:00
fe8bf16d3a [Clean] RandomRankingArtworksSender 清理无用导入;
[Change] AutoCleanTimer 统一LoggerName名称;
2020-05-25 20:23:14 +08:00
99c66babec [Add] BotEventHandler 增加对'Sender.sendMessage(String)'返回值的处理; 2020-05-25 09:45:09 +08:00
cb8b01fd74 Merge branch 'master' of github.com:LamGC/ContentGrabbingJi 2020-05-25 09:25:49 +08:00
1f48b3ffdc [Change] MessageSender, MessageEvent 允许'senderMessage(String)'抛出异常;
[Change] RandomRankingArtworksSender 调整异常捕获日志输出;
[Change] MiraiMessageEvent, MiraiMessageSender, SpringCQMessageEvent 适配更改;
2020-05-25 09:21:46 +08:00
68feed8d3c Merge remote-tracking branch 'origin/master' 2020-05-22 23:09:39 +08:00
d89711b0b9 [Version] 更新版本(2.5.2-20200520.1-SNAPSHOT -> 2.5.2-20200522.1-SNAPSHOT); 2020-05-22 20:42:29 +08:00
a3376e96ee [Add] BotCommandProcess, BotEventHandler, PixivDownload 对作品Id不存在的情况做出反应(而不是作为内部异常反馈); 2020-05-22 20:39:49 +08:00
33d18cef6b Merge remote-tracking branch 'origin/master' 2020-05-22 20:34:32 +08:00
7aa00ff98b [Fix] BotCommandProcess 修复ImageStore不允许使用软链接的问题; 2020-05-22 20:34:15 +08:00
e956e36584 [Fix] RankingUpdateTimer 修复首次执行时间计算错误的问题; 2020-05-22 20:23:46 +08:00
8e27221457 [Change] BufferMessageEvent 调整成员变量修饰符, 补充新的构造方法'<init>(int, int, String)'; 2020-05-22 08:49:48 +08:00
a5fca68ef5 [Change] MessageEventExcutionDebugger 在配置文件无法转换的情况下不再忽略'NumberFormatException'异常; 2020-05-20 15:31:43 +08:00
8edb728fe9 [Update] README.md 补充Readme内容 2020-05-20 14:34:55 +08:00
cc05af8a24 Create README.md 2020-05-20 11:00:25 +08:00
60fa641962 [Version] 更新版本(2.5.2-20200517.1-SNAPSHOT -> 2.5.2-20200520.1-SNAPSHOT); 2020-05-20 10:36:30 +08:00
3ab373cc70 [Update] .gitignore 补充VSCode忽略项; 2020-05-20 10:34:55 +08:00
461cd246d8 [Add] BotCommandProcess 新增'色图'功能以随机从排行榜中获取一张图片;
[Add] BufferMessageEvent 增加用于获取Sender发送内容的Event实现;
[Change] BotEventHandler Handler现在允许命令处理方法不返回消息(返回'null');
2020-05-20 09:55:50 +08:00
f040f130d7 [Update] Main, AutoArtworksSender, ImageCacheStore, PixivAccessProxyServer, PagesQualityParser 整理代码问题, 删除无用类; 2020-05-20 09:05:50 +08:00
1f402fbbac [Change] LocalHashCacheStore 实现'Cleanable'接口, 增加'<init>(int, int, boolean)'构造函数以允许自动清理
[Update] LocalHashCacheStore 补充Javadoc内容;
[Update] .gitignore 补充并整理忽略项;
2020-05-20 08:56:27 +08:00
f7f3c3beaf [Fix] BotCommandProcess 修复help命令中帮助信息的错误; 2020-05-17 19:38:17 +08:00
04c1753c22 [Version] 更新版本(2.5.1 -> 2.5.2-20200517.1-SNAPSHOT); 2020-05-17 17:45:06 +08:00
d993e9d719 [Add] MiraiMessageSender 增加对默认表情的支持; 2020-05-12 15:25:29 +08:00
60e91987d1 Merge branch 'mirai' 2020-05-12 15:14:58 +08:00
65392fc2fe [Fix] net.mamoe:mirai-core, net.mamoe:mirai-core-qqandroid 更新Mirai-Core版本号以修复VerifyError问题(1.0-RC2 -> 1.0-RC2-1);
[Fix] MiraiMessageSender 调整日志输出, 以修复日志打印处理后消息顺序混乱的问题;
2020-05-12 15:14:28 +08:00
d38934a0f4 [Change] RankingUpdateTimer 调整时间补齐的阀值(12:00 -> 11:30);
[Add] documents/interfaces/* 整理部分Pixiv接口的文档;
2020-05-12 15:03:08 +08:00
7943357d96 [Fix] MiraiMain 修复未登录时异常退出, 抛出NPE的问题; 2020-05-11 12:16:35 +08:00
a170ff040b [Update] net.mamoe:mirai-core, net.mamoe:mirai-core-qqandroid 更新依赖项版本(0.39.4 -> 1.0-RC2);
[Update] MiraiMain, MiraiMessageEvent 适配mirai新版本;
[Change] MiraiMessageSenderFactory 增加参数检查并优化过程;
[Change] MiraiMain 显性指定Mirai所用协议为'BotConfiguration.MiraiProtocol.ANDROID_PAD';
2020-05-11 11:36:37 +08:00
cb2ebfdb73 [Change] MessageEvent 对消息内容删除首尾空格以提高用户体验性; 2020-05-09 23:13:53 +08:00
66b22c543a [Fix] SettingProperties 修复了配置文件无法创建, 导致无法保存配置的问题;
[Fix] Main 修复了潜在的文件创建失败的问题;
2020-05-09 22:54:32 +08:00
6bace4b048 [Version] 更新版本(2.5.0 -> 2.5.1); 2020-05-09 22:35:46 +08:00
597aac4e95 [Fix] BotCommandProcess 修复部分指令无法使用群组配置的问题; 2020-05-09 22:35:21 +08:00
181d60285b [Version] 更新版本(2.5.0-20200509.1-SNAPSHOT -> 2.5.0); 2020-05-09 22:20:07 +08:00
c06b8717c3 [Add] CacheStoreUtils 增加用于缓存库的工具类;
[Update] BotCommandProcess 补充注释, 优化"info"命令输出内容的格式;
2020-05-09 22:19:29 +08:00
b4a28b6735 [Version] 更新版本(2.5.0-20200506.1-SNAPSHOT -> 2.5.0-20200509.1-SNAPSHOT); 2020-05-09 18:16:54 +08:00
40057c3683 [Fix] Issue #1 修复了ImageCache对重复缓存请求的策略(忽略并返回)导致的后续处理发生NPE的问题;
[Update] ImageCache 更新了ImageCache, 将该部分单独分离到 ImageCacheStore;
[Update] BotCommandProcess 适配新的ImageCache, 调整内容检查的时机以减少不必要的API访问;
2020-05-09 18:16:16 +08:00
4beb4d78fb [Fix] BotCommandProcess 修复Search命令格式不正确的问题; 2020-05-09 12:52:15 +08:00
d9edaa681f [Add] BotAdminCommandProcess 为 addPushGroup 增加参数检查; 2020-05-09 10:10:48 +08:00
7b52abde60 [Change] BotCommandProcess 增加Search命令的反馈内容(可能必须要QQ支持长文本才能使用); 2020-05-08 23:31:24 +08:00
2cb4fe1dbc [Fix] SettingProperties 修复无法设置全局配置项的问题; 2020-05-08 12:23:43 +08:00
5830327dad [Add] BotEventHandler 增加禁言记录, 防止应用因过多的在禁言状态反馈导致被封号;
[Update] MiraiMain 适配禁言记录功能;
2020-05-07 19:46:54 +08:00
cbd10ff281 [Fix] Issue #4 修复因Mirai API变动导致应用无法从消息获取ImageId的问题; 2020-05-07 18:24:09 +08:00
84b8006f89 [Change] RankingUpdateTimer 将定时更新时间调整为"+8 11:30", 并增加日期指定; 2020-05-07 15:21:05 +08:00
d32d891ad7 [Update] BotCommandProcess 补充Javadoc内容;
[Add] BotCommandProcess 在Ranking命令增加force参数以允许强制查询预计不在排行榜范围的日期;
2020-05-07 11:29:31 +08:00
aabd35ce71 [Add] AutoCleanTimer, Cleanable 增加缓存库自动清理;
[Update] HotDataCacheStore 增加对自动清理的支持;
[Change] RedisPoolCacheStore 将Redis缓存库抽象类设为default(原public);
[Change] MiraiMessageSender, BotCommandProcess 适配HotDataCacheStore的修改;
[Change] BotCommandProcess 调整ranking命令的输出格式;
2020-05-07 10:26:11 +08:00
a36eb713d7 [Version] 更新版本(2.5.0-20200505.1-SNAPSHOT -> 2.5.0-20200506.1-SNAPSHOT); 2020-05-06 11:09:18 +08:00
f452c1c128 Merge branch 'add-MultiProperties' 2020-05-06 11:08:16 +08:00
cb42aadb15 [Fix] SettingProperties 修复在初次设置群组配置时未创建该群组所属Properties导致NPE的问题;
[Update] BotAdminCommandProcess 更新日志内容及命令返回信息;
[Change] BotCommandProcess 调整配置项key格式;
2020-05-06 10:55:32 +08:00
b969bb29b2 Merge branch 'master' into add-MultiProperties 2020-05-06 10:18:00 +08:00
0d74007a98 [Change] search.txt 整理接口相关文档的存储目录; 2020-05-06 09:42:34 +08:00
edade24883 [Change] Issue #2 调整Logger Name以统一Logger Name格式; 2020-05-06 09:34:28 +08:00
34f57404ca [Update] 重构Setting Properties, 使其支持分群配置; 2020-05-06 00:57:48 +08:00
99b6e14ff7 [Change] RandomIntervalSendTimer 调整日志输出内容; 2020-05-05 17:09:45 +08:00
c8fe2c3fdd [Version] 更新版本(2.5.0-20200504.2-SNAPSHOT -> 2.5.0-20200505.1-SNAPSHOT);
[Update] net.lamgc:java-utils 更新依赖版本(1.1.0 -> 1.2.0_20200505.1-SNAPSHOT);
2020-05-05 01:29:11 +08:00
49a33d4078 [Add] MessageEventExecutionDebugger 添加对消息处理的调试器Enum;
[Add] BotEventHandler 添加对 MessageEventExecutionDebugger 的支持;
[Add] VirtualLoadMessageEvent 增加 toVirtualLoadMessageEvent(MessageEvent) 方法;
2020-05-05 01:27:10 +08:00
cd1d2316ee [Add] BotEventHandler 添加executeMessageEvent(MessageEvent)方法;
[Change] BotEventHandler 将BotEventHandler.executor设为private;
[Change] MiraiMain, CQPluginMain, RankingUpdateTimer 适配BotEventHandler的调整
2020-05-04 23:06:07 +08:00
cf08353ed9 [Change] BotEventHandler 调整事件处理线程池的线程数(min: 4, Max: 32); 2020-05-04 22:52:47 +08:00
bef6a684b9 [Change] BotEventHandler 调整事件处理线程池的线程数; 2020-05-04 22:46:06 +08:00
04960889b4 [Version] 更新版本(2.5.0-20200504.01-SNAPSHOT -> 2.5.0-20200504.2-SNAPSHOT);
[Update] net.lamgc:java-utils 更新依赖版本(1.1.0_5-SNAPSHOT -> 1.1.0);
[Update] net.lamgc:PixivLoginProxyServer 更新依赖版本(1.1.0 -> 1.1.1);
2020-05-04 19:44:11 +08:00
0dc8fc78b4 [Version] 更新版本(2.4.0 -> 2.5.0-20200504.01-SNAPSHOT); 2020-05-04 17:07:47 +08:00
40c5284be2 [Change] Dockerfile.sample 调整构建指令顺序以充分利用构建缓存;
[Fix] Main 修复SystemProperties设置null抛出NPE的问题;
2020-05-04 17:07:39 +08:00
46cb47c7d1 [Add] LICENSE 添加开源许可证(LGPLv3); 2020-05-04 16:55:16 +08:00
fe213deecb [Change] Main 调整参数的接收形式;
[Fix] Dockerfile.sample 修复Java应用无法接收stop命令发送信号的问题;
[Change] RankingUpdateTimer 调整更新投递形式, 将同步更新调整为异步更新;
[Change] BotEventHandler 实装TimeLimitThreadPoolExecutor;
2020-05-04 02:07:46 +08:00
f279d99fda [Add] TimeLimitThreadPoolExecutor 增加一个带有执行时间限制的线程池及对应单元测试类; 2020-05-04 02:04:59 +08:00
a09aef5be2 [Delete] 删除已完成或已经没有存在意义的TODO; 2020-05-02 10:40:11 +08:00
766411fa09 [Fix] Dockerfile.sample 修复应用无法接收容器退出信号的问题(现在应用将可以接受信号主动退出, 即使ExitCode不为0); 2020-05-02 00:56:05 +08:00
84b544cea9 [Change] Main 调整日志输出级别, 补充日志输出内容;
[Change] Dockerfile.sample 调整镜像内应用运行路径;
2020-05-01 22:28:15 +08:00
2f492f5b03 [Add] log4j2.xml, log4j2-test.xml 添加允许指定日志存储目录的功能(sys:cgj.logsPath); 2020-05-01 13:19:08 +08:00
15af939c3f [Fix] PixivDownload 修复因排行榜总量的不确定导致排行榜获取异常的问题;
[Change] BotEventHandler 调整事件处理完成后的日志输出形式;
2020-04-30 19:08:34 +08:00
a28cb142b4 [Change] 调整数据存储的路径设置及存储目录参数名; 2020-04-30 17:51:57 +08:00
e8fda3214b [Version] 更新版本(2.3.0 -> 2.4.0); 2020-04-30 02:24:21 +08:00
fba6d3532c [Delete] 删除不再使用的类和方法;
[Update] 优化Javadoc并设置编译注解, 优化代码;
[Change] 调整部分日志的输出级别;
[Change] RedisPoolCacheStore clear方法将根据返回信息确定是否清空成功;
[Change] Dockerfile.sample 调整镜像构建步骤;
2020-04-30 02:23:31 +08:00
0075446412 [Fix] MiraiMessageSender 修复因BotCode-Image中flashImage参数不存在导致NPE的问题; 2020-04-30 01:54:27 +08:00
2388cd419e [Add] Dockerfile.sample 增加一个用于构建CGJ-BotMode镜像的Dockerfile模板; 2020-04-30 00:45:41 +08:00
7f83b16118 [Add] Main 添加日志输出内容; 2020-04-29 23:44:38 +08:00
10cad32efa [Version] 更新版本(2.2.1 -> 2.3.0); 2020-04-28 23:16:10 +08:00
22a113ef38 [Change] 调整CGJ包结构, 将机器人平台相关类单独存放在framework包内; 2020-04-28 23:15:55 +08:00
6824b12e8f [Add] MiraiMessageSender 增加对闪照的支持; 2020-04-28 10:57:17 +08:00
1c556c5b94 [Update] MiraiMessageSender 对已上传的图片所属ImageId设置10天有效期; 2020-04-28 00:32:40 +08:00
dbc9f4c90b [Fix] BotCommandProcess 修复了通过缓存获取排行榜时排行榜从第2名开始读取的bug;
[Update] MiraiMain, MiraiMessageSender, MiraiMessageEvent 更新Mirai框架(0.32.0 -> 0.39.4)并适配框架新版本的变动;
2020-04-28 00:19:24 +08:00
d3c1975722 [Fix] PixivURL 修复由于智障修复导致的bug; 2020-04-27 23:26:22 +08:00
c09b750fe6 [Version] 更新版本(2.2.0 -> 2.2.1); 2020-04-27 14:13:51 +08:00
ca479ef1af [Change] log4j2.xml 调整SYSTEM_OUT控制台输出配置; 2020-04-27 14:06:33 +08:00
c2c49d2355 [Add] PixivDownload 增加对排行榜获取失败抛出异常的信息;
[Fix] PixivURL.RankingContentType 修复isSupportedMode判断错误的问题;
[Fix] RankingUpdateTimer 调整每日更新时间(12:10 -> 12:30)以尝试修复排行榜更新时机过早的问题;
2020-04-27 12:32:25 +08:00
22e74e8cd5 [Add] PixivIllustType 增加了对IllustInfo的解析; 2020-04-26 15:53:39 +08:00
d549c5674d [Update] BotCommandProcess 补充Javadoc;
[Change] PixivUgoiraBuilder 调整日志输出级别;
2020-04-26 15:45:23 +08:00
c6952de84c [Add] log4j2-test.xml 增加测试运行时的日志配置文件;
[Add] pom.xml 设置package goal的文件排除项, 将测试运行所使用的日志配置文件(log4j2-test.xml)排除;
2020-04-25 17:07:13 +08:00
410d6c0828 [Add] BotAdminCommandProcess, PixivDownload 增加日志输出内容;
[Change] RandomIntervalSendTimer 对AutoSender增加异常捕获以减少对其他Timer执行的影响;
2020-04-25 17:02:01 +08:00
aaa1bc932b [Add] RandomRankingArtworksSender 增加参数;
[Add] BotAdminCommandProcess 增加更多可自定义的参数项, 增加json配置项常量;
2020-04-25 02:29:38 +08:00
a1e54e70d9 [Change] PixivURL 调整RankingContentType.ALL名称(ALL -> TYPE_ALL);
[Add] PixivDownload 增加调试日志信息;
[Fix] RandomIntervalSendTimer 修复cancel方法无法停止定时器的问题;
2020-04-24 23:54:26 +08:00
16522155e1 [Add] 通过增加参数信息来为命令方法提供来源群和来源QQ; 2020-04-24 23:52:05 +08:00
5f796f7da0 [Update] BotCommandProcess 更新帮助信息, 补充Javadoc, 调整命令名;
[Change] RandomIntervalSendTimer 将下一执行时间从分钟调整为具体时间;
[Change] MessageEvent toString方法调整为输出具体实现类名而不是"MessageEvent";
2020-04-24 10:45:03 +08:00
5a52dd9208 [Add] 初步添加成人内容检测器, 尚未使用; 2020-04-24 01:42:47 +08:00
b53aafa81b [Fix] RandomRankingArtworksSender 修复排行榜选取错误的问题;
[Add] RandomRankingArtworksSender 增加对rankingStart, rankingStop的范围选取检查;
[Update] RandomRankingArtworksSender 更新Javadoc;
2020-04-24 01:41:53 +08:00
28aa086f15 [Version] 更新版本(2.1.0 -> 2.2.0);
[Change] groupId纳入net.lamgc;
2020-04-24 01:28:06 +08:00
0eadefa74f [Add] BotAdminCommandProcess 完善作品报告管理功能;
[Add] BotCommandProcess 对Report增加报告时间;
2020-04-24 01:24:03 +08:00
4afa414725 [Add] CacheStore 增加keys, remove方法;
[Update] HotDataCacheStore, LocalHashCacheStore, RedisPoolCacheStore 适配CacheStore的更改;
2020-04-24 01:23:11 +08:00
0f202cb076 [Update] RedisPoolCacheStore 优化代码; 2020-04-24 00:42:41 +08:00
1f3d99ac10 [Add] BotCommandProcess 增加作品报告功能, 以允许用户主动报告不当作品;
[Update] 适配报告功能;
2020-04-24 00:34:50 +08:00
d33f4028d1 [Fix] BotEventHandler 修复无命令帮助信息无法发送的问题(前缀识别失效); 2020-04-24 00:33:51 +08:00
4020bbfea8 [Change] BotAdminCommandProcess 调整方法名, 增加提示信息; 2020-04-23 23:18:41 +08:00
19605a9401 [Add] BotEventHandler 增加启动预处理方法;
[Add] BotAdminCommandProcess 增加推送功能相关管理命令;
[Change] RandomIntervalSendTimer 调整Timer管理过程;
[Change] BotCommandProcess ranking方法对参数错误的处理调整为返回错误信息;
[Fix] RankingUpdateTimer 修复参数错误的问题;
2020-04-23 16:23:06 +08:00
21466a49f9 [Update] pom.xml 补充插件版本;
[Delete] pom.xml 删除kotlin编译插件;
2020-04-23 09:23:26 +08:00
73ae9a268b [Add] MessageSenderBuilder, MessageSenderFactory 增加用于创建MessageSender而又与平台无关的Builder, 初步增加对应Factory; 2020-04-21 22:43:24 +08:00
620c3785ad [Change] MessageSender 调整包路径; 2020-04-21 22:33:20 +08:00
2df5513727 [Change] BotEventHandler 修改命令前缀以防止错误的触发; 2020-04-20 01:35:02 +08:00
8798633c2c [Fix] BotCommandProcess 修复排行榜数据选取错误的问题; 2020-04-20 01:33:30 +08:00
9dfc20a525 [Add] AutoSender 增加自动发送器接口;
[Add] RandomIntervalSendTimer 增加随机延迟自动发送器;
[Add] RecommendArtworksSender 增加随机排行榜作品发送器;
[Change] PixivDownload 重新开发 getRanking 方法;
2020-04-20 01:32:11 +08:00
dd88f2acab [Update] MiraiMessageSender 增加新构造器, 补充Javadoc; 2020-04-19 23:58:33 +08:00
36460e4c34 [Add] MiraiMain 增加正常关闭过程; 2020-04-19 13:03:10 +08:00
084be3970a [Change] 将各MessageEvent的消息发送部分分离成单独的MessageSender, 将MessageEvent依赖于对应的MessageSender; 2020-04-19 12:23:02 +08:00
11005b0f6c [Add] BotCode 增加 getCodePattern 方法以允许其他类获取BotCode所使用的正则匹配表达式; 2020-04-19 12:20:58 +08:00
970be847a0 [Change] RankingUpdateTimer 调整更新时间(12:30 -> 12:10);
[Change] RankingUpdateTimer 调整日志级别, 补充日志输出信息;
2020-04-19 12:16:03 +08:00
09fa1bd2e8 [Add] MessageSource 将MiraiMessageEvent的MessageSource分离成独立的类;
[Change] SpringCQMessageEvent, MiraiMessageEvent 适配新的更改;
2020-04-19 01:34:45 +08:00
ae27141fea [Add] MessageSender 增加消息发送器接口;
[Change] MessageEvent 实现 MessageSender 接口;
2020-04-19 01:00:52 +08:00
b328def8f9 [Change] BotCommandProcess 将isNoSafe方法公开;
[Change] MiraiMessageEvent 将UploadImage方法公开;
[Add] MiraiMessageEvent 增加MessageSource枚举类;
2020-04-19 00:31:24 +08:00
dbfed874c0 [Add] VirtualLoadMessageEvent 增加假负载消息事件, 该事件在消息发送上将没有操作, 纯属执行命令;
[Change] RankingUpdateTimer 调整排行榜更新方式, 利用假负载消息触发一次完全执行;
2020-04-18 01:15:04 +08:00
f02b0e9e98 [Update] 补充Javadoc, 设置@SuppressWarning注解; 2020-04-18 01:13:01 +08:00
260cfe3dd0 [Delete] MessageEvent 删除可能会导致兼容性降低的未使用方法;
[Delete] MiraiMessageEvent, SpringCQMessageEvent 适配 MessageEvent 的更改;
2020-04-17 18:29:59 +08:00
c1427379c6 [Add] StringListRedisCacheStore 增加类型为String的RedisListCacheStore实现;
[Change] BotCommandProcess 更换pagesCache的缓存组件(RedisPoolCacheStore<List<String>> -> StringListRedisCacheStore);
2020-04-17 18:25:48 +08:00
f844d150e8 [Change] 更改artifactId(CGJ_2 -> ContentGrabbingJi); 2020-04-17 17:46:42 +08:00
2ec696b3cd [Change] PreLoadDataComparator Attribute类属性attrName访问权修改(default -> public); 2020-04-17 17:46:01 +08:00
c8c6dbe4fd [Update] 优化Logger名称;
[Update] 为 processMessage(MessageEvent) 方法增加 @SuppressWarnings("unused") 注解并补充Javadoc;
2020-04-17 17:20:37 +08:00
4dff477ded [Change] log4j2.xml 调整Root Logger最低日志级别(Debug -> Info); 2020-04-17 15:54:13 +08:00
103 changed files with 7121 additions and 2295 deletions

35
.github/ISSUE_TEMPLATE/Bug_Report.md vendored Normal file
View File

@ -0,0 +1,35 @@
---
name: Bug 反馈
about: 使用这个模板反馈应用问题。
title: ''
labels: bug
assignees: ''
---
## 环境信息 ##
系统(例如: Windows 10 1909): ``
Java版本(例如: Oracle Jdk 8.242): ``
<!-- 如果直接使用发布版那么就填写发布版版本号v开头 -->
<!-- 如果你通过编译运行的方式运行开发版请你填写运行所使用的Commit Id -->
发生问题所在的版本: ``
## 问题描述 ##
<!-- 尽可能的清晰描述问题的信息 -->
## 预期行为 ##
<!-- 你觉得正常情况下应该会发生什么? -->
## 实际行为 ##
<!-- 实际上这个功能做了什么? -->
## 复现步骤 ##
<!-- 将复现步骤详细的写出,如果是偶发问题,可以写已知的,触发几率高的方法 -->
1.
## 相关信息 ##
### 日志 ###
<!-- 如果日志涉及了问题,请务必将日志一同提交,这对排查问题非常有用 -->
```
```

View File

@ -0,0 +1,11 @@
---
name: 功能/想法 提议
about: 使用这个模板将你对应用的想法提出来,或许我们会采纳!
title: ''
labels: function, question
assignees: ''
---
<!-- 如果可以,尽可能的清晰、详细的表达你的想法 -->
<!-- 没关系的,我们会进一步向你交流以尝试了解你的想法! -->

14
.gitignore vendored
View File

@ -1,6 +1,16 @@
# Ignore test date folder
/pluginData/
/logs/
/.idea/
/CGJ_2.iml
/cookies.store
/target/
# Ignore Idea files
/.idea/
*.iml
# Ignore Visual Studio Code files
.classpath
.factorypath
.project
/.settings/
/.vscode/

11
Dockerfile.sample Normal file
View File

@ -0,0 +1,11 @@
FROM openjdk:14
COPY --from=hengyunabc/arthas:latest /opt/arthas /opt/arthas
ENV jarFileName=ContentGrabbingJi-exec.jar
ENV CGJ_REDIS_URI="127.0.0.1:6379"
ENV CGJ_PROXY=""
RUN mkdir /data/
ENTRYPOINT ["/usr/java/openjdk-14/bin/java", "-Duser.timezone=GMT+8"]
CMD ["-Dcgj.logsPath=/data/logs", "-jar", "/CGJ.jar", "botMode", "-botDataDir=/data"]
COPY ${jarFileName} /CGJ.jar

661
LICENSE Normal file
View File

@ -0,0 +1,661 @@
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published
by the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<https://www.gnu.org/licenses/>.

138
README.md Normal file
View File

@ -0,0 +1,138 @@
# ContentGrabbingJi - 色图姬 #
一个以高性能、高效率为目标,多平台/框架支持、持续维护的Pixiv聊天机器人
色图姬存在的目的最初是作为**爬虫 + 机器人**的形式开发,在开发过程中逐渐以多聊天平台,高效率为目标进行。
## 安装 ##
### 通过Jar文件部署 ###
1. 从项目的[版本页面](https://github.com/LamGC/ContentGrabbingJi/releases)下载最新版色图姬主程序jar文件
2. 准备一个目录, 用于存放运行数据(这里以`./runData`为例子).
3. 将通过PixivLoginProxyServer获得的Pixiv登录会话文件放置在目录中(`./runData/cookies.store`).
4. 使用命令`java -jar <CGJ.jar> buildPassword -password "QQ机器人账号的密码"`构造一个登录用的密码.
5. 在数据目录创建一个配置文件`bot.properties`, 并添加如下内容:
```properties
bot.qq=<机器人QQ账号>
bot.password=<通过buildPassword获得的密码>
```
6. 在数据目录创建新文件夹`setting`, 并创建一个全局配置文件`global.properties`, 然后设置如下内容:
```properties
# 该配置为全局配置文件, 当群组没有特定配置时, 将使用全局配置.
# 管理员QQ (必填)
admin.adminId=<管理员QQ号>
# 是否允许r18作品
image.allowR18=false
# 查询排行榜默认的长度(比如15就是发送1~15名的作品), 该选项请适当调整, 设置过长可能导致超出聊天平台的最长消息长度, 导致发送失败!
ranking.itemCountLimit=15
# 排行榜图片数量(比如排行榜长度为15, 可以设置前10名有图片, 后5名没有图片), 调整该配置可有效控制消息发送所需时间.
ranking.imageCountLimit=15
# 搜索结果缓存时间, 默认2小时, 单位毫秒
cache.searchBody.expire=7200000
# 搜索结果长度. 该选项请适当调整, 设置过长可能导致超出聊天平台的最长消息长度, 导致发送失败!
search.itemCountLimit=8
```
7. 配置完成后, 准备一台Redis服务端, 用于缓存数据.
8. Redis服务器准备好后, 使用命令启动色图姬:`java -jar <CGJ.jar> botMode -botDataDir <数据目录> -redisAddress <Redis服务器地址> [-proxy 代理服务器地址]`
9. 完成!好好享受!
### 通过Docker部署 ###
使用Docker将可以更好的管理色图姬所使用的资源和管理色图姬的运行。
(正在完善中...)
## 使用 ##
### 普通用户 ###
将色图姬部署完成,并且让色图姬在某个平台登录完成后,你就可以通过聊天平台向色图姬发起会话了!
使用 `.cgj` 向色图姬询问帮助信息!
另外,由于色图姬在开发过程中直接使用了原本应用在命令行中的参数解析工具,所以你需要了解一下色图姬命令的格式,
色图姬的命令格式为:
```bash
.cgj <命令> -<参数名> <参数值> ...
```
如果色图姬无法识别你的命令,那么它会发送一次帮助信息给你。
### 管理员用户 ###
你应该注意到了在部署过程中你需要设置一个管理员QQ的配置色图姬支持通过命令来管理色图姬的运行。
目前支持的管理员命令:
```bash
# 清除缓存(慎用)
# 该操作将会清除Redis服务器内的所有数据, 以及色图姬下载到本地的所有图片缓存.
.cgjadmin cleanCache
# 设置配置项
# 如果不使用group参数, 则设置全局配置
# 注意: 配置项设置后需要使用`.cgjadmin saveProperties`才会保存到文件中,
# 如不保存, 则仅能在本次运行中生效(或使用`.cgjadmin loadProperties`重新加载后失效).
.cgjadmin setProperty <-key 配置项名> <-value 配置项新值> [-group 指定群组]
# 查询配置项
# 如果不使用group参数, 则查询全局配置
.cgjadmin getProperty <-key 配置项名> [-group 指定群组]
# 保存所有配置
.cgjadmin saveProperties
# 读取所有配置
# 使用 reload 参数将会重载所有配置, 而不是覆盖读取
.cgjadmin loadProperties [-reload]
# 运行定时更新任务
# 可指定要更新数据的日期
.cgjadmin runUpdateTask [-date yyyy-MM-dd]
# 增加群组作品推送
# 如果增加了original参数, 则图片为原图发送
# 如果不指定group参数, 则群组为命令发送所在群组
# 最长发送时间 = 最短发送时间 + 随机时间范围
.cgjadmin addPushGroup [-group 指定群组号] [-minTime 最短发送时间] [-floatTime 随机时间范围] [-rankingStart 排行榜起始排名]
[-rankingStop 排行榜结束排名] [-mode 排行榜模式] [-type 排行榜类型] [-original]
# 删除群组推送功能
# 如果不指定group参数, 则群组为命令发送所在群组
.cgjadmin removePushGroup [-group 指定群组号]
# 加载作品推送配置
.cgjadmin loadPushList
# 保存作品推送配置
.cgjadmin savePushList
# 获取被报告的作品列表
# 该命令会返回被其他用户报告其存在问题的作品列表
.cgjadmin getReportList
# 解封被报告的作品
.cgjadmin unBanArtwork <-id 被ban作品的Id>
```
## 贡献 ##
向色图姬贡献不一定需要编程知识,向色图姬项目提出意见,反馈问题同样会为色图姬项目提供很大的帮助!
如果你在使用色图姬的过程中遇到了Bug可以通过色图姬项目的**Issues**使用[Bug反馈模板](https://github.com/LamGC/ContentGrabbingJi/issues/new?assignees=&labels=bug&template=Bug_Report.md&title=)向色图姬提供Bug信息。
如果是为色图姬提供一些新功能想法或者对色图姬有什么意见则可以直接通过Issues发起讨论。
如果你会Java开发又想为色图姬提供一些新功能可以通过Fork仓库的方法实现后发起PR合并到色图姬项目中
向色图姬贡献代码需要遵循一些贡献事项如果你的代码不符合这些事项PR有可能会被关闭
> 注意色图姬的初衷并没有任何恶意的意图如果尝试向色图姬提供恶意功能或代码PR将会被拒绝、关闭。
## LICENSE ##
本项目基于 `GNU Affero General Public License 3.0` 开源许可协议开源,
你可以在本项目目录下的 `LICENSE` 文件查阅协议副本,
或浏览 [https://www.gnu.org/licenses/agpl-3.0.html](https://www.gnu.org/licenses/agpl-3.0.html) 查阅协议副本。
```
ContentGrabbingJi - A pixiv robot used in chat
Copyright (C) 2020 LamGC (lam827@lamgc.net)
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as
published by the Free Software Foundation, either version 3 of the
License, or any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/agpl-3.0.html>.
```
尤其注意:根据协议,如果你基于本项目开发本项目的衍生版本并通过网络为他人提供服务,则必须遵守该协议,向 *服务的使用者* 提供 **为其服务的衍生版本****完整源代码**

View File

@ -0,0 +1,108 @@
## Pixiv作品信息批量获取接口 ##
### 说明 ###
接口可一次获取多个作品的基础信息
### 接口地址 ###
```
GET https://www.pixiv.net/ajax/illust/recommend/illusts
```
- 需要登录: `否`
- 是否为Pixiv接口标准返回格式: `是`
### 参数 ###
- `illust_ids[]`: 作品Id, 可重复添加该参数
### 请求示例 ###
```
GET https://www.pixiv.net/ajax/illust/recommend/illusts?illust_ids[]=82030844&illust_ids[]=82029098&illust_ids[]=82028913
```
### 返回数据 ###
#### 数据示例 ####
```json
{
"error":false,
"message":"",
"body":{
"illusts":[
{
"illustId":"82030844",
"illustTitle":"3rd anniversary",
"id":"82030844",
"title":"3rd anniversary",
"illustType":0,
"xRestrict":0,
"restrict":0,
"sl":2,
"url":"https:\/\/i.pximg.net\/c\/360x360_70\/custom-thumb\/img\/2020\/06\/02\/11\/24\/49\/82030844_p0_custom1200.jpg",
"description":"",
"tags":[
"\u30a2\u30ba\u30fc\u30eb\u30ec\u30fc\u30f3",
"\u6bd4\u53e1(\u30a2\u30ba\u30fc\u30eb\u30ec\u30fc\u30f3)",
"\u6c34\u7740",
"\u8db3\u88cf",
"\u88f8\u8db3",
"\u30a2\u30ba\u30fc\u30eb\u30ec\u30fc\u30f35000users\u5165\u308a",
"\u30a2\u30ba\u30fc\u30eb\u30ec\u30fc\u30f310000users\u5165\u308a",
"\u30dd\u30cb\u30fc\u30c6\u30fc\u30eb",
"\u30db\u30eb\u30bf\u30fc\u30cd\u30c3\u30af"],
"userId":"6662895",
"userName":"ATDAN-",
"width":1500,
"height":844,
"pageCount":1,
"isBookmarkable":true,
"bookmarkData":null,
"alt":"#\u30a2\u30ba\u30fc\u30eb\u30ec\u30fc\u30f3 3rd anniversary - ATDAN-\u7684\u63d2\u753b",
"isAdContainer":false,
"titleCaptionTranslation":{
"workTitle":null,
"workCaption":null
},
"createDate":"2020-06-02T01:29:40+09:00",
"updateDate":"2020-06-02T11:24:49+09:00",
"profileImageUrl":"https:\/\/i.pximg.net\/user-profile\/img\/2016\/01\/11\/21\/46\/50\/10371466_80f6ad67eab3b8abd44a2fb74ddd1ba1_50.jpg",
"type":"illust"
}, // ...
]
}
}
```
#### 参数详解 ####
- `illusts`: (`Object[]`) 存储查询作品信息的数组
- `illustId`: (`string` -> `number`) 作品Id
- `illustTitle`: (`string`) 作品标题
- `id`: (`string` -> `number`) 与`illustId`一致, 猜测是以兼容旧版本为目录而存在
- `title`: (`string`) 与`illustTitle`一致, 猜测是以兼容旧版本为目录而存在
- `illustType`: (`number`) 作品类型
- `0`: 插画作品
- `1`: 漫画作品
- `2`: 动图作品
- `xRestrict`: (`number`) 作品是否为限制级, 基本准确, 少部分不一定(看Pixiv审核怎么理解了)
- `0`: 非限制级内容(即非R18作品)
- `1`: 限制级内容(即R18作品)
- `restrict`: (`number`) 作品限制级(意义不明, 可能是兼容性问题?)?
- `sl`: (`number`) 不明?
- `url`: (`string`) 作品预览图链接, 需要`Referer`请求头
- `description`: (`string`) 作品说明
- `tags`: (`string[]`) 作品标签数组
- `userId`: (`string` -> `number`) 作者用户Id
- `userName`: (`string`) 作者用户名
- `width`: (`number`) 作品长度
- `height`: (`number`) 作品高度
- `pageCount`: (`number`) 作品页数
- `isBookmarkable`: (`boolean`) 不明?
- `alt`: (`string`) 简略介绍信息(在图片加载失败时可提供给`img`标签使用)
- `isAdContainer`: (`boolean`) 不明?
- `titleCaptionTranslation`: (`Object`) 不明?
- `workTitle`: (`Unknown`) 不明?
- `workCaption`: (`Unknown`) 不明?
- `createDate`: (`string`) 作品创建时间(或者是完成时间?)
- `updateDate`: (`string`) 作品上传时间
- `profileImageUrl`: (`string`) 作者用户头像图片链接
- `type`: (`string`) 作品类型名

View File

@ -0,0 +1,166 @@
## Pixiv 排行榜获取接口 ##
### 接口地址 ###
```
GET https://www.pixiv.net/ranking.php
```
- 是否需要登录: `是`
- 是否为Pixiv标准接口返回格式: `否`
- 是否需要Referer请求头: `否`
### 参数 ###
> 提示: 该接口参数较为复杂,请结合表格查看
- `date`: 排行榜时间与Mode有关 (格式: yyyy-MM-dd)
- `mode`: 排行榜模式
- `daily`: 每天
- `weekly`: 每周
- `monthly`: 每月
- `rookie`: 新人
- `original`: 原创
- `male`: 男性偏好
- `female`: 女性偏好
- `daily_r18`: 每天 - 仅成人内容
- `weekly_r18`: 每周 - 仅成人内容
- `male_r18`: 男性偏好 - 仅成人内容
- `female_r18`: 女性偏好 - 仅成人内容
- `content`: 排行榜内容类型
- `all`: 全部内容 (实际使用请直接省略`content`参数)
- `illust`: 插画
- `ugoira`: 动图
- `manga`: 漫画
- `p`: 排行榜页数 (如超出范围,则返回错误信息)
- `format`: 格式
- `json`: 以Json返回数据
- (留空): 返回完整的排行榜网页
#### 参数关系表 ####
`mode`参数与`content`参数有一些支持关系,并不是所有的`mode`参数都能被所有的`content`参数支持,故附下表。
参数 |all|illust|ugoira|manga
:-: |:-:| :-: | :-: | :-:
daily |`√`|`√`|`√`|`√`
weekly |`√`|`√`|`√`|`√`
monthly |`√`|`√`|×|`√`
rookie |`√`|`√`|×|`√`
original |`√`|×|×|×
male |`√`|×|×|×
female |`√`|×|×|×
daily_r18 |`√`|`√`|`√`|`√`
weekly_r18|`√`|`√`|`√`|`√`
male_r18 |`√`|×|×|×
female_r18|`√`|×|×|×
### 返回数据 ###
#### 数据示例 ####
```json
{
"contents":[
{
"title":"【伊アオ】髪結い。",
"date":"2020年05月31日 14:26",
"tags":[
"鬼滅の刃",
"伊アオ",
"嘴平伊之助",
"神崎アオイ",
"鬼滅の刃1000users入り"
],
"url":"https:\/\/i.pximg.net\/c\/240x480\/img-master\/img\/2020\/05\/31\/14\/26\/41\/81987309_p0_master1200.jpg",
"illust_type":"0",
"illust_book_style":"0",
"illust_page_count":"1",
"user_name":"シロウ",
"profile_img":"https:\/\/i.pximg.net\/user-profile\/img\/2020\/05\/01\/02\/18\/18\/18450100_ac34872504959f8cc26f086248066b39_50.png",
"illust_content_type":{
"sexual":0,
"lo":false,
"grotesque":false,
"violent":false,
"homosexual":false,
"drug":false,
"thoughts":false,
"antisocial":false,
"religion":false,
"original":false,
"furry":false,
"bl":false,
"yuri":false
},
"illust_series":false,
"illust_id":81987309,
"width":600,
"height":2226,
"user_id":174995,
"rank":51,
"yes_rank":83,
"rating_count":707,
"view_count":19759,
"illust_upload_timestamp":1590902801,
"attr":"",
"is_bookmarked":false,
"bookmarkable":true
}, // ....
],
"mode":"daily",
"content":"all",
"page":2,
"prev":1,
"next":3,
"date":"20200601",
"prev_date":"20200531",
"next_date":false,
"rank_total":500
}
```
#### 字段说明 ####
- `contents`: (`Object[]`) 排行榜数组, 最多50行排行榜信息
- `illust_id`: (`number`) 作品Id
- `title`: (`string`) 作品标题
- `attr`: (`string`) 不明?
- `tags`: (`string[]`) 原始标签数组
- `url`: (`string`) 预览画质的原始尺寸图下载链接(存在防盗链)
- `illust_type`: (`string` -> `number`) 作品类型
- `illust_book_style`: (`string` -> `number`) 不明?
- `illust_page_count`: (`string` -> `number`) 作品页数
- `user_name`: (`string`) 画师用户名
- `user_id`: (`number`) 画师用户Id
- `profile_img`: (`string`) 画师用户头像
- `illust_content_type`: (`Object`) 作品内容信息(警告: 文档内容仅作为开发参考, 并不传播相关内容!!!)
- `sexual`: (`number`) 作品内容分级
- `0`: 全年龄
- `1`: 青少年
- `2`: 成人级
- `lo`: (`boolean`) 是否为loli作品
- `grotesque`: (`boolean`) 是否为怪诞作品
- `violent`: (`boolean`) 作品是否含有暴力/强暴相关元素
- `homosexual`: (`boolean`) 作品是否含有同性恋相关元素
- `drug`: (`boolean`) 作品是否含有药物相关元素
- `thoughts`: (`boolean`) 作品是否含有思维/记忆相关元素(这个属性翻译起来有些问题, 待纠正)?
- `antisocial`: (`boolean`) 作品是否含有反社会, 令人厌恶的相关元素
- `religion`: (`boolean`) 作品是否含有宗教, 信仰相关元素
- `original`: (`boolean`) 作品是否为原创作品
- `furry`: (`boolean`) 作品是否有兽人相关元素
- `bl`: (`boolean`) 作品是否有耽美相关元素
- `yuri`: (`boolean`) 作品是否有百合相关元素
- `illust_series`: (`boolean`) 是否为系列作品
- `width`: (`number`) 作品宽度(建议以原图为准)
- `height`: (`number`) 作品高度(建议以原图为准)
- `rank`: (`number`) 本期排行榜排名
- `yes_rank`: (`number`) 上期同排行榜排名
- `rating_count`:
- `view_count`: (`number`) 浏览量
- `illust_upload_timestamp`: (`number`) 作品上传时间戳(10位)
- `is_bookmarked`: (`boolean`) 不明?
- `bookmarkable`: (`boolean`) 不明?
- `mode`: (`string`) 请求的排行榜模式字段
- `content`: (`string`) 请求的内容类型
- `page`: (`number`) 当前排行榜页数
- `prev`: (`string` / `boolean`) 上一页排行榜页数, 如果该请求的页数为首页, 则为`false`
- `next`: (`string` / `boolean`) 下一页排行榜页数, 如果该请求的页数为页尾, 则为`false`
- `date`: (`string`) 排行榜日期(格式:`yyyyMMdd`)
- `prev_date`: (`string` / `boolean`) 如果存在上一期排行榜, 则该属性为上期排行榜日期字符串, 否则为`false`
- `next_date`: (`string` / `boolean`) 如果存在下一期排行榜, 则该属性为下期排行榜日期字符串, 否则为`false`
- `rank_total`: (`number`) 该排行榜的总排行数

View File

@ -0,0 +1,32 @@
## 接口名 ##
### 说明 ###
### 接口地址 ###
```
GET/POST https://www.pixiv.net/...
```
- 是否需要登录: `是/否`
- 是否为Pixiv标准接口返回格式: `是/否`
- 是否需要Referer请求头: `是/否`
### 参数 ###
### 请求示例 ###
```
GET/POST
---- Request Body ---- // 如果没有, 可以不写, 如没有记得删除
```
### 返回数据 ###
#### 数据示例 ####
```json
// Object[] 中只留一个, 其他删除后保留逗号, 增加 '// ...' 注释
```
#### 字段说明 ####
- `属性名`: (`JS类型`) 属性说明
- `对象属性名`: (`原始JS类型` -> `可转换JS类型`) 属性说明
- `属性名`: (`JS类型1` / `JS类型2` / ...) 属性说明, 需要清晰说明在什么情况下类型为`JS类型1`, 什么情况下是`JS类型2`.
- `不明属性`: (`Unknown`) 如果属性用途不明, 则在说明后面加上`?`符号, 类型不明则填`Unknown`.

View File

@ -0,0 +1,18 @@
## Pixiv接口标准返回格式 ##
Pixiv大部分接口在返回数据时都会遵循以下格式
```json
{
"error": false,
"message": "",
"body": {
}
}
```
大部分是如此(部分接口比较特别, 不是这个格式)
属性|类型|说明
---|---|---
error|Boolean|如果接口返回错误信息, 该属性为`true`
message|String|如果`error`属性为`true`, 则该属性不为空字符串
body|Object/Array|如果`error`不为`true`, 该属性有数据, 否则属性可能不存在

View File

@ -0,0 +1,365 @@
## Pixiv内容搜索接口 ##
### 说明 ###
> 注意: 本接口可能会影响Pixiv对账号的行为判断猜测不一定会
该接口用于在Pixiv搜索内容。
### 接口地址 ###
```
GET https://www.pixiv.net/ajax/search/{Type}/{Content}?{Param...}
```
- 是否需要登录: `是/否`
- 是否为Pixiv标准接口返回格式: `是/否`
- 是否需要Referer请求头: `否`
### 参数 ###
#### Url参数 ####
- `Type`: 内容类型
- illustrations(插画)
- top(推荐)
- manga(漫画)
- novels(小说)
- `Content`: 搜索内容
#### GET参数 ####
##### 必填 #####
- `word`: 与搜索内容一致 (经测试似乎可以省略)
- `s_mode`: 匹配模式
- `s_tag`: 标签,部分一致
- `s_tag_full`: 标签,完全一致
- `s_tc`: 标签和说明文字
- `type`: 作品类型
- `all`: 插画、漫画和动图
- `illust_and_ugoira`: 插画和动图
- `illust`: 仅插画
- `manga`: 仅漫画
- `ugoira`: 仅动图
- `p`: 指定页数 (当指定页数超出范围后,`body.illust.data`为空数组)
- `order`: 排序设置
- `date`: 按时间从旧到新
- `date_d`: 按时间从新到旧
- `(Unknown)`: 未知, 猜测是会员功能的热门搜索
- `mode`: 内容分级设置
- `all`: 全部内容
- `safe`: 排除成人内容
- `r18`: 仅成人内容
##### 选填 #####
- `wlt`: 作品最低宽度(px)
- `wgt`: 作品最高宽度(px)
- `hlt`: 作品最低高度(px)
- `hgt`: 作品最高高度(px)
- `ratio`: 作品横宽比过滤 (初步测试表明,该参数无法指定横宽比,可能暂不支持该功能)
- `0.5`: 仅横图
- `-0.5`: 仅纵图
- `0`: 仅正方形图
- `tool`: 限定作品绘制工具
- `scd`: 过滤作品发布时间 - 开始时间(yyyy-MM-dd)
- `scd`: 过滤作品发布时间 - 结束时间(yyyy-MM-dd)
- `(Unknown)`: 最小收藏数 (该参数为会员限定功能,后续补充)
### 返回数据 ###
#### 数据示例 ####
```json
{
"error":false,
"body":{
"illustManga":{
"data":[
{
"illustId":"82130571",
"illustTitle":"空の絵",
"id":"82130571",
"title":"空の絵",
"illustType":0,
"xRestrict":0,
"restrict":0,
"sl":2,
"url":"https:\/\/i.pximg.net\/c\/250x250_80_a2\/img-master\/img\/2020\/06\/06\/17\/51\/14\/82130571_p0_square1200.jpg",
"description":"",
"tags":[
"風景",
"空",
"草",
"雲"],
"userId":"31507675",
"userName":"昏omeme",
"width":1600,
"height":1600,
"pageCount":2,
"isBookmarkable":true,
"bookmarkData":null,
"alt":"#風景 空の絵 - 昏omeme的插画",
"isAdContainer":false,
"titleCaptionTranslation":{
"workTitle":null,
"workCaption":null
},
"createDate":"2020-06-06T17:51:14+09:00",
"updateDate":"2020-06-06T17:51:14+09:00",
"profileImageUrl":"https:\/\/i.pximg.net\/user-profile\/img\/2020\/05\/06\/19\/21\/04\/18509741_e3166e69809c44d6926454ecaac89590_50.png"
}, // ...
],
"total":165875,
"bookmarkRanges":[
{
"min":null,
"max":null
},
{
"min":10000,
"max":null
},
{
"min":5000,
"max":null
},
{
"min":1000,
"max":null
},
{
"min":500,
"max":null
},
{
"min":300,
"max":null
},
{
"min":100,
"max":null
},
{
"min":50,
"max":null
}
]
},
"popular":{
"recent":[
{
"illustId":"82062770",
"illustTitle":"Still you remember",
"id":"82062770",
"title":"Still you remember",
"illustType":0,
"xRestrict":0,
"restrict":0,
"sl":2,
"url":"https:\/\/i.pximg.net\/c\/250x250_80_a2\/img-master\/img\/2020\/06\/03\/18\/02\/15\/82062770_p0_square1200.jpg",
"description":"",
"tags":[
"オリジナル",
"女の子",
"カラス",
"风景",
"線路"],
"userId":"1069005",
"userName":"へちま",
"width":2000,
"height":1415,
"pageCount":1,
"isBookmarkable":true,
"bookmarkData":null,
"alt":"#オリジナル Still you remember - へちま的插画",
"isAdContainer":false,
"titleCaptionTranslation":{
"workTitle":null,
"workCaption":null
},
"createDate":"2020-06-03T18:02:15+09:00",
"updateDate":"2020-06-03T18:02:15+09:00",
"profileImageUrl":"https:\/\/i.pximg.net\/user-profile\/img\/2013\/05\/10\/00\/38\/05\/6213543_c94edc0d13776214467bd0c47ee6491a_50.jpg"
}, // ...
],
"permanent":[
{
"illustId":"60993044",
"illustTitle":"無題",
"id":"60993044",
"title":"無題",
"illustType":0,
"xRestrict":0,
"restrict":0,
"sl":2,
"url":"https:\/\/i.pximg.net\/c\/250x250_80_a2\/img-master\/img\/2017\/01\/18\/13\/07\/46\/60993044_p0_square1200.jpg",
"description":"",
"tags":[
"少女",
"女の子",
"原创",
"オリジナル",
"场景",
"落書き",
"創作",
"背景",
"风景",
"オリジナル7500users入り"],
"userId":"18811972",
"userName":"淅洵",
"width":3507,
"height":2480,
"pageCount":1,
"isBookmarkable":true,
"bookmarkData":null,
"alt":"#少女 無題 - 淅洵的插画",
"isAdContainer":false,
"titleCaptionTranslation":{
"workTitle":null,
"workCaption":null
},
"createDate":"2017-01-18T13:07:46+09:00",
"updateDate":"2017-01-18T13:07:46+09:00",
"profileImageUrl":"https:\/\/i.pximg.net\/user-profile\/img\/2017\/05\/29\/17\/17\/49\/12623968_6cf3f1979e10643425972ae205a7920d_50.jpg"
}, // ...
]
},
"relatedTags":[
"風景",
"背景",
"風景画",
"空",
"雲",
"創作",
"ファンタジー",
"夏",
"青",
"建物",
"青空",
"少女",
"東京",
"抽象画",
"男の子",
"透明水彩"
],
"tagTranslation":{
"風景":{
"zh":"风景"
},
"背景":{
"zh":"background"
},
"風景画":{
"zh":"landscape painting"
},
"空":{
"zh":"sky"
},
"雲":{
"zh":"云"
},
"創作":{
"zh":"原创"
},
"ファンタジー":{
"zh":"奇幻"
},
"夏":{
"zh":"夏天"
},
"青":{
"zh":"蓝"
},
"建物":{
"zh":"building"
},
"青空":{
"zh":"蓝天"
},
"少女":{
"zh":"young girl"
},
"東京":{
"zh":"tokyo"
},
"抽象画":{
"zh":"abstract art"
},
"男の子":{
"zh":"男孩子"
},
"透明水彩":{
"zh":"transparent watercolor"
}
},
"zoneConfig":{
"header":{
"url":"https:\/\/pixon.ads-pixiv.net\/show?zone_id=header&format=js&s=1&up=0&a=22&ng=g&l=zh&uri=%2Fajax%2Fsearch%2Fartworks%2F_PARAM_&is_spa=1&K=59bba275c645c&ab_test_digits_first=20&yuid=FwdzEnA&suid=Pgfip96ymw5tvu9l9&num=5edb6277927"
},
"footer":{
"url":"https:\/\/pixon.ads-pixiv.net\/show?zone_id=footer&format=js&s=1&up=0&a=22&ng=g&l=zh&uri=%2Fajax%2Fsearch%2Fartworks%2F_PARAM_&is_spa=1&K=59bba275c645c&ab_test_digits_first=20&yuid=FwdzEnA&suid=Pgfip96yn1fgocj2&num=5edb6277775"
},
"infeed":{
"url":"https:\/\/pixon.ads-pixiv.net\/show?zone_id=illust_search_grid&format=js&s=1&up=0&a=22&ng=g&l=zh&uri=%2Fajax%2Fsearch%2Fartworks%2F_PARAM_&is_spa=1&K=59bba275c645c&ab_test_digits_first=20&yuid=FwdzEnA&suid=Pgfip96yn4t7cho88&num=5edb6277137"
}
},
"extraData":{
"meta":{
"title":"#风景のイラスト・マンガ作品(投稿超过10万件 - pixiv",
"description":"pixiv",
"canonical":"https:\/\/www.pixiv.net\/tags\/%E9%A3%8E%E6%99%AF",
"alternateLanguages":{
"ja":"https:\/\/www.pixiv.net\/tags\/%E9%A3%8E%E6%99%AF",
"en":"https:\/\/www.pixiv.net\/en\/tags\/%E9%A3%8E%E6%99%AF"
},
"descriptionHeader":"pixiv"
}
}
}
}
```
#### 字段说明 ####
- `novel`: (`Object`) 小说搜索结果
- `data`: (`Object`) 搜索结果(仅当前页数)
- (待补充)
- `total`: (`number`) 搜索结果总量
- `popular`: (`Object`) 受欢迎的搜索结果
- `relatedTags`: (`string[]`) 与搜索结果相关的标签
- `tagTranslation`: (`Object`) 相关标签的翻译信息
- `{Attr: 标签名}`: 标签名为属性名
- `语言名(例如 中文是 zh)`: (`string`) 标签翻译名
- `zoneConfig`: (`Object`) 猜测是广告相关信息?
- `extraData`: (`Object`) 扩展信息
- `meta`: (`Object`) 网页元数据
- `title`: (`string`) 网页标题
- `description`: 搜索结果说明内容
- `descriptionHeader`: (`string`) 说明内容的Html代码
- `alternateLanguages`: (`Object`) 不明链接?
- `{语言名}`: 对应语言的链接
- `illustManga`: (`Object`) 漫画和插画的搜索结果
- `total`: (`number`) 搜索结果总量
- `data`: (`Object[]`) 搜索结果(仅当前页数)
- `illustId`: (`string` -> `number`) 作品Id
- `illustTitle`: (`string`) 作品标题
- `id`: (`string` -> `number`) 与`illustId`一致, 猜测是以兼容旧版本为目录而存在
- `title`: (`string`) 与`illustTitle`一致, 猜测是以兼容旧版本为目录而存在
- `illustType`: (`number`) 作品类型
- `0`: 插画作品
- `1`: 漫画作品
- `2`: 动图作品
- `xRestrict`: (`number`) 作品是否为限制级, 基本准确, 少部分不一定(看Pixiv审核怎么理解了)
- `0`: 非限制级内容(即非R18作品)
- `1`: 限制级内容(即R18作品)
- `restrict`: (`number`) 作品限制级(意义不明, 可能是兼容性问题?)?
- `sl`: (`number`) 不明?
- `url`: (`string`) 作品预览图链接, 需要`Referer`请求头
- `description`: (`string`) 作品说明
- `tags`: (`string[]`) 作品标签数组
- `userId`: (`string` -> `number`) 作者用户Id
- `userName`: (`string`) 作者用户名
- `width`: (`number`) 作品长度
- `height`: (`number`) 作品高度
- `pageCount`: (`number`) 作品页数
- `isBookmarkable`: (`boolean`) 不明?
- `bookmarkData`: (`Unknown`) 不明?
- `alt`: (`string`) 简略介绍信息(在图片加载失败时可提供给`img`标签使用)
- `isAdContainer`: (`boolean`) 不明?
- `titleCaptionTranslation`: (`Object`) 不明?
- `workTitle`: (`Unknown`) 不明?
- `workCaption`: (`Unknown`) 不明?
- `createDate`: (`string`) 作品创建时间(或者是完成时间?)
- `updateDate`: (`string`) 作品上传时间
- `profileImageUrl`: (`string`) 作者用户头像图片链接

View File

@ -0,0 +1,47 @@
## 搜索推荐接口 ##
### 说明 ###
可用于优化搜索内容。
### 接口地址 ###
```
GET https://www.pixiv.net/rpc/cps.php?
```
- 是否需要登录: `否`
- 是否为Pixiv标准接口返回格式: `否`
- 是否需要Referer请求头: `是`
> 补充: Referer请求头只要是Pixiv的就可以了.
### 参数 ###
Url参数:
- `keyword`: 搜索内容
> 注意: 搜索内容需要进行Url编码空格要转换成`%20`而不是`+`
### 请求示例 ###
```
GET https://www.pixiv.net/rpc/cps.php?keyword=幸运星
```
### 返回数据 ###
#### 数据示例 ####
```json
{
"candidates":[
{
"tag_name":"\u3089\u304d\u2606\u3059\u305f",
"access_count":"68286498",
"tag_translation":"\u5e78\u8fd0\u661f",
"type":"tag_translation"
}, // ...
]
}
```
#### 字段说明 ####
- `candidates`: (`Object[]`) 搜索推荐候选列表
- `tag_name`: (`string`) 推荐词原名
- `access_count`: (`string` -> `number`) 推荐词访问量
- `tag_translation`: (`string`) 推荐词对应翻译名
- `type`: (`string`) 推荐词类型
- `tag_translation`: 标签翻译信息
- `prefix`: 前缀

View File

@ -0,0 +1,319 @@
## Pixiv预加载数据 ##
### 说明 ###
作品预加载数据仅在加载作品页面时存在,处理后删除。
### 接口地址 ###
```
GET https://www.pixiv.net/artworks/{IllustId}
```
- 是否需要登录: `是`
- 是否为Pixiv标准接口返回格式: `否`
- 是否需要Referer请求头: `否`
### 参数 ###
Url参数:
- `IllustId`: 作品Id
### 请求示例 ###
```
GET https://www.pixiv.net/artworks/82647306
```
### 返回数据 ###
> 注意: 该接口返回HTML格式数据并不是JSON格式数据。
预加载数据需要对返回的Html数据进行解析路径如下
- CSS Select: meta#meta-preload-data
- html>head>meta#meta-preload-data
获得标签后,获取`content`属性即可获得预加载数据内容
#### 数据示例 ####
```json
{
"timestamp":"2020-07-01T11:32:30+09:00",
"illust":{
"82647306":{
"illustId":"82647306",
"illustTitle":"水着キャルちゃん!",
"illustComment":"水着のキャルちゃんはかわいいぞ!!",
"id":"82647306",
"title":"水着キャルちゃん!",
"description":"水着のキャルちゃんはかわいいぞ!!",
"illustType":0,
"createDate":"2020-06-29T12:28:06+00:00",
"uploadDate":"2020-06-29T12:28:06+00:00",
"restrict":0,
"xRestrict":0,
"sl":2,
"urls":{
"mini":"https://i.pximg.net/c/48x48/img-master/img/2020/06/29/21/28/06/82647306_p0_square1200.jpg",
"thumb":"https://i.pximg.net/c/250x250_80_a2/img-master/img/2020/06/29/21/28/06/82647306_p0_square1200.jpg",
"small":"https://i.pximg.net/c/540x540_70/img-master/img/2020/06/29/21/28/06/82647306_p0_master1200.jpg",
"regular":"https://i.pximg.net/img-master/img/2020/06/29/21/28/06/82647306_p0_master1200.jpg",
"original":"https://i.pximg.net/img-original/img/2020/06/29/21/28/06/82647306_p0.jpg"
},
"tags":{
"authorId":"55859246",
"isLocked":false,
"tags":[
{
"tag":"プリンセスコネクト!Re:Dive",
"locked":true,
"deletable":false,
"userId":"55859246",
"translation":{
"en":"公主连结Re:Dive"
},
"userName":"秋鳩むぎ"
}, // ...
],
"writable":true
},
"alt":"#プリンセスコネクト!Re:Dive 水着キャルちゃん! - 秋鳩むぎ的插画",
"storableTags":[
"_bee-JX46i",
"nAtxkwJ5Sy",
"q303ip6Ui5"
],
"userId":"55859246",
"userName":"秋鳩むぎ",
"userAccount":"pigeonwheat",
"userIllusts":{
"82647306":{
"illustId":"82647306",
"illustTitle":"水着キャルちゃん!",
"id":"82647306",
"title":"水着キャルちゃん!",
"illustType":0,
"xRestrict":0,
"restrict":0,
"sl":2,
"url":"https://i.pximg.net/c/250x250_80_a2/img-master/img/2020/06/29/21/28/06/82647306_p0_square1200.jpg",
"description":"水着のキャルちゃんはかわいいぞ!!",
"tags":[
"プリンセスコネクト!Re:Dive",
"キャル(プリコネ)",
"おへそ"
],
"userId":"55859246",
"userName":"秋鳩むぎ",
"width":2000,
"height":3000,
"pageCount":1,
"isBookmarkable":true,
"bookmarkData":null,
"alt":"#プリンセスコネクト!Re:Dive 水着キャルちゃん! - 秋鳩むぎ的插画",
"isAdContainer":false,
"titleCaptionTranslation":{
"workTitle":null,
"workCaption":null
},
"createDate":"2020-06-29T21:28:06+09:00",
"updateDate":"2020-06-29T21:28:06+09:00",
"seriesId":null,
"seriesTitle":null
}
},
"likeData":false,
"width":2000,
"height":3000,
"pageCount":1,
"bookmarkCount":39,
"likeCount":31,
"commentCount":2,
"responseCount":0,
"viewCount":239,
"isHowto":false,
"isOriginal":false,
"imageResponseOutData":[
],
"imageResponseData":[
],
"imageResponseCount":0,
"pollData":null,
"seriesNavData":null,
"descriptionBoothId":null,
"descriptionYoutubeId":null,
"comicPromotion":null,
"fanboxPromotion":null,
"contestBanners":[
],
"isBookmarkable":true,
"bookmarkData":null,
"contestData":null,
"zoneConfig":{
"responsive":{
"url":"https://pixon.ads-pixiv.net/show?zone_id=illust_responsive&amp;format=js&amp;s=1&amp;up=0&amp;a=22&amp;ng=w&amp;l=zh&amp;uri=%2Fartworks%2F_PARAM_&amp;is_spa=1&amp;K=59bba275c645c&amp;ab_test_digits_first=32&amp;yuid=NDJ3gQk&amp;suid=Pggb9mua3yznnr7lz&amp;num=5efbf5be273&amp;t=_bee-JX46i&amp;t=b8b4-hqot7&amp;t=kY01H5r3Pd"
},
"rectangle":{
"url":"https://pixon.ads-pixiv.net/show?zone_id=illust_rectangle&amp;format=js&amp;s=1&amp;up=0&amp;a=22&amp;ng=w&amp;l=zh&amp;uri=%2Fartworks%2F_PARAM_&amp;is_spa=1&amp;K=59bba275c645c&amp;ab_test_digits_first=32&amp;yuid=NDJ3gQk&amp;suid=Pggb9mua42776dfuu&amp;num=5efbf5be810&amp;t=_bee-JX46i&amp;t=b8b4-hqot7&amp;t=kY01H5r3Pd"
},
"500x500":{
"url":"https://pixon.ads-pixiv.net/show?zone_id=bigbanner&amp;format=js&amp;s=1&amp;up=0&amp;a=22&amp;ng=w&amp;l=zh&amp;uri=%2Fartworks%2F_PARAM_&amp;is_spa=1&amp;K=59bba275c645c&amp;ab_test_digits_first=32&amp;yuid=NDJ3gQk&amp;suid=Pggb9mua442sjsueo&amp;num=5efbf5be568&amp;t=_bee-JX46i&amp;t=b8b4-hqot7&amp;t=kY01H5r3Pd"
},
"header":{
"url":"https://pixon.ads-pixiv.net/show?zone_id=header&amp;format=js&amp;s=1&amp;up=0&amp;a=22&amp;ng=w&amp;l=zh&amp;uri=%2Fartworks%2F_PARAM_&amp;is_spa=1&amp;K=59bba275c645c&amp;ab_test_digits_first=32&amp;yuid=NDJ3gQk&amp;suid=Pggb9mua45spzoimt&amp;num=5efbf5be155&amp;t=_bee-JX46i&amp;t=b8b4-hqot7&amp;t=kY01H5r3Pd"
},
"footer":{
"url":"https://pixon.ads-pixiv.net/show?zone_id=footer&amp;format=js&amp;s=1&amp;up=0&amp;a=22&amp;ng=w&amp;l=zh&amp;uri=%2Fartworks%2F_PARAM_&amp;is_spa=1&amp;K=59bba275c645c&amp;ab_test_digits_first=32&amp;yuid=NDJ3gQk&amp;suid=Pggb9mua47f9zcoim&amp;num=5efbf5be400&amp;t=_bee-JX46i&amp;t=b8b4-hqot7&amp;t=kY01H5r3Pd"
},
"expandedFooter":{
"url":"https://pixon.ads-pixiv.net/show?zone_id=multiple_illust_viewer&amp;format=js&amp;s=1&amp;up=0&amp;a=22&amp;ng=w&amp;l=zh&amp;uri=%2Fartworks%2F_PARAM_&amp;is_spa=1&amp;K=59bba275c645c&amp;ab_test_digits_first=32&amp;yuid=NDJ3gQk&amp;suid=Pggb9mua4928ct0yw&amp;num=5efbf5be471&amp;t=_bee-JX46i&amp;t=b8b4-hqot7&amp;t=kY01H5r3Pd"
},
"logo":{
"url":"https://pixon.ads-pixiv.net/show?zone_id=logo_side&amp;format=js&amp;s=1&amp;up=0&amp;a=22&amp;ng=w&amp;l=zh&amp;uri=%2Fartworks%2F_PARAM_&amp;is_spa=1&amp;K=59bba275c645c&amp;ab_test_digits_first=32&amp;yuid=NDJ3gQk&amp;suid=Pggb9mua4aqu6i4sr&amp;num=5efbf5be844&amp;t=_bee-JX46i&amp;t=b8b4-hqot7&amp;t=kY01H5r3Pd"
}
},
"extraData":{
"meta":{
"title":"#プリンセスコネクト!Re:Dive 水着キャルちゃん! - 秋鳩むぎ的插画 - pixiv",
"description":"この作品 「水着キャルちゃん!」 は 「プリンセスコネクト!Re:Dive」「キャル(プリコネ)」 等のタグがつけられた「秋鳩むぎ」さんのイラストです。 「水着のキャルちゃんはかわいいぞ!!」",
"canonical":"https://www.pixiv.net/artworks/82647306",
"alternateLanguages":{
"ja":"https://www.pixiv.net/artworks/82647306",
"en":"https://www.pixiv.net/en/artworks/82647306"
},
"descriptionHeader":"本作「水着キャルちゃん!」为附有「プリンセスコネクト!Re:Dive」「キャル(プリコネ)」等标签的插画。",
"ogp":{
"description":"水着のキャルちゃんはかわいいぞ!!",
"image":"https://embed.pixiv.net/decorate.php?illust_id=82647306",
"title":"#プリンセスコネクト!Re:Dive 水着キャルちゃん! - 秋鳩むぎ的插画 - pixiv",
"type":"article"
},
"twitter":{
"description":"水着のキャルちゃんはかわいいぞ!!",
"image":"https://embed.pixiv.net/decorate.php?illust_id=82647306",
"title":"水着キャルちゃん!",
"card":"summary_large_image"
}
}
},
"titleCaptionTranslation":{
"workTitle":null,
"workCaption":null
}
}
},
"user":{
"55859246":{
"userId":"55859246",
"name":"秋鳩むぎ",
"image":"https://i.pximg.net/user-profile/img/2020/06/29/21/20/14/18907670_b3f87d819f705ec418f120cd57f9dc41_50.jpg",
"imageBig":"https://i.pximg.net/user-profile/img/2020/06/29/21/20/14/18907670_b3f87d819f705ec418f120cd57f9dc41_170.jpg",
"premium":false,
"isFollowed":false,
"isMypixiv":false,
"isBlocking":false,
"background":null,
"partial":0
}
}
}
```
#### 字段说明 ####
- `timestamp`: (`string`) 请求时间
- `illust`: (`Object`) 作品预加载信息
- `{illustId}`: 作品ID(跟页面请求的IllustId一样)
- `illustId`: (`string` -> `number`) 作品Id
- `illustTitle`: (`string`) 作品标题
- `illustComment`: (`string`) 作品说明
- `id`: (`string` -> `number`) 与`illustId`一致, 猜测是以兼容旧版本为目录而存在
- `title`: (`string`) 与`illustTitle`一致, 猜测是以兼容旧版本为目录而存在
- `description`: (`string`) 作品说明
- `illustType`: (`number`) 作品类型
- `0`: 插画作品
- `1`: 漫画作品
- `2`: 动图作品
- `createDate`: (`string`) 作品创建时间(或者是完成时间?)
- `updateDate`: (`string`) 作品上传时间
- `restrict`: (`number`) 作品限制级(意义不明, 可能是兼容性问题?)?
- `xRestrict`: (`number`) 作品是否为限制级, 基本准确, 少部分不一定(看Pixiv审核怎么理解了)
- `0`: 非限制级内容(即非R18作品)
- `1`: 限制级内容(即R18作品)
- `sl`: (`number`) 不明?
- `urls`: (`string`) 作品图片链接, 需要`Referer`请求头
- `mini`: (`string`) 小尺寸预览图
- `thumb`: (`string`) 小尺寸预览图
- `small`: (`string`) 小尺寸预览图
- `regular`: (`string`) 经压缩,没啥画质损失的原尺寸预览图
- `original`: (`string`) 原图
- `tags`: (`Object`) 作品标签信息
- `authorId`: (`string` -> `number`) 作者用户Id
- `isLocked`: (`boolean`) 标签是否锁定(即不可被访客更改)
- `tags`: (`Object[]`) 标签信息数组
- `tag`: (`string`) 标签原始名
- `locked`: (`boolean`) 标签是否不可更改
- `deletable`: (`boolean`) 标签能否被删除?
- `userId`: (`string` -> `number`) 用户Id
- `translation`: (`Object`) 标签翻译
- `{语种}`: 翻译名
- `userName`: (`string`) 用户名
- `alt`: (`string`) 简略介绍信息(在图片加载失败时可提供给`img`标签使用)
- `storableTags`: (`string[]`) 不明?
- `userId`: (`string` -> `number`) 作者用户Id
- `userName`: (`string`) 作者用户名
- `userAccount`: (`string`) 作者登录名
- `userIllusts`: (`Object`) 作品信息?
- `{IllustId}`: 与请求IllustId一样
- (请转到Pixiv作品信息获取接口.md)
- `likeData`: (`boolean?`) 不明?
- `width`: (`number`) 作品长度
- `height`: (`number`) 作品高度
- `pageCount`: (`number`) 作品页数
- `bookmarkCount`: (`number`) 作品公开的收藏数
- `likeCount`: (`number`) 作品喜欢(点赞)数
- `commentCount`: (`number`) 作品评论数
- `responseCount`: (`number`) 作品响应数?
- `viewCount`: (`number`) 作品阅览数
- `isHowto`: (`boolean`) 不明?
- `isOriginal`: (`boolean`) 不明?
- `imageResponseOutData`: (`Unknown[]`) 不明?
- `imageResponseData`: (`Unknown[]`) 不明?
- `imageResponseCount`: (`number`) 不明?
- `pollData`: (`Unknown`) 不明?
- `seriesNavData`: (`Unknown`) 不明?
- `descriptionBoothId`: (`Unknown`) 不明?
- `descriptionYoutubeId`: (`Unknown`) 不明?
- `comicPromotion`: (`Unknown`) 不明?
- `fanboxPromotion`: (`Unknown`) 不明?
- `contestBanners`: (`Unknown[]`) 不明?
// TODO 待补充
- `isBookmarkable`: (`boolean`) 不明?
- `bookmarkData`: (`Unknown`) 不明?
- `contestData`: (`Unknown`) 不明?
- `zoneConfig`: (`Object`) 猜测是广告信息?
- (基本不用, 忽略...)
- `extraData`: (`Object`) 扩展数据
- `meta`: (`Object`) 元数据
- `title`: (`string`) 网页标题
- `description`: (`string`) Pixiv生成的作品说明
- `canonical`: (`string`) 作品页面链接
- `alternateLanguages`: (`Object`) 不同语言的作品页面链接
- `{语种}`: (`string`) 对应语种的作品链接
- `descriptionHeader`: (`string`) 说明文档(不过似乎是对应了会话所属账号的语种?)
- `ogp`: (`Object`) 猜测是某平台的分享数据?
- `description`: (`string`) 说明内容
- `image`: (`string`) 预览图链接
- `title`: (`string`) 分享标题
- `type`: (`string`) 分享类型?
- `twitter`: (`Object`)
- `description`: (`string`) 说明内容
- `image`: (`string`) 预览图链接
- `title`: (`string`) 分享标题
- `card`: (`string`) 分享类型?
- `titleCaptionTranslation`: (`Object`) 不明?
- `workTitle`: (`Unknown`) 不明?
- `workCaption`: (`Unknown`) 不明?
- `user`: (`Object`) 作者预加载信息
- `{userId}`: 可通过`illust.{illustId}.userId`获得
- `userId`: (`string` -> `number`) 作者用户Id
- `name`: (`string`) 作者用户名
- `image`: (`string`) 作者用户头像(小尺寸)
- `imageBig`: (`string`) 作者用户头像(大尺寸)
- `premium`: (`boolean`) 作者是否为Pixiv会员
- `isFollowed`: (`boolean`) 当前会话用户是否已关注
- `isMypixiv`: (`boolean`) 是否为当前会话本人?
- `isBlocking`: (`boolean`) 是否正在被封禁
- `background`: (`Object`) 背景图片?
- `partial`: (`number`) 不明?

View File

@ -0,0 +1,129 @@
## Pixiv首页数据接口 ##
### 说明 ###
> 注意: 该接口涉及用户账户隐私, 不要尝试对该接口返回数据做不安全云端存储, 或未经用户允许的发送出去.
该接口用于获取Pixiv推荐给账号的首页作品信息每次调用都会有不同结果。
### 接口地址 ###
```
GET https://www.pixiv.net/ajax/top/{type}?mode={mode}&lang={lang}
```
### 参数 ###
- `type`: 首页类型
- `illust`: 插画
- `manga`: 漫画
- `novel`: 小说
- `mode`: 内容类型
- `all`: 不限类型
- `r18`: 成人内容
- `lang`: 语言(只写几个)
- `zh`: 中文
- 是否需要登录: `是`
- 是否为Pixiv接口标准返回格式: `是`
- 是否需要Referer请求头: `是`
### 请求示例 ###
```
GET https://www.pixiv.net/ajax/top/illust?mode=all&lang=zh
```
### 返回数据 ###
#### 数据示例 ####
```
(内容过长, 略)
```
#### 字段说明 ####
- `page`: 网页相关内容
- `tags`: (`Object[]`) 热门标签
- `tag`: (`String`) 标签名
- `count`: (`Number`) 作品数量?
- `lev`: (`Number`) 不明?
- `ids`: (`Number[]`) 作品数组
- `follow`: (`Number[]`) 已关注作者的作品推荐
- `mypixiv`: (`?[]`) 不明
- `recommend`: (`String[]` -> `Number[]`) 推荐作品的Id?
- `recommendByTag`: (`Object[]`) 标签的推荐作品
- `tag`: (`String`) 标签名
- `ids`: ((`String[]` -> `Number[]`) 作品Id数组
- `ranking`: (`Object`) 排行榜前100名数据
- `date`: (`String`) 排行榜日期(`yyyyMMdd`)
- `items`: (`Object[]`) 排行榜简略数据
- `rank`: (`String` -> `Number`) 排行榜名次
- `id`: (`String` -> `Number`) 作品Id
- `pixivision`: (`Object[]`) Pixiv的推荐文章
- `id`: (`String` -> `Number`) 文章Id
- `title`: (`String`) 文章标题
- `thumbnailUrl`: (`String`) 文章封面图的Url地址
- `abtestId`: (`String` -> `Number`) pixivision文章地址的参数`utm_content`的值
- `recommendUser`: (`Object[]`) 推荐用户及其作品
- `id`: (`Number`) 用户Id
- `illustIds`: (`String[]` -> `Number[]`) 插画作品Id
- `novelIds`: (`String[]` -> `Number[]`) 小说作品Id
- `contestOngoing`: (`Object[]`) 进行中的比赛活动(没找到更多信息了, 先这么说着)
- (待完善)
- `contestResult`: (`Object[]`) 比赛结果信息
- `slug`: (`String`) 比赛代号?
- `type`: (`String`) 比赛作品类型?
- `name`: (`String`) 比赛名
- `url`: (`String`) 结果公布链接
- `iconUrl`: (`String`) 图标链接(不明意义的图标)
- `editorRecommend`: (`Object[]`) 编辑推荐(小说), 后续可能会删除, 这个应该是配合活动出的数据
- (待完善)
- `boothFollowItemIds`: (`String[]` -> `Number[]`) 已关注用户的最新商品
- `sketchLiveFollowIds`: (`?[]`) 不明?
- `sketchLivePopularIds`: (`String[]` -> `Number[]`) 不明?
- `myFavoriteTags`: (`?[]`) 关注的标签
- `newPost`: (`String[]` -> `Number[]`) 不明?
- `thumbnails`: (`Object`) 已关注用户的作品
- `illust`: (`Object[]`) 插画作品
- `illustId`: (`String` -> `Number`) 作品Id(或者准确来讲是 插画Id?)
- `illustTitle`: (`String`) 插画标题
- `id`: (`String` -> `Number`) 作品Id
- `title`: (`String`) 作品标题
- `illustType`: (`Number`) 作品类型
- `xRestrict`: (`Number`) 不明?
- `restrict`: (`Number`) 不明?
- `sl`: (`Number`) 不明?
- `url`: (`String`) 作品在主页的封面图Url
- `description`: (`String`) 作品说明?
- `tags`: (`String[]`) 标签原始名数组(不带翻译的原始名称)
- `userId`: (`String` -> `Number`) 用户Id
- `userName`: (`String`) 用户名
- `width`: (`Number`) 作品宽度
- `height`: (`Number`) 作品高度
- `pageCount`: (`Number`) 作品页数
- `isBookmarkable`: (`Boolean`) 不明?
- `bookmarkData`: (`?`) 不明?
- `alt`: (`String`) 简略信息
- `isAdContainer`: (`Boolean`) 广告标识?
- `titleCaptionTranslation`: (`Object`) 不明?
- (待完善)
- `urls`: (`Object`) 其他封面图尺寸的链接
- (略)
- `seriesId`: (`?`) 不明?
- `seriesTitle`: (`?`) 不明?
- `profileImageUrl`: (`String`) 用户头像图链接
- `users`: (`Object[]`) 用户信?
- `userId`: (`String` -> `Number`) 用户Id
- `name`: (`String`) 用户名
- `image`: (`String`) 用户头像图链接
- `imageBig`: (`String`) 用户大尺寸头像图链接
- `premium`: (`Boolean`) 是否为Pixiv高级会员
- `isFollowed`: (`Boolean`) 是否关注
- `isMypixiv`: (`Boolean`) 不明?
- `isBlocking`: (`Boolean`) 是否为黑名单?
- `background`: (`?`) 不明?
- `partial`: (`Number`) 不明?
- `tagTranslation`: 标签翻译名
- `key=${标签原始名}`: (`Object`) 标签翻译信息
- `${语言代码}`: (`String`) 对应语言代码的翻译, 不一定有
- `boothItems`: (`Object[]`) 商品信息
- (待完善)
- `sketchLives`: (`Object[]`) 不明?
- (待完善)
- `zoneConfig`: (`Object`) 不明?
- (待完善)

81
pom.xml
View File

@ -4,9 +4,17 @@
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>CGJ_2</artifactId>
<version>2.1.0</version>
<groupId>net.lamgc</groupId>
<artifactId>ContentGrabbingJi</artifactId>
<version>2.5.2-20200709.1-SNAPSHOT</version>
<licenses>
<license>
<name>GNU Affero General Public License 3.0</name>
<url>https://www.gnu.org/licenses/agpl-3.0.txt</url>
<distribution>repo</distribution>
</license>
</licenses>
<repositories>
<repository>
@ -19,7 +27,7 @@
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<maven.compiler.encoding>UTF-8</maven.compiler.encoding>
<mirai.CoreVersion>0.32.0</mirai.CoreVersion>
<mirai.CoreVersion>1.0.4</mirai.CoreVersion>
<mirai.JaptVersion>1.1.1</mirai.JaptVersion>
<kotlin.version>1.3.71</kotlin.version>
<ktor.version>1.3.2</ktor.version>
@ -30,14 +38,26 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<source>8</source>
<target>8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.0.2</version>
<configuration>
<excludes>
<exclude>log4j2-test.xml</exclude>
</excludes>
</configuration>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>2.2.6.RELEASE</version>
<configuration>
<mainClass>net.lamgc.cgj.Main</mainClass>
<classifier>exec</classifier>
@ -50,20 +70,6 @@
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-maven-plugin</artifactId>
<version>${kotlin.version}</version>
<executions>
<execution>
<id>compile</id>
<phase>process-sources</phase>
<goals>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
@ -88,7 +94,7 @@
<dependency>
<groupId>net.lamgc</groupId>
<artifactId>java-utils</artifactId>
<version>1.1.0_5-SNAPSHOT</version>
<version>1.3.1</version>
</dependency>
<dependency>
@ -112,25 +118,10 @@
<artifactId>gson</artifactId>
<version>2.8.5</version>
</dependency>
<dependency>
<groupId>net.sourceforge.htmlunit</groupId>
<artifactId>htmlunit</artifactId>
<version>2.29</version>
</dependency>
<dependency>
<groupId>com.github.rholder</groupId>
<artifactId>guava-retrying</artifactId>
<version>2.0.0</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>8.0.19</version>
</dependency>
<dependency>
<groupId>net.lz1998</groupId>
<artifactId>spring-cq</artifactId>
<version>4.14.0.6</version>
<version>4.15.0.1</version>
<exclusions>
<exclusion>
<groupId>ch.qos.logback</groupId>
@ -153,12 +144,12 @@
</dependency>
<dependency>
<groupId>net.mamoe</groupId>
<artifactId>mirai-core-jvm</artifactId>
<artifactId>mirai-core</artifactId>
<version>${mirai.CoreVersion}</version>
</dependency>
<dependency>
<groupId>net.mamoe</groupId>
<artifactId>mirai-core-qqandroid-jvm</artifactId>
<artifactId>mirai-core-qqandroid</artifactId>
<version>${mirai.CoreVersion}</version>
</dependency>
<dependency>
@ -171,16 +162,22 @@
<artifactId>ktor-server-core</artifactId>
<version>${ktor.version}</version>
</dependency>
<dependency>
<groupId>net.lamgc</groupId>
<artifactId>PixivLoginProxyServer</artifactId>
<version>1.1.0</version>
</dependency>
<dependency>
<groupId>com.squareup</groupId>
<artifactId>gifencoder</artifactId>
<version>0.10.1</version>
</dependency>
<dependency>
<groupId>org.jline</groupId>
<artifactId>jline</artifactId>
<version>3.15.0</version>
</dependency>
<dependency>
<groupId>com.google.inject</groupId>
<artifactId>guice</artifactId>
<version>4.2.3</version>
</dependency>
</dependencies>
</project>

View File

@ -1,24 +0,0 @@
搜索标签信息https://www.pixiv.net/ajax/search/tags/标签名
搜索接口:
https://www.pixiv.net/ajax/search/{Type}/搜索内容
Type = illustrations(插画) / top(顶部?) / manga(漫画) / novels(小说)
word=搜索内容 [参数可能不是必须的]
s_mode=s_tag(标签-部分一致) / s_tag_full(标签-完全一致) / s_tc(标题、说明文字)
type= all(插画、漫画、动图_动态插图) / illust_and_ugoira(插画、动图) / illust(插画) / manga(漫画) / ugoira(动图)
p=页数 [超出页数的情况下将获取不到数据(即"body.illust.data"是空数组)]
order=date(按旧排序) / date_d(按新排序) / Unknown(按热门度排序, 需要会员)
mode= all(全部) / safe(全年龄) / r18(咳咳)
可选参数:
wlt=最小宽度像素
wgt=最高宽度像素
hlt=最小高度像素
hgt=最高高度像素
ratio=0.5(横图) / -0.5(纵图) / 0(正方形) [可能不能改变参数, 三个值是固定的]
tool=使用工具, 不是很重要晚些再加
scd=开始时间(yyyy-MM-dd)
ecd=结束时间(yyyy-MM-dd)
最小收藏数 = 收藏数限定参数为会员功能, 无法获取

View File

@ -1,135 +1,123 @@
package net.lamgc.cgj;
import com.github.monkeywie.proxyee.proxy.ProxyConfig;
import com.github.monkeywie.proxyee.proxy.ProxyType;
import com.google.common.base.Strings;
import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.CQConfig;
import net.lamgc.cgj.bot.MiraiMain;
import net.lamgc.cgj.pixiv.*;
import net.lamgc.cgj.proxy.PixivAccessProxyServer;
import net.lamgc.plps.PixivLoginProxyServer;
import net.lamgc.utils.base.ArgumentsProperties;
import net.lamgc.cgj.bot.boot.ApplicationBoot;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.framework.FrameworkManager;
import net.lamgc.cgj.bot.framework.cli.ConsoleMain;
import net.lamgc.cgj.bot.framework.coolq.SpringCQApplication;
import net.lamgc.cgj.bot.framework.mirai.MiraiMain;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivSearchLinkBuilder;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.utils.base.runner.Argument;
import net.lamgc.utils.base.runner.ArgumentsRunner;
import net.lamgc.utils.base.runner.Command;
import org.apache.commons.io.IOUtils;
import net.lamgc.utils.encrypt.MessageDigestUtils;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.client.CookieStore;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.message.BasicHeader;
import org.apache.http.util.EntityUtils;
import org.apache.tomcat.util.http.fileupload.util.Streams;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import java.io.*;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
@SpringBootApplication
public class Main {
private final static Logger log = LoggerFactory.getLogger("Main");
private final static Logger log = LoggerFactory.getLogger(Main.class);
private final static File storeDir = new File("store/");
public static CookieStore cookieStore;
private static CookieStore cookieStore;
public static HttpHost proxy;
private static HttpHost proxy;
public static void main(String[] args) throws IOException, ClassNotFoundException {
ArgumentsProperties argsProp = new ArgumentsProperties(args);
if(argsProp.containsKey("proxy")) {
URL proxyUrl = new URL(argsProp.getValue("proxy"));
proxy = new HttpHost(proxyUrl.getHost(), proxyUrl.getPort());
log.info("已启用Http协议代理{}", proxy.toHostString());
if(args.length != 0 && args[0].equalsIgnoreCase("buildpassword")) {
ArgumentsRunner.run(Main.class, args);
} else {
proxy = null;
standardStart(args);
}
}
if(!storeDir.exists() && !storeDir.mkdirs()) {
log.error("创建文件夹失败!");
}
private static void standardStart(String[] args) throws IOException, ClassNotFoundException {
log.info("ContentGrabbingJi 正在启动...");
log.debug("Args: {}, LogsPath: {}", Arrays.toString(args), System.getProperty("cgj.logsPath"));
log.debug("运行目录: {}", System.getProperty("user.dir"));
// TODO: 需要修改参数名了, 大概改成类似于 workerDir这样的吧
if(argsProp.containsKey("cqRootDir")) {
log.info("cqRootDir: {}", argsProp.getValue("cqRootDir"));
System.setProperty("cgj.cqRootDir", argsProp.getValue("cqRootDir"));
} else {
log.warn("未设置cqRootDir, 当前运行目录将作为酷Q机器人所在目录.");
System.setProperty("cgj.cqRootDir", "./");
}
ApplicationBoot.initialApplication(args);
log.debug("botDataDir: {}", System.getProperty("cgj.botDataDir"));
if(argsProp.containsKey("redisAddr")) {
log.info("redisAddress: {}", argsProp.getValue("redisAddr"));
System.setProperty("cgj.redisAddress", argsProp.getValue("redisAddr"));
} else {
log.info("未设置RedisAddress, 将使用默认值连接Redis服务器(127.0.0.1:6379)");
System.setProperty("cgj.redisAddress", "127.0.0.1");
}
File cookieStoreFile = new File("cookies.store");
proxy = BotGlobal.getGlobal().getProxy();
File cookieStoreFile = new File(BotGlobal.getGlobal().getDataStoreDir(), "cookies.store");
if(!cookieStoreFile.exists()) {
log.error("未找到cookies.store文件, 是否启动PixivLoginProxyServer? (yes/no)");
Scanner scanner = new Scanner(System.in);
if(scanner.nextLine().equalsIgnoreCase("yes")) {
startPixivLoginProxyServer();
} else {
System.exit(1);
return;
}
log.warn("未找到cookies.store文件, 请检查数据存放目录下是否存在'cookies.store'文件!");
System.exit(1);
return;
}
ObjectInputStream ois = new ObjectInputStream(new FileInputStream(cookieStoreFile));
cookieStore = (CookieStore) ois.readObject();
BotGlobal.getGlobal().setCookieStore(cookieStore);
ois.close();
log.info("已载入CookieStore");
log.debug("传入参数: {}", Arrays.toString(args));
ArgumentsRunner.run(Main.class, args);
System.exit(0);
}
@Command
public static void buildPassword(@Argument(name = "password") String password) {
System.out.println("Password: " +
Base64.getEncoder().encodeToString(MessageDigestUtils.encrypt(password.getBytes(),
MessageDigestUtils.Algorithm.MD5)));
}
@Command
public static void botMode(@Argument(name = "args", force = false) String argsStr) {
new MiraiMain().init();
try {
FrameworkManager.registerFramework(new MiraiMain()).join();
} catch (InterruptedException ignored) {
}
}
@Command
public static void consoleMode() {
try {
FrameworkManager.registerFramework(new ConsoleMain()).join();
} catch (InterruptedException ignored) {
}
}
@Command
public static void pluginMode(@Argument(name = "args", force = false) String argsStr) {
if(!System.getProperty("cgj.cqRootDir").endsWith("\\") && !System.getProperty("cgj.cqRootDir").endsWith("/")) {
System.setProperty("cgj.cqRootDir", System.getProperty("cgj.cqRootDir") + "/");
try {
FrameworkManager.registerFramework(new SpringCQApplication()).join();
} catch (InterruptedException ignored) {
}
log.info("酷Q机器人根目录: {}", System.getProperty("cgj.cqRootDir"));
CQConfig.init();
Pattern pattern = Pattern.compile("/\\s*(\".+?\"|[^:\\s])+((\\s*:\\s*(\".+?\"|[^\\s])+)|)|(\".+?\"|[^\"\\s])+");
Matcher matcher = pattern.matcher(Strings.nullToEmpty(argsStr));
ArrayList<String> argsList = new ArrayList<>();
while (matcher.find()) {
argsList.add(matcher.group());
}
String[] args = new String[argsList.size()];
argsList.toArray(args);
SpringApplication.run(Main.class, args);
}
@Command
public static void collectionDownload() throws IOException {
PixivDownload pixivDownload = new PixivDownload(Objects.requireNonNull(cookieStore), proxy);
File outputFile = new File(storeDir, "collection.zip");
File outputFile = new File(getStoreDir(), "collection.zip");
if(!outputFile.exists() && !outputFile.createNewFile()) {
log.error("文件创建失败: " + outputFile.getAbsolutePath());
}
@ -141,7 +129,7 @@ public class Main {
ZipEntry entry = new ZipEntry(link.substring(link.lastIndexOf("/") + 1));
log.info("正在写入: " + entry.getName());
zos.putNextEntry(entry);
IOUtils.copy(inputStream, zos);
Streams.copy(inputStream, zos, false);
zos.flush();
} catch (IOException e) {
log.error("写入文件项时发生异常", e);
@ -156,10 +144,10 @@ public class Main {
PixivDownload pixivDownload = new PixivDownload(Objects.requireNonNull(cookieStore), proxy);
String date = new SimpleDateFormat("yyyyMMdd").format(new Date());
int id = 1;
File outputFile = new File(storeDir, "recommends-" + date + "-" + id + ".zip");
File outputFile = new File(getStoreDir(), "recommends-" + date + "-" + id + ".zip");
while(outputFile.exists()) {
id++;
outputFile = new File(storeDir, "recommends-" + date + "-" + id + ".zip");
outputFile = new File(getStoreDir(), "recommends-" + date + "-" + id + ".zip");
}
if(!outputFile.createNewFile()) {
@ -173,7 +161,7 @@ public class Main {
ZipEntry entry = new ZipEntry(link.substring(link.lastIndexOf("/") + 1));
log.info("正在写入: " + entry.getName());
zos.putNextEntry(entry);
IOUtils.copy(inputStream, zos);
Streams.copy(inputStream, zos, false);
zos.flush();
log.info("已成功写入 {}", entry.getName());
} catch (IOException e) {
@ -223,10 +211,10 @@ public class Main {
}
int id = 1;
File outputFile = new File(storeDir, "ranking" + rankingMode.modeParam + "-" + date + "-" + id + ".zip");
File outputFile = new File(getStoreDir(), "ranking" + rankingMode.modeParam + "-" + date + "-" + id + ".zip");
while(outputFile.exists()) {
id++;
outputFile = new File(storeDir, "ranking" + rankingMode.modeParam + "-" + date + "-" + id + ".zip");
outputFile = new File(getStoreDir(), "ranking" + rankingMode.modeParam + "-" + date + "-" + id + ".zip");
}
if(!outputFile.createNewFile()) {
@ -244,7 +232,7 @@ public class Main {
entry.setComment(rankInfo.toString());
log.info("正在写入: " + entry.getName());
zos.putNextEntry(entry);
IOUtils.copy(inputStream, zos);
Streams.copy(inputStream, zos, false);
zos.flush();
log.info("已成功写入 {}", entry.getName());
} catch (IOException e) {
@ -268,24 +256,24 @@ public class Main {
@Argument(name = "excludeKeywords", force = false) String excludeKeywords,
@Argument(name = "contentOption", force = false) String contentOption
) throws IOException {
PixivSearchBuilder searchBuilder = new PixivSearchBuilder(Strings.isNullOrEmpty(content) ? "" : content);
PixivSearchLinkBuilder searchBuilder = new PixivSearchLinkBuilder(Strings.isNullOrEmpty(content) ? "" : content);
if (type != null) {
try {
searchBuilder.setSearchType(PixivSearchBuilder.SearchType.valueOf(type.toUpperCase()));
searchBuilder.setSearchType(PixivSearchLinkBuilder.SearchType.valueOf(type.toUpperCase()));
} catch (IllegalArgumentException e) {
log.warn("不支持的SearchType: {}", type);
}
}
if(area != null) {
try {
searchBuilder.setSearchArea(PixivSearchBuilder.SearchArea.valueOf(area));
searchBuilder.setSearchArea(PixivSearchLinkBuilder.SearchArea.valueOf(area));
} catch (IllegalArgumentException e) {
log.warn("不支持的SearchArea: {}", area);
}
}
if(contentOption != null) {
try {
searchBuilder.setSearchContentOption(PixivSearchBuilder.SearchContentOption.valueOf(contentOption));
searchBuilder.setSearchContentOption(PixivSearchLinkBuilder.SearchContentOption.valueOf(contentOption));
} catch (IllegalArgumentException e) {
log.warn("不支持的SearchContentOption: {}", contentOption);
}
@ -324,7 +312,7 @@ public class Main {
JsonObject resultBody = jsonObject.getAsJsonObject("body");
for(PixivSearchBuilder.SearchArea searchArea : PixivSearchBuilder.SearchArea.values()) {
for(PixivSearchLinkBuilder.SearchArea searchArea : PixivSearchLinkBuilder.SearchArea.values()) {
if(!resultBody.has(searchArea.jsonKey) || resultBody.getAsJsonObject(searchArea.jsonKey).getAsJsonArray("data").size() == 0) {
//log.info("返回数据不包含 {}", searchArea.jsonKey);
continue;
@ -353,204 +341,21 @@ public class Main {
builder,
PixivURL.getPixivRefererLink(illustId)
);
/*log.info("正在下载...");
List<String> list = PixivDownload.getIllustAllPageDownload(
HttpClientBuilder.create()
.setProxy(proxy)
.build(),
illustId, PixivDownload.PageQuality.ORIGINAL);*/
}
}
/*
if(searchBuilder.getSearchArea().equals(PixivSearchBuilder.SearchArea.TOP)) {
} else {
JsonArray illustsArray = resultBody
.getAsJsonObject(searchBuilder.getSearchArea().jsonKey).getAsJsonArray("data");
log.info("已找到与 {} 相关插图信息:", content);
int count = 1;
for (JsonElement jsonElement : illustsArray) {
JsonObject illustObj = jsonElement.getAsJsonObject();
//TODO: 防止数据内混入无效内容, 需要检查对象是否有illustId
if(!illustObj.has("illustId")) {
continue;
}
int illustId = illustObj.get("illustId").getAsInt();
StringBuilder builder = new StringBuilder("[");
illustObj.get("tags").getAsJsonArray().forEach(el -> builder.append(el.getAsString()).append(", "));
builder.replace(builder.length() - 2, builder.length(), "]");
log.info("({} / {})\n\t作品id: {}, \n\t作者名(作者id): {} ({}), \n\t作品标题: {}, \n\t作品Tags: {}, \n\t作品链接: {}",
count++,
illustsArray.size(),
illustId,
illustObj.get("userName").getAsString(),
illustObj.get("userId").getAsInt(),
illustObj.get("illustTitle").getAsString(),
builder,
PixivURL.getPixivRefererLink(illustId)
);
}
}
*/
}
@Command(defaultCommand = true)
public static void testRun() {
/*loadCookieStoreFromFile();
if(cookieStore == null){
startPixivLoginProxyServer();
}*/
//accessPixivToFile();
//startPixivAccessProxyServer();
//saveCookieStoreToFile();
log.info("这里啥都没有哟w");
}
private static void loadCookieStoreFromFile() throws IOException {
log.info("正在加载CookieStore...");
File storeFile = new File("./cookies.store");
if(!storeFile.exists()){
log.info("未找到CookieStore, 跳过加载.");
return;
private static File getStoreDir() {
if(!storeDir.exists() && !storeDir.mkdirs()) {
log.error("创建文件夹失败!");
}
ObjectInputStream stream = new ObjectInputStream(new FileInputStream(storeFile));
Object result;
try {
result = stream.readObject();
} catch (ClassNotFoundException e) {
log.error("加载出错", e);
return;
}
cookieStore = (CookieStore) result;
cookieStore.getCookies().forEach(cookie -> log.debug(cookie.getName() + ": " + cookie.getValue() + ", isExpired: " + cookie.isExpired(new Date())));
log.info("CookieStore加载完成.");
return storeDir;
}
private static void saveCookieStoreToFile() throws IOException {
log.info("正在保存CookieStore...");
File outputFile = new File("./cookies.store");
if(!outputFile.exists() && !outputFile.delete() && !outputFile.createNewFile()){
log.error("保存CookieStore失败.");
return;
}
ObjectOutputStream stream = new ObjectOutputStream(new FileOutputStream(outputFile));
stream.writeObject(cookieStore);
stream.flush();
stream.close();
log.info("CookieStore保存成功.");
}
private static void startPixivLoginProxyServer(){
ProxyConfig proxyConfig = null;
if(proxy != null) {
proxyConfig = new ProxyConfig(ProxyType.HTTP, proxy.getHostName(), proxy.getPort());
}
PixivLoginProxyServer proxyServer = new PixivLoginProxyServer(proxyConfig);
Thread proxyServerStartThread = new Thread(() -> {
log.info("启动代理服务器...");
proxyServer.start(1006);
log.info("代理服务器已关闭.");
});
proxyServerStartThread.setName("LoginProxyServerThread");
proxyServerStartThread.start();
//System.console().readLine();
Scanner scanner = new Scanner(System.in);
log.info("登录完成后, 使用\"done\"命令结束登录过程.");
while(true) {
if (scanner.nextLine().equalsIgnoreCase("done")) {
log.info("关闭PLPS服务器...");
proxyServer.close();
cookieStore = proxyServer.getCookieStore();
try {
log.info("正在保存CookieStore...");
saveCookieStoreToFile();
log.info("CookieStore保存完成.");
} catch (IOException e) {
log.error("CookieStore保存时发生异常, 本次CookieStore仅可在本次运行使用.", e);
}
break;
} else {
log.warn("要结束登录过程, 请使用\"done\"命令.");
}
}
}
private static void accessPixivToFile() throws IOException {
File cookieStoreFile = new File("./cookie.txt");
if (!cookieStoreFile.exists() && !cookieStoreFile.createNewFile()) {
log.info("Cookie文件存储失败");
}
/*log.info("正在写出Cookie, Cookie count: " + cookieStore.getCookies().size());
FileWriter cookieWriter = new FileWriter(cookieStoreFile);
cookieStore.getCookies().forEach(cookie -> {
try {
StringBuilder sb = new StringBuilder().append(cookie.getName()).append(" = ").append(cookie.getValue());
log.info("正在导出Cookie: " + sb.toString());
cookieWriter.append(sb.toString()).append("\n").flush();
} catch (IOException e) {
e.printStackTrace();
}
});
log.info("Cookie写出完成");*/
log.info("尝试通过捕获的Cookie访问Pixiv...");
HttpClient httpClient = new PixivSession(new HttpHost("127.0.0.1", 1080), cookieStore).getHttpClient();
HttpGet request = new HttpGet("https://www.pixiv.net");
request.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:73.0) Gecko/20100101 Firefox/73.0");
request.addHeader(new BasicHeader("accept-encoding", "gzip, deflate, br"));
request.addHeader(new BasicHeader("accept-language", "zh-CN,zh;q=0.9"));
StringBuilder cookieBuilder = new StringBuilder();
cookieStore.getCookies().forEach(cookie -> {
if(cookie.isExpired(new Date())){
return;
}
cookieBuilder.append(cookie.getName()).append("=").append(cookie.getValue()).append("; ");
});
request.addHeader("cookie", cookieBuilder.toString());
HttpResponse response = httpClient.execute(request);
log.info("正在写入文件...");
File outFile = new File("./pixiv.html");
if (outFile.createNewFile() && !outFile.exists()) {
log.info("文件创建失败!");
}else {
new FileWriter(outFile).append(EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8)).flush();
}
Pixiv pixiv = new Pixiv(httpClient);
pixiv.getRecommend().forEach(illustMap -> {
StringBuilder builder = new StringBuilder();
illustMap.forEach((key, value) -> builder.append(key).append(": ").append(value).append("\n"));
try {
builder.append("download Link: ").append(Arrays.toString(pixiv.getAllDownloadLink(Integer.parseInt(illustMap.get(Pixiv.ATTR_ILLUST_ID)))));
} catch (IOException e) {
log.error("获取下载链接时出错!", e);
}
log.info(builder.append("\n").toString());
});
}
private static void startPixivAccessProxyServer(){
log.info("正在启动访问代理服务器, 将浏览器相关缓存清空后, 使用浏览器进行访问以尝试Cookie正确性.");
final PixivAccessProxyServer accessProxyServer = new PixivAccessProxyServer(cookieStore, new ProxyConfig(ProxyType.SOCKS5, "127.0.0.1", 1080));
Thread accessProxyServerThread = new Thread(() -> {
log.info("正在启动PAPS...");
accessProxyServer.start(1007);
log.info("PAPS已关闭.");
});
accessProxyServerThread.setName("AccessProxyServerThread");
accessProxyServerThread.start();
new Scanner(System.in).nextLine();
log.info("关闭PAPS服务器...");
accessProxyServer.close();
}
}

View File

@ -1,57 +0,0 @@
package net.lamgc.cgj.bot;
import net.lz1998.cq.robot.CoolQ;
import org.apache.http.client.methods.HttpGet;
import java.util.Random;
import java.util.Timer;
import java.util.TimerTask;
public class AutoArtworksSender {
private final CoolQ CQ;
private final ReceiveType receiveType;
private final long targetReceiveId;
private Timer timer = new Timer();
private TimerTask task = new TimerTask() {
@Override
public void run() {
HttpGet request = new HttpGet();
// https://api.imjad.cn/pixiv/v2/?type=tags
}
};
public AutoArtworksSender(CoolQ cq, ReceiveType receiveType, long receiveId) {
this.CQ = cq;
this.receiveType = receiveType;
this.targetReceiveId = receiveId;
}
public void reset(long time) {
if(time <= 0) {
timer.schedule(task, new Random().nextInt(10 * 60 * 60 * 1000) + 7200000L); //2H ~ 12H
} else {
timer.schedule(task, time);
}
}
public void sendMessage(String message, boolean auto_escape) {
switch (receiveType) {
case GROUP:
CQ.sendGroupMsg(targetReceiveId, message, auto_escape);
break;
case Discuss:
CQ.sendDiscussMsg(targetReceiveId, message, auto_escape);
break;
case PRIVATE:
CQ.sendPrivateMsg(targetReceiveId, message, auto_escape);
break;
}
}
public enum ReceiveType {
PRIVATE, GROUP, Discuss
}
}

View File

@ -0,0 +1,32 @@
package net.lamgc.cgj.bot;
import net.lamgc.cgj.bot.message.MessageSender;
import java.util.Objects;
/**
* 自动发送器
*/
public abstract class AutoSender {
private final MessageSender messageSender;
/**
* 构造一个自动发送器
* @param messageSender 自动发送器所使用的消息发送器
*/
public AutoSender(MessageSender messageSender) {
this.messageSender = Objects.requireNonNull(messageSender);
}
/**
* 获取设置等等消息发送器
* @return 消息发送器
*/
MessageSender getMessageSender() {
return this.messageSender;
}
public abstract void send();
}

View File

@ -1,85 +1,88 @@
package net.lamgc.cgj.bot;
import com.google.common.base.Strings;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonObject;
import com.google.gson.reflect.TypeToken;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.utils.base.runner.Argument;
import net.lamgc.utils.base.runner.Command;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.*;
import java.util.Date;
import java.util.Properties;
import java.text.SimpleDateFormat;
import java.util.*;
public class BotAdminCommandProcess {
private final static Logger log = LoggerFactory.getLogger(BotAdminCommandProcess.class.getSimpleName());
private final static Logger log = LoggerFactory.getLogger(BotAdminCommandProcess.class);
private final static File globalPropFile = new File("./global.properties");
private final static File pushListFile = new File(BotGlobal.getGlobal().getDataStoreDir(), "pushList.json");
private final static Hashtable<Long, JsonObject> pushInfoMap = new Hashtable<>();
private final static Gson gson = new GsonBuilder()
.setPrettyPrinting()
.create();
@Command
public String clearCache() {
public static String cleanCache() {
BotCommandProcess.clearCache();
return "操作已完成.";
}
@Command
public String setGlobalProperty(@Argument(name = "key") String key, @Argument(name = "value") String value, @Argument(name = "save", force = false) boolean saveNow) {
String lastValue = BotCommandProcess.globalProp.getProperty(key);
BotCommandProcess.globalProp.setProperty(key, Strings.nullToEmpty(value));
if(saveNow) {
saveGlobalProperties();
public static String setProperty(
@Argument(name = "group", force = false) long groupId,
@Argument(name = "key") String key,
@Argument(name = "value") String value
) {
if(Strings.isNullOrEmpty(key)) {
return "未选择配置项key.";
}
return "全局配置项 " + key + " 现已设置为: " + value + " (设置前的值: " + lastValue + ")";
String lastValue = SettingProperties.setProperty(groupId, key, value.equals("null") ? null : value);
return (groupId <= 0 ? "已更改全局配置 " : "已更改群组 " + groupId + " 配置 ") +
key + " 的值: '" + value + "' (原配置值: '" + lastValue + "')";
}
@Command
public String getGlobalProperty(@Argument(name = "key") String key) {
return "全局配置项 " + key + " 当前值: " + BotCommandProcess.globalProp.getProperty(key, "(Empty)");
public static String getProperty(
@Argument(name = "group", force = false) long groupId,
@Argument(name = "key") String key
) {
if(Strings.isNullOrEmpty(key)) {
return "未选择配置项key.";
}
return (groupId <= 0 ? "全局配置 " : "群组 " + groupId + " 配置 ") +
key + " 设定值: '" + SettingProperties.getProperty(groupId, key, "(empty)") + "'";
}
@Command
public String saveGlobalProperties() {
log.info("正在保存全局配置文件...");
try {
if(!globalPropFile.exists()) {
if(!globalPropFile.createNewFile()) {
log.error("全局配置项文件保存失败!({})", "文件创建失败");
return "全局配置项文件保存失败!";
}
}
BotCommandProcess.globalProp.store(new FileOutputStream(globalPropFile), "");
log.info("全局配置文件保存成功!");
return "保存全局配置文件 - 操作已完成.";
} catch (IOException e) {
log.error("全局配置项文件保存失败!", e);
return "全局配置项文件保存失败!";
}
public static String saveProperties() {
log.info("正在保存配置文件...");
SettingProperties.saveProperties();
log.info("配置文件保存操作已完成.");
return "保存配置 - 操作已完成.";
}
@Command
public String loadGlobalProperties(@Argument(name = "reload", force = false) boolean reload) {
Properties cache = new Properties();
if(!globalPropFile.exists()) {
return "未找到全局配置文件, 无法重载";
}
try(Reader reader = new BufferedReader(new FileReader(globalPropFile))) {
cache.load(reader);
} catch (IOException e) {
log.error("重载全局配置文件时发生异常", e);
return "加载全局配置文件时发生错误!";
}
public static String loadProperties(@Argument(name = "reload", force = false) boolean reload) {
if(reload) {
BotCommandProcess.globalProp.clear();
SettingProperties.clearProperties();
}
BotCommandProcess.globalProp.putAll(cache);
return "全局配置文件重载完成.";
SettingProperties.loadProperties();
return "操作已完成.";
}
@Command
public String runUpdateTask(@Argument(force = false, name = "date") Date queryTime) {
public static String runUpdateTask(@Argument(force = false, name = "date") Date queryTime) {
try {
BotCommandProcess.runUpdateTimer(queryTime);
} catch (Exception e) {
@ -89,4 +92,253 @@ public class BotAdminCommandProcess {
return "操作已完成.";
}
private final static String RANKING_SETTING_TIME_MIN = "time.min";
private final static String RANKING_SETTING_TIME_FLOAT = "time.float";
private final static String RANKING_SETTING_RANKING_START = "ranking.start";
private final static String RANKING_SETTING_RANKING_END = "ranking.end";
private final static String RANKING_SETTING_RANKING_MODE = "ranking.mode";
private final static String RANKING_SETTING_RANKING_CONTENT_TYPE = "ranking.contentType";
private final static String RANKING_SETTING_PAGE_QUALITY = "page.quality";
@Command
public static String addPushGroup(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "group", force = false, defaultValue = "0") long groupId,
@Argument(name = "minTime", force = false, defaultValue = "21600000") long minTime,
@Argument(name = "floatTime", force = false, defaultValue = "10800000") int floatTime,
@Argument(name = "rankingStart", force = false, defaultValue = "1") int rankingStart,
@Argument(name = "rankingStop", force = false, defaultValue = "150") int rankingStop,
@Argument(name = "mode", force = false, defaultValue = "DAILY") String rankingMode,
@Argument(name = "type", force = false, defaultValue = "ILLUST") String rankingContentType,
@Argument(name = "original", force = false, defaultValue = "false") boolean original
) {
if(minTime <= 0 || floatTime <= 0) {
return "时间不能为0或负数";
} else if(rankingStart <= 0 || rankingStop - rankingStart <= 0) {
return "排行榜范围选取错误!";
}
PixivURL.RankingContentType type;
PixivURL.RankingMode mode;
try {
type = PixivURL.RankingContentType.valueOf("TYPE_" + rankingContentType.toUpperCase());
} catch(IllegalArgumentException e) {
return "无效的排行榜类型参数!";
}
try {
mode = PixivURL.RankingMode.valueOf("MODE_" + rankingMode.toUpperCase());
} catch(IllegalArgumentException e) {
return "无效的排行榜模式参数!";
}
if(!type.isSupportedMode(mode)) {
return "不兼容的排行榜模式与类型!";
}
long group = groupId <= 0 ? fromGroup : groupId;
JsonObject setting = new JsonObject();
setting.addProperty(RANKING_SETTING_TIME_MIN, minTime);
setting.addProperty(RANKING_SETTING_TIME_FLOAT, floatTime);
setting.addProperty(RANKING_SETTING_RANKING_START, rankingStart);
setting.addProperty(RANKING_SETTING_RANKING_END, rankingStop);
setting.addProperty(RANKING_SETTING_RANKING_MODE, rankingMode);
setting.addProperty(RANKING_SETTING_RANKING_CONTENT_TYPE, rankingContentType);
setting.addProperty(RANKING_SETTING_PAGE_QUALITY, original ?
PixivDownload.PageQuality.ORIGINAL.name() :
PixivDownload.PageQuality.REGULAR.name());
if(pushInfoMap.containsKey(group)) {
log.info("群 {} 已存在Timer, 删除Timer...", group);
removePushGroup(fromGroup, groupId);
}
log.info("群组 {} 新推送配置: {}", group, setting);
log.info("正在增加Timer...");
pushInfoMap.put(group, setting);
addPushTimer(group, setting);
return "已在 " + group + " 开启定时推送功能。";
}
/**
* 重载推送列表
*/
@Command
public static String loadPushList() {
pushInfoMap.clear();
if(!pushListFile.exists()) {
log.warn("推送列表文件不存在, 跳过加载.");
return "文件不存在, 跳过加载.";
}
try (Reader reader = new BufferedReader(new FileReader(pushListFile))) {
pushInfoMap.putAll(gson.fromJson(reader, new TypeToken<Map<Long, JsonObject>>(){}.getType()));
loadAllPushTimer(false);
return "列表重载完成";
} catch (IOException e) {
log.error("重载推送列表时发生错误", e);
return "加载时发生异常";
}
}
@Command
public static String savePushList() {
try {
if(!pushListFile.exists() && !pushListFile.createNewFile()) {
throw new IOException("文件夹创建失败!(Path: " + pushListFile.getPath() + ")");
}
} catch (IOException e) {
log.error("PushList.json文件创建失败", e);
return "保存失败!请检查控制台信息.";
}
try (Writer writer = new FileWriter(pushListFile)) {
writer.write(gson.toJson(pushInfoMap));
return "保存成功.";
} catch (IOException e) {
log.error("写入PushList.json文件失败!", e);
return "保存失败!请检查控制台信息.";
}
}
/**
* 加载所有推送Timer
* @param flush 是否完全重载, 如为true则加载前会删除所有已加载的Timer
*/
public static void loadAllPushTimer(boolean flush) {
if(flush) {
RandomIntervalSendTimer.timerIdSet().forEach(id -> RandomIntervalSendTimer.getTimerById(id).destroy());
} else {
cleanPushTimer();
}
pushInfoMap.forEach(BotAdminCommandProcess::addPushTimer);
}
/**
* 根据设置增加Timer
* @param id 群组id
* @param setting jsonObject设置集
*/
private static void addPushTimer(long id, JsonObject setting) {
try {
RandomIntervalSendTimer.getTimerById(id);
return;
} catch(NoSuchElementException ignored) {
}
int rankingStart = setting.has(RANKING_SETTING_RANKING_START) ? setting.get(RANKING_SETTING_RANKING_START).getAsInt() : 1;
int rankingEnd = setting.has(RANKING_SETTING_RANKING_END) ? setting.get(RANKING_SETTING_RANKING_END).getAsInt() : 150;
PixivURL.RankingMode rankingMode = PixivURL.RankingMode.MODE_DAILY;
PixivURL.RankingContentType rankingContentType = PixivURL.RankingContentType.TYPE_ILLUST;
PixivDownload.PageQuality pageQuality = PixivDownload.PageQuality.REGULAR;
if(rankingStart <= 0 || rankingStart > 500) {
log.warn("群组 [{}] - 无效的RankingStart设定值, 将重置为默认设定值(1): {}", id, rankingStart);
rankingStart = 1;
} else if(rankingEnd > 500 || rankingEnd <= 0) {
log.warn("群组 [{}] - 无效的RankingEnd设定值, 将重置为默认设定值(150): {}", id, rankingEnd);
rankingEnd = 150;
} else if(rankingStart > rankingEnd) {
log.warn("群组 [{}] - 无效的排行榜选取范围, 将重置为默认设定值(1 ~ 150): start={}, end={}", id, rankingStart, rankingEnd);
rankingStart = 1;
rankingEnd = 150;
}
if(setting.has(RANKING_SETTING_RANKING_MODE)) {
String value = setting.get(RANKING_SETTING_RANKING_MODE).getAsString().trim().toUpperCase();
try {
rankingMode = PixivURL.RankingMode.valueOf(value.startsWith("MODE_") ? value : "MODE_" + value);
} catch(IllegalArgumentException e) {
log.warn("群组ID [{}] - 无效的RankingMode设定值, 将重置为默认值: {}", id, value);
}
}
if(setting.has(RANKING_SETTING_RANKING_CONTENT_TYPE)) {
String value = setting.get(RANKING_SETTING_RANKING_CONTENT_TYPE).getAsString().trim().toUpperCase();
try {
rankingContentType = PixivURL.RankingContentType.valueOf(value.startsWith("TYPE_") ? value : "TYPE_" + value);
} catch(IllegalArgumentException e) {
log.warn("群组ID [{}] - 无效的RankingContentType设定值: {}", id, value);
}
}
if(setting.has(RANKING_SETTING_PAGE_QUALITY)) {
String value = setting.get(RANKING_SETTING_PAGE_QUALITY).getAsString().trim().toUpperCase();
try {
pageQuality = PixivDownload.PageQuality.valueOf(value);
} catch(IllegalArgumentException e) {
log.warn("群组ID [{}] - 无效的PageQuality设定值: {}", id, value);
}
}
AutoSender sender = new RandomRankingArtworksSender(
MessageSenderBuilder.getMessageSender(MessageSource.GROUP, id),
id,
rankingStart,
rankingEnd,
rankingMode, rankingContentType,
pageQuality
);
RandomIntervalSendTimer timer = RandomIntervalSendTimer.createTimer(
id,
sender,
setting.get("time.min").getAsLong(),
setting.get("time.float").getAsInt(),
true, true);
log.info("群组 {} 已创建对应Timer: {}", id, Integer.toHexString(timer.hashCode()));
}
/**
* 删除一个推送定时器
* @param id 群号
* @throws NoSuchElementException 当这个群号没有定时器的时候抛出异常
*/
@Command
public static String removePushGroup(@Argument(name = "$fromGroup") long fromGroup, @Argument(name = "group", force = false) long id) {
long group = id <= 0 ? fromGroup : id;
RandomIntervalSendTimer.getTimerById(group).destroy();
pushInfoMap.remove(group);
return "已关闭群 " + group + " 的美图推送功能。";
}
/**
* 根据已修改的pushInfoMap将已经被删除的Timer取消
*/
private static void cleanPushTimer() {
RandomIntervalSendTimer.timerIdSet().forEach(id -> {
if(!pushInfoMap.containsKey(id)) {
RandomIntervalSendTimer.getTimerById(id).destroy();
}
});
}
@Command
public static String getReportList() {
SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd_HH:mm:ss");
Set<String> keys = BotCommandProcess.reportStore.keys();
StringBuilder msgBuilder = new StringBuilder();
msgBuilder.append("当前被报告的作品列表:\n");
int count = 1;
for(String key : keys) {
String illustIdStr = key.substring(key.indexOf(".") + 1);
JsonObject report = BotCommandProcess.reportStore.getCache(illustIdStr).getAsJsonObject();
log.debug("{} - Report: {}", illustIdStr, report);
String reason = report.get("reason").isJsonNull() ? "" : report.get("reason").getAsString();
msgBuilder.append(count).append(". 作品Id: ").append(illustIdStr)
.append("(").append(dateFormat.format(new Date(report.get("reportTime").getAsLong()))).append(")\n")
.append("报告者QQ").append(report.get("fromQQ").getAsLong()).append("\n")
.append("报告所在群:").append(report.get("fromGroup").getAsLong()).append("\n")
.append("报告原因:\n").append(reason).append("\n");
}
return msgBuilder.toString();
}
@Command
public static String unBanArtwork(@Argument(name = "id") int illustId) {
if(illustId <= 0) {
return "无效的作品id!";
}
boolean removeResult = BotCommandProcess.reportStore.remove(String.valueOf(illustId));
return removeResult ? "作品已解封!" : "解封失败!可能该作品并未被封禁。";
}
}

View File

@ -54,9 +54,17 @@ public class BotCode {
return new BotCode(keys[0], keys[1], param);
}
/**
* 获取BotCode所使用的匹配正则表达式
* @return 用于匹配BotCode的正则表达式对象
*/
public static Pattern getCodePattern() {
return Pattern.compile(codePattern.pattern());
}
private String platformName;
private String functionName;
private Hashtable<String, String> parameter = new Hashtable<>();
private final Hashtable<String, String> parameter = new Hashtable<>();
/**
* 构造一个机器功能码

View File

@ -1,109 +1,49 @@
package net.lamgc.cgj.bot;
import com.google.common.base.Strings;
import com.google.common.base.Throwables;
import com.google.common.reflect.TypeToken;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import com.google.gson.*;
import io.netty.handler.codec.http.HttpHeaderNames;
import net.lamgc.cgj.Main;
import net.lamgc.cgj.bot.cache.*;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.sort.PreLoadDataComparator;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.cache.CacheStore;
import net.lamgc.cgj.bot.cache.CacheStoreCentral;
import net.lamgc.cgj.bot.cache.JsonRedisCacheStore;
import net.lamgc.cgj.bot.event.BufferedMessageSender;
import net.lamgc.cgj.bot.sort.PreLoadDataAttribute;
import net.lamgc.cgj.bot.sort.PreLoadDataAttributeComparator;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivSearchBuilder;
import net.lamgc.cgj.pixiv.PixivDownload.PageQuality;
import net.lamgc.cgj.pixiv.PixivSearchLinkBuilder;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.cgj.util.URLs;
import net.lamgc.utils.base.runner.Argument;
import net.lamgc.utils.base.runner.Command;
import net.lamgc.utils.event.EventExecutor;
import net.lz1998.cq.utils.CQCode;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpHead;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.regex.Pattern;
@SuppressWarnings({"SameParameterValue"})
public class BotCommandProcess {
private final static PixivDownload pixivDownload = new PixivDownload(Main.cookieStore, Main.proxy);
private final static Logger log = LoggerFactory.getLogger(BotCommandProcess.class.getSimpleName());
private final static File imageStoreDir = new File(System.getProperty("cgj.cqRootDir"), "data/image/cgj/");
public final static Properties globalProp = new Properties();
private final static Gson gson = new GsonBuilder()
.serializeNulls()
.create();
private final static Hashtable<String, File> imageCache = new Hashtable<>();
private final static CacheStore<JsonElement> illustInfoCache = new JsonRedisCacheStore(BotEventHandler.redisServer, "illustInfo", gson);
private final static CacheStore<JsonElement> illustPreLoadDataCache = new HotDataCacheStore<>(
new JsonRedisCacheStore(BotEventHandler.redisServer, "illustPreLoadData", gson),
new LocalHashCacheStore<>(), 3600000, 900000);
private final static CacheStore<JsonElement> searchBodyCache = new JsonRedisCacheStore(BotEventHandler.redisServer, "searchBody", gson);
private final static CacheStore<List<JsonObject>> rankingCache = new JsonObjectRedisListCacheStore(BotEventHandler.redisServer, "ranking", gson);
private final static CacheStore<List<String>> pagesCache = new RedisPoolCacheStore<List<String>>(BotEventHandler.redisServer, "imagePages") {
@Override
protected String parse(List<String> dataObj) {
return gson.toJson(dataObj);
}
@Override
protected List<String> analysis(String dataStr) {
return gson.fromJson(dataStr, new TypeToken<List<String>>(){}.getType());
}
};
private final static Logger log = LoggerFactory.getLogger(BotCommandProcess.class);
/**
* 图片异步缓存执行器
* 作品报告存储 - 不过期
*/
private final static EventExecutor imageCacheExecutor = new EventExecutor(new ThreadPoolExecutor(
Runtime.getRuntime().availableProcessors() >= 2 ? 2 : 1,
(int) Math.ceil(Runtime.getRuntime().availableProcessors() / 2F),
5L, TimeUnit.SECONDS,
new LinkedBlockingQueue<>(128),
new ThreadFactoryBuilder()
.setNameFormat("imageCacheThread-%d")
.build(),
new ThreadPoolExecutor.DiscardOldestPolicy()
));
public final static CacheStore<JsonElement> reportStore =
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(),
"report", BotGlobal.getGlobal().getGson());
private final static RankingUpdateTimer updateTimer = new RankingUpdateTimer();
public static void initialize() {
log.info("正在初始化...");
File globalPropFile = new File("./global.properties");
if(globalPropFile.exists() && globalPropFile.isFile()) {
log.info("正在加载全局配置文件...");
try {
globalProp.load(new FileInputStream(globalPropFile));
log.info("全局配置文件加载完成.");
} catch (IOException e) {
log.error("加载全局配置文件时发生异常", e);
}
} else {
log.info("未找到全局配置文件,跳过加载.");
}
try {
imageCacheExecutor.addHandler(new ImageCacheHandler());
} catch (IllegalAccessException e) {
e.printStackTrace();
}
SettingProperties.loadProperties();
updateTimer.schedule(null);
log.info("初始化完成.");
@ -121,24 +61,90 @@ public class BotCommandProcess {
@Command(defaultCommand = true)
public static String help() {
StringBuilder helpStrBuilder = new StringBuilder();
helpStrBuilder.append("CGJ Bot使用指南").append("\n");
helpStrBuilder.append("目前可用的命令:").append("\n");
helpStrBuilder.append("\t").append("ranking - 获取今天或指定日期排行榜的前10名作品").append("\n");
helpStrBuilder.append("\t\t").append("-date - 指定查询日期(年-月-日)").append("\n");
helpStrBuilder.append("\t").append("search - 搜索指定关键词并显示前10作品").append("\n");
helpStrBuilder.append("\t\t").append("-content - 搜索内容").append("\n");
helpStrBuilder.append("\t").append("artworks - 获取作品的Pixiv页面").append("\n");
helpStrBuilder.append("\t\t").append("-id - 作品id").append("\n");
return helpStrBuilder.toString();
return "CGJ Bot使用指南" + "\n" +
"使用方法:.cgj <命令> [参数...]" + "\n" +
"例如查询作品信息功能:" + "\n" +
".cgj info -id 80846159" + "\n" +
"目前可用的命令:" + "\n" +
"\t" + "ranking - 获取今天或指定日期排行榜的前10作品" + "\n" +
"\t\t" + "-date - 指定查询日期(年-月-日)" + "\n" +
"\t\t" + "-type - 排行榜类型(illust/插画, ugoira/动图, manga/漫画)" + "\n" +
"\t" + "search - 搜索指定关键词并显示前10个作品" + "\n" +
"\t\t" + "-content - 搜索内容" + "\n" +
"\t" + "link - 获取作品的Pixiv页面" + "\n" +
"\t\t" + "-id - 作品id" + "\n" +
"\t" + "info - 获取Pixiv作品信息" + "\n" +
"\t\t" + "-id - 作品id" + "\n" +
"\t" + "image - 获取指定作品的图片" + "\n" +
"\t\t" + "-id - 作品id" + "\n" +
"\t\t" + "-quality - 图片质量(original/原图 regular/预览图)" + "\n" +
"\t\t" + "-page - 页数" + "\n" +
"\t" + "report - 报告不当作品" + "\n" +
"\t\t" + "-id - 作品Id" + "\n" +
"\t\t" + "-msg - 报告原因" + "\n";
}
/**
* 作品信息查询
* @param fromGroup 来源群(系统提供)
* @param illustId 作品Id
* @return 返回作品信息
*/
@Command(commandName = "info")
public static String artworkInfo(@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "id") int illustId)
throws InterruptedException {
if(illustId <= 0) {
return "这个作品Id是错误的";
}
try {
if(isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false) || isReported(illustId)) {
return "阅览禁止:该作品已被封印!!";
}
JsonObject illustPreLoadData = CacheStoreCentral.getCentral().getIllustPreLoadData(illustId, false);
// 在 Java 6 开始, 编译器会将用'+'进行的字符串拼接将自动转换成StringBuilder拼接
return "色图姬帮你了解了这个作品的信息!\n" + "---------------- 作品信息 ----------------" +
"\n作品Id: " + illustId +
"\n作品标题" + illustPreLoadData.get("illustTitle").getAsString() +
"\n作者(作者Id)" + illustPreLoadData.get("userName").getAsString() +
"(" + illustPreLoadData.get("userId").getAsInt() + ")" +
"\n点赞数" + illustPreLoadData.get(PreLoadDataAttribute.LIKE.attrName).getAsInt() +
"\n收藏数" + illustPreLoadData.get(PreLoadDataAttribute.BOOKMARK.attrName).getAsInt() +
"\n围观数" + illustPreLoadData.get(PreLoadDataAttribute.VIEW.attrName).getAsInt() +
"\n评论数" + illustPreLoadData.get(PreLoadDataAttribute.COMMENT.attrName).getAsInt() +
"\n页数" + illustPreLoadData.get(PreLoadDataAttribute.PAGE.attrName).getAsInt() + "" +
"\n作品链接" + artworksLink(fromGroup, illustId) + "\n" +
"---------------- 作品图片 ----------------\n" +
CacheStoreCentral.getCentral().getImageById(fromGroup, illustId, PageQuality.REGULAR, 1) + "\n" +
"使用 \".cgj image -id " +
illustId +
"\" 获取原图。\n如有不当作品可使用\".cgj report -id " +
illustId + "\"向色图姬反馈。";
} catch (IOException e) {
e.printStackTrace();
}
return "尚未支持";
}
/**
* 排行榜命令
* @param fromGroup 来源群(系统提供)
* @param queryTime 查询时间, 格式: 年-月-日
* @param force 是否强制查询, 当主动提供的时间不在查询范围时, 是否强制查询, 仅系统可用
* @param contentMode 内容模式
* @param contentType 排行榜类型
* @return 返回排行榜信息
*/
@Command
public static String ranking(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(force = false, name = "date") Date queryTime,
@Argument(force = false, name = "force") boolean force,
@Argument(force = false, name = "mode", defaultValue = "DAILY") String contentMode,
@Argument(force = false, name = "type", defaultValue = "ILLUST") String contentType
) {
@Argument(force = false, name = "type", defaultValue = "ALL") String contentType
) throws InterruptedException {
Date queryDate = queryTime;
if (queryDate == null) {
queryDate = new Date();
@ -152,24 +158,30 @@ public class BotCommandProcess {
}
queryDate = gregorianCalendar.getTime();
} else {
if(new Date().before(queryDate)) {
if(new Date().before(queryDate) && !force) {
log.warn("查询的日期过早, 无法查询排行榜.");
return "查询日期过早, 暂未更新指定日期的排行榜!";
}
}
PixivURL.RankingMode mode = PixivURL.RankingMode.MODE_DAILY;
PixivURL.RankingMode mode;
try {
mode = PixivURL.RankingMode.valueOf("MODE_" + contentMode.toUpperCase());
String rankingModeValue = contentMode.toUpperCase();
mode = PixivURL.RankingMode.valueOf(rankingModeValue.startsWith("MODE_") ?
rankingModeValue : "MODE_" + rankingModeValue);
} catch (IllegalArgumentException e) {
log.warn("无效的RankingMode值: {}", contentMode);
return "参数无效, 请查看帮助信息";
}
PixivURL.RankingContentType type = PixivURL.RankingContentType.TYPE_ILLUST;
PixivURL.RankingContentType type;
try {
type = PixivURL.RankingContentType.valueOf("TYPE_" + contentType.toUpperCase());
String contentTypeValue = contentType.toUpperCase();
type = PixivURL.RankingContentType.valueOf(
contentTypeValue.startsWith("TYPE_") ? contentTypeValue : "TYPE_" + contentTypeValue);
} catch (IllegalArgumentException e) {
log.warn("无效的RankingContentType值: {}", contentType);
return "参数无效, 请查看帮助信息";
}
if(!type.isSupportedMode(mode)) {
@ -178,13 +190,15 @@ public class BotCommandProcess {
return "不支持的内容类型或模式!";
}
StringBuilder resultBuilder = new StringBuilder(mode.name() + " - 以下是 ").append(new SimpleDateFormat("yyyy-MM-dd").format(queryDate)).append(" 的Pixiv插画排名榜前十名\n");
StringBuilder resultBuilder = new StringBuilder(mode.name() + " - 以下是 ")
.append(new SimpleDateFormat("yyyy-MM-dd").format(queryDate)).append(" 的Pixiv插画排名榜前十名\n");
try {
int index = 0;
int itemLimit = 10;
String itemLimitPropertyKey = "ranking.ItemCountLimit";
String itemLimitPropertyKey = "ranking.itemCountLimit";
try {
itemLimit = Integer.parseInt(globalProp.getProperty(itemLimitPropertyKey, "10"));
itemLimit = Integer.parseInt(SettingProperties
.getProperty(fromGroup, itemLimitPropertyKey, "10"));
} catch(NumberFormatException e) {
log.warn("配置项 {} 的参数值格式有误!", itemLimitPropertyKey);
}
@ -192,13 +206,14 @@ public class BotCommandProcess {
int imageLimit = 3;
String imageLimitPropertyKey = "ranking.imageCountLimit";
try {
imageLimit = Integer.parseInt(globalProp.getProperty(imageLimitPropertyKey, "3"));
imageLimit = Integer.parseInt(
SettingProperties.getProperty(fromGroup, imageLimitPropertyKey, "3"));
} catch(NumberFormatException e) {
log.warn("配置项 {} 的参数值格式有误!", imageLimitPropertyKey);
}
//TODO(LamGC, 2020.4.11): 将JsonRedisCacheStore更改为使用Redis的List集合, 以提高性能
List<JsonObject> rankingInfoList = getRankingInfoByCache(type, mode, queryDate, 1, Math.max(0, itemLimit), false);
List<JsonObject> rankingInfoList = CacheStoreCentral.getCentral()
.getRankingInfoByCache(type, mode, queryDate, 1, Math.max(0, itemLimit), false);
if(rankingInfoList.isEmpty()) {
return "无法查询排行榜,可能排行榜尚未更新。";
}
@ -212,127 +227,110 @@ public class BotCommandProcess {
String authorName = rankInfo.get("user_name").getAsString();
String title = rankInfo.get("title").getAsString();
resultBuilder.append(rank).append(". (id: ").append(illustId).append(") ").append(title)
.append("(Author: ").append(authorName).append(",").append(authorId).append(") ").append(pagesCount).append("p.\n");
.append("(Author: ").append(authorName).append(",").append(authorId).append(") ")
.append(pagesCount).append("p.\n");
if (index <= imageLimit) {
resultBuilder.append(getImageById(illustId, PixivDownload.PageQuality.REGULAR, 1)).append("\n");
resultBuilder
.append(CacheStoreCentral.getCentral()
.getImageById(fromGroup, illustId, PixivDownload.PageQuality.REGULAR, 1))
.append("\n");
}
}
} catch (IOException e) {
log.error("消息处理异常", e);
return "排名榜获取失败!详情请查看机器人控制台。";
}
return resultBuilder.append("如查询当前时间获取到昨天时间,则今日排名榜未更新。").toString();
return resultBuilder.append("如查询当前时间获取到昨天时间,则今日排名榜未更新。\n" +
"如有不当作品,可使用\".cgj report -id 作品id\"向色图姬反馈。").toString();
}
/**
* 查询指定作者的作品(尚未完成)
* @return 返回作者信息和部分作品
*/
@Command(commandName = "userArt")
public static String userArtworks() {
return "功能未完成";
}
private final static Object searchCacheLock = new Object();
/**
* 随机获取一副作品
*/
@Command(commandName = "random")
public static String randomImage(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(force = false, name = "mode", defaultValue = "DAILY") String contentMode,
@Argument(force = false, name = "type", defaultValue = "ILLUST") String contentType) {
PixivURL.RankingMode mode;
try {
String rankingModeValue = contentMode.toUpperCase();
mode = PixivURL.RankingMode.valueOf(rankingModeValue.startsWith("MODE_") ?
rankingModeValue : "MODE_" + rankingModeValue);
} catch (IllegalArgumentException e) {
log.warn("无效的RankingMode值: {}", contentMode);
return "参数无效, 请查看帮助信息";
}
PixivURL.RankingContentType type;
try {
String contentTypeValue = contentType.toUpperCase();
type = PixivURL.RankingContentType.valueOf(
contentTypeValue.startsWith("TYPE_") ? contentTypeValue : "TYPE_" + contentTypeValue);
} catch (IllegalArgumentException e) {
log.warn("无效的RankingContentType值: {}", contentType);
return "参数无效, 请查看帮助信息";
}
BufferedMessageSender bufferedSender = new BufferedMessageSender();
RandomRankingArtworksSender artworksSender =
new RandomRankingArtworksSender(bufferedSender, fromGroup, 1, 200, mode, type,
PageQuality.ORIGINAL);
artworksSender.send();
return bufferedSender.getBufferContent();
}
/**
* 搜索命令
* @param fromGroup 来源群(系统提供)
* @param content 搜索内容
* @param type 搜索类型
* @param area 搜索区域
* @param includeKeywords 包括关键字
* @param excludeKeywords 排除关键字
* @param contentOption 搜索选项
* @param pagesIndex 搜索页索引
* @return 返回搜索内容消息
* @throws IOException 当搜索发生异常时抛出
*/
@Command
public static String search(@Argument(name = "content") String content,
@Argument(name = "type", force = false) String type,
@Argument(name = "area", force = false) String area,
@Argument(name = "in", force = false) String includeKeywords,
@Argument(name = "ex", force = false) String excludeKeywords,
@Argument(name = "contentOption", force = false) String contentOption,
@Argument(name = "page", force = false, defaultValue = "1") int pagesIndex
) throws IOException {
log.info("正在执行搜索...");
PixivSearchBuilder searchBuilder = new PixivSearchBuilder(Strings.isNullOrEmpty(content) ? "" : content);
if (type != null) {
try {
searchBuilder.setSearchType(PixivSearchBuilder.SearchType.valueOf(type.toUpperCase()));
} catch (IllegalArgumentException e) {
log.warn("不支持的SearchType: {}", type);
}
}
if (area != null) {
try {
searchBuilder.setSearchArea(PixivSearchBuilder.SearchArea.valueOf(area));
} catch (IllegalArgumentException e) {
log.warn("不支持的SearchArea: {}", area);
}
}
if (contentOption != null) {
try {
searchBuilder.setSearchContentOption(PixivSearchBuilder.SearchContentOption.valueOf(contentOption));
} catch (IllegalArgumentException e) {
log.warn("不支持的SearchContentOption: {}", contentOption);
}
}
if (!Strings.isNullOrEmpty(includeKeywords)) {
for (String keyword : includeKeywords.split(";")) {
searchBuilder.removeExcludeKeyword(keyword);
searchBuilder.addIncludeKeyword(keyword);
log.debug("已添加关键字: {}", keyword);
}
}
if (!Strings.isNullOrEmpty(excludeKeywords)) {
for (String keyword : excludeKeywords.split(";")) {
searchBuilder.removeIncludeKeyword(keyword);
searchBuilder.addExcludeKeyword(keyword);
log.debug("已添加排除关键字: {}", keyword);
}
}
log.info("正在搜索作品, 条件: {}", searchBuilder.getSearchCondition());
String requestUrl = searchBuilder.buildURL();
log.debug("RequestUrl: {}", requestUrl);
JsonObject resultBody = null;
if(!searchBodyCache.exists(requestUrl)) {
synchronized (searchCacheLock) {
if (!searchBodyCache.exists(requestUrl)) {
log.debug("searchBody缓存失效, 正在更新...");
JsonObject jsonObject;
HttpGet httpGetRequest = pixivDownload.createHttpGetRequest(requestUrl);
HttpResponse response = pixivDownload.getHttpClient().execute(httpGetRequest);
String responseBody = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
log.debug("ResponseBody: {}", responseBody);
jsonObject = gson.fromJson(responseBody, JsonObject.class);
if (jsonObject.get("error").getAsBoolean()) {
log.error("接口请求错误, 错误信息: {}", jsonObject.get("message").getAsString());
return "处理命令时发生错误!";
}
long expire = 7200 * 1000;
String propValue = globalProp.getProperty("cache.searchBody.expire", "7200000");
try {
expire = Long.parseLong(propValue);
} catch (Exception e) {
log.warn("全局配置项 \"{}\" 值非法, 已使用默认值: {}", propValue, expire);
}
resultBody = jsonObject.getAsJsonObject().getAsJsonObject("body");
searchBodyCache.update(requestUrl, jsonObject, expire);
log.debug("searchBody缓存已更新(有效时间: {})", expire);
} else {
log.debug("搜索缓存命中.");
}
}
} else {
log.debug("搜索缓存命中.");
}
if(Objects.isNull(resultBody)) {
resultBody = searchBodyCache.getCache(requestUrl).getAsJsonObject().getAsJsonObject("body");
}
public static String search(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "content") String content,
@Argument(name = "type", force = false) String type,
@Argument(name = "area", force = false) String area,
@Argument(name = "in", force = false) String includeKeywords,
@Argument(name = "ex", force = false) String excludeKeywords,
@Argument(name = "option", force = false) String contentOption,
@Argument(name = "p", force = false, defaultValue = "1") int pagesIndex
) throws IOException, InterruptedException {
log.debug("正在执行搜索...");
JsonObject resultBody = CacheStoreCentral.getCentral()
.getSearchBody(content, type, area, includeKeywords, excludeKeywords, contentOption, pagesIndex);
StringBuilder result = new StringBuilder("内容 " + content + " 的搜索结果:\n");
log.debug("正在处理信息...");
int limit = 8;
try {
limit = Integer.parseInt(globalProp.getProperty("search.ItemCountLimit", "8"));
limit = Integer.parseInt(SettingProperties.
getProperty(fromGroup, "search.itemCountLimit", "8"));
} catch (Exception e) {
log.warn("参数转换异常!将使用默认值(" + limit + ")", e);
}
for (PixivSearchBuilder.SearchArea searchArea : PixivSearchBuilder.SearchArea.values()) {
if (!resultBody.has(searchArea.jsonKey) || resultBody.getAsJsonObject(searchArea.jsonKey).getAsJsonArray("data").size() == 0) {
int totalCount = 0;
for (PixivSearchLinkBuilder.SearchArea searchArea : PixivSearchLinkBuilder.SearchArea.values()) {
if (!resultBody.has(searchArea.jsonKey) ||
resultBody.getAsJsonObject(searchArea.jsonKey).getAsJsonArray("data").size() == 0) {
log.debug("返回数据不包含 {}", searchArea.jsonKey);
continue;
}
@ -340,9 +338,9 @@ public class BotCommandProcess {
.getAsJsonObject(searchArea.jsonKey).getAsJsonArray("data");
ArrayList<JsonElement> illustsList = new ArrayList<>();
illustsArray.forEach(illustsList::add);
illustsList.sort(new PreLoadDataComparator(PreLoadDataComparator.Attribute.LIKE));
illustsList.sort(new PreLoadDataAttributeComparator(PreLoadDataAttribute.BOOKMARK));
log.info("已找到与 {} 相关插图信息({})", content, searchArea.name().toLowerCase());
log.debug("已找到与 {} 相关插图信息({})", content, searchArea.name().toLowerCase());
int count = 1;
for (JsonElement jsonElement : illustsList) {
if (count > limit) {
@ -356,7 +354,8 @@ public class BotCommandProcess {
StringBuilder builder = new StringBuilder("[");
illustObj.get("tags").getAsJsonArray().forEach(el -> builder.append(el.getAsString()).append(", "));
builder.replace(builder.length() - 2, builder.length(), "]");
log.debug("{} ({} / {})\n\t作品id: {}, \n\t作者名(作者id): {} ({}), \n\t作品标题: {}, \n\t作品Tags: {}, \n\t页数: {}, \n\t作品链接: {}",
log.debug("{} ({} / {})\n\t作品id: {}, \n\t作者名(作者id): {} ({}), \n\t" +
"作品标题: {}, \n\t作品Tags: {}, \n\t页数: {}页, \n\t作品链接: {}",
searchArea.name(),
count,
illustsList.size(),
@ -369,33 +368,77 @@ public class BotCommandProcess {
PixivURL.getPixivRefererLink(illustId)
);
//pageCount
String imageMsg = getImageById(illustId, PixivDownload.PageQuality.REGULAR, 1);
if (isNoSafe(illustId, globalProp, true)) {
String imageMsg;
try {
imageMsg = CacheStoreCentral.getCentral()
.getImageById(fromGroup, illustId, PixivDownload.PageQuality.REGULAR, 1);
} catch (NoSuchElementException e) {
if(e.getMessage().startsWith("No work found: ")) {
log.warn("作品 {} 不存在, 跳过该作品...", illustId);
continue;
}
throw e;
}
if (isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) {
log.warn("作品Id {} 为R-18作品, 跳过.", illustId);
continue;
} else if(isReported(illustId)) {
log.warn("作品Id {} 被报告, 正在等待审核, 跳过该作品.", illustId);
continue;
}
result.append(searchArea.name()).append(" (").append(count).append(" / ").append(limit).append(")\n\t作品id: ").append(illustId)
JsonObject illustPreLoadData = CacheStoreCentral.getCentral()
.getIllustPreLoadData(illustId, false);
result.append(searchArea.name()).append(" (").append(count).append(" / ")
.append(limit).append(")\n\t作品id: ").append(illustId)
.append(", \n\t作者名: ").append(illustObj.get("userName").getAsString())
.append("\n\t作品标题: ").append(illustObj.get("illustTitle").getAsString())
.append("\n\t作品页数: ").append(illustObj.get("pageCount").getAsInt())
.append("\n\t作品页数: ").append(illustObj.get("pageCount").getAsInt()).append("")
.append("\n\t点赞数")
.append(illustPreLoadData.get(PreLoadDataAttribute.LIKE.attrName).getAsInt())
.append("\n\t收藏数")
.append(illustPreLoadData.get(PreLoadDataAttribute.BOOKMARK.attrName).getAsInt())
.append("\n\t围观数")
.append(illustPreLoadData.get(PreLoadDataAttribute.VIEW.attrName).getAsInt())
.append("\n\t评论数")
.append(illustPreLoadData.get(PreLoadDataAttribute.COMMENT.attrName).getAsInt())
.append("\n").append(imageMsg).append("\n");
count++;
totalCount++;
}
if (count > limit) {
break;
}
}
return Strings.nullToEmpty(result.toString()) + "预览图片并非原图,使用“.cgj image -id 作品id”获取原图";
return totalCount <= 0 ?
"搜索完成,未找到相关作品。" :
Strings.nullToEmpty(result.toString()) + "预览图片并非原图,使用“.cgj image -id 作品id”获取原图\n" +
"如有不当作品,可使用\".cgj report -id 作品id\"向色图姬反馈。";
}
/**
* 获取作品页面的下载链接
* @param illustId 作品Id
* @param quality 画质类型
* @return 返回作品所有页面在Pixiv的下载链接(有防盗链, 考虑要不要设置镜像站)
*/
@Command(commandName = "pages")
public static String getPagesList(@Argument(name = "id") int illustId, @Argument(name = "quality", force = false) PixivDownload.PageQuality quality) {
public static String getPagesList(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "id") int illustId,
@Argument(name = "quality", force = false) PixivDownload.PageQuality quality) {
try {
List<String> pagesList = PixivDownload.getIllustAllPageDownload(pixivDownload.getHttpClient(), pixivDownload.getCookieStore(), illustId, quality);
StringBuilder builder = new StringBuilder("作品ID ").append(illustId).append(" 共有").append(pagesList.size()).append("页:").append("\n");
if(isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) {
log.warn("来源群 {} 查询的作品Id {} 为R18作品, 根据配置设定, 屏蔽该作品.", fromGroup, illustId);
return "该作品已被封印!";
}
List<String> pagesList =
PixivDownload.getIllustAllPageDownload(
BotGlobal.getGlobal().getPixivDownload().getHttpClient(),
BotGlobal.getGlobal().getPixivDownload().getCookieStore(),
illustId, quality);
StringBuilder builder = new StringBuilder("作品ID ").append(illustId)
.append(" 共有").append(pagesList.size()).append("页:").append("\n");
int index = 0;
for (String link : pagesList) {
builder.append("Page ").append(++index).append(": ").append(link).append("\n");
@ -407,12 +450,22 @@ public class BotCommandProcess {
}
}
@Command(commandName = "artworks")
public static String artworksLink(@Argument(name = "id") int illustId) {
/**
* 获取作品链接
* @param fromGroup 来源群(系统提供)
* @param illustId 作品Id
* @return 返回作品在Pixiv的链接
*/
@Command(commandName = "link")
public static String artworksLink(@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "id") int illustId) {
try {
if (isNoSafe(illustId, globalProp, false)) {
if (isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) {
log.warn("作品Id {} 已被屏蔽.", illustId);
return "由于相关设置,该作品已被屏蔽!";
} else if(isReported(illustId)) {
log.warn("作品Id {} 被报告, 正在等待审核, 跳过该作品.", illustId);
return "该作品暂时被封印,请等待色图姬进一步审核!";
}
} catch (IOException e) {
log.error("获取作品信息失败!", e);
@ -421,103 +474,10 @@ public class BotCommandProcess {
return PixivURL.getPixivRefererLink(illustId);
}
/**
* 通过illustId获取作品图片
* @param illustId 作品Id
* @param quality 图片质量
* @param pageIndex 指定页面索引, 从1开始
* @return 如果成功, 返回BotCode, 否则返回错误信息.
*/
@Command(commandName = "image")
public static String getImageById(@Argument(name = "id") int illustId,
@Argument(name = "quality", force = false) PixivDownload.PageQuality quality,
@Argument(name = "page", force = false, defaultValue = "1") int pageIndex) {
log.debug("IllustId: {}, Quality: {}, PageIndex: {}", illustId, quality.name(), pageIndex);
List<String> pagesList;
try {
pagesList = getIllustPages(illustId, quality, false);
} catch (IOException e) {
log.error("获取下载链接列表时发生异常", e);
return "发生网络异常,无法获取图片!";
}
if(log.isDebugEnabled()) {
StringBuilder logBuilder = new StringBuilder("作品Id {} 所有页面下载链接: \n");
AtomicInteger index = new AtomicInteger();
pagesList.forEach(item -> logBuilder.append(index.incrementAndGet()).append(". ").append(item).append("\n"));
log.debug(logBuilder.toString());
}
if (pagesList.size() < pageIndex || pageIndex <= 0) {
log.warn("指定的页数超出了总页数({} / {})", pageIndex, pagesList.size());
return "指定的页数超出了范围(总共 " + pagesList.size() + " 页)";
}
try {
if (isNoSafe(illustId, globalProp, false)) {
log.warn("作品 {} 存在R-18内容且设置\"image.allowR18\"为false将屏蔽该作品不发送.", illustId);
return "(根据设置,该作品已被屏蔽!)";
}
} catch (IOException e) {
log.warn("作品信息无法获取!", e);
return "发生网络异常,无法获取图片!";
}
String downloadLink = pagesList.get(pageIndex - 1);
String fileName = URLs.getResourceName(Strings.nullToEmpty(downloadLink));
File imageFile = new File(getImageStoreDir(), downloadLink.substring(downloadLink.lastIndexOf("/") + 1));
log.debug("FileName: {}, DownloadLink: {}", fileName, downloadLink);
if(!imageCache.containsKey(fileName)) {
if(imageFile.exists()) {
HttpHead headRequest = new HttpHead(downloadLink);
headRequest.addHeader("Referer", PixivURL.getPixivRefererLink(illustId));
HttpResponse headResponse;
try {
headResponse = pixivDownload.getHttpClient().execute(headRequest);
} catch (IOException e) {
log.error("获取图片大小失败!", e);
return "图片获取失败!";
}
String contentLengthStr = headResponse.getFirstHeader(HttpHeaderNames.CONTENT_LENGTH.toString()).getValue();
log.debug("图片大小: {}B", contentLengthStr);
if (imageFile.length() == Long.parseLong(contentLengthStr)) {
imageCache.put(URLs.getResourceName(downloadLink), imageFile);
log.debug("作品Id {} 第 {} 页缓存已补充.", illustId, pageIndex);
return getImageToBotCode(imageFile, false).toString();
}
}
ImageCacheObject taskObject = new ImageCacheObject(imageCache, illustId, downloadLink, imageFile);
try {
imageCacheExecutor.executorSync(taskObject);
} catch (InterruptedException e) {
log.error("等待图片下载时发生中断", e);
return "图片获取失败!";
}
} else {
log.debug("图片 {} 缓存命中.", fileName);
}
return getImageToBotCode(imageCache.get(fileName), false).toString();
}
private static BotCode getImageToBotCode(File targetFile, boolean updateCache) {
String fileName = targetFile.getName();
BotCode code = BotCode.parse(CQCode.image(getImageStoreDir().getName() + "/" + fileName));
code.addParameter("absolutePath", targetFile.getAbsolutePath());
code.addParameter("imageName", fileName.substring(0, fileName.lastIndexOf(".")));
code.addParameter("updateCache", updateCache ? "true" : "false");
return code;
}
static void clearCache() {
log.warn("正在清除所有缓存...");
imageCache.clear();
illustInfoCache.clear();
illustPreLoadDataCache.clear();
pagesCache.clear();
searchBodyCache.clear();
File imageStoreDir = new File(System.getProperty("cgj.cqRootDir") + "data/image/cgj/");
CacheStoreCentral.getCentral().clearCache();
File imageStoreDir = new File(BotGlobal.getGlobal().getDataStoreDir(), "data/image/cgj/");
File[] listFiles = imageStoreDir.listFiles();
if (listFiles == null) {
log.debug("图片缓存目录为空或内部文件获取失败!");
@ -530,165 +490,80 @@ public class BotCommandProcess {
log.warn("缓存删除完成.");
}
/*
下一目标:
添加定时发图
定时发图支持设置关注标签
标签....标签支持搜索吧
*/
private static boolean isNoSafe(int illustId, Properties settingProp, boolean returnRaw) throws IOException {
boolean rawValue = getIllustInfo(illustId, false).getAsJsonArray("tags").contains(new JsonPrimitive("R-18"));
return returnRaw || settingProp == null ? rawValue : rawValue && !settingProp.getProperty("image.allowR18", "false").equalsIgnoreCase("true");
}
private static JsonObject getIllustInfo(int illustId, boolean flushCache) throws IOException {
String illustIdStr = buildSyncKey(Integer.toString(illustId));
JsonObject illustInfoObj = null;
if (!illustInfoCache.exists(illustIdStr) || flushCache) {
synchronized (illustIdStr) { // TODO: 这里要不做成HashMap存储key而避免使用常量池?
if (!illustInfoCache.exists(illustIdStr) || flushCache) {
illustInfoObj = pixivDownload.getIllustInfoByIllustId(illustId);
illustInfoCache.update(illustIdStr, illustInfoObj, null);
}
}
}
if(Objects.isNull(illustInfoObj)) {
illustInfoObj = illustInfoCache.getCache(illustIdStr).getAsJsonObject();
log.debug("作品Id {} IllustInfo缓存命中.", illustId);
}
return illustInfoObj;
@Command(commandName = "image")
public static String getImageById(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "id") int illustId,
@Argument(name = "quality", force = false) PixivDownload.PageQuality quality,
@Argument(name = "p", force = false, defaultValue = "1") int pageIndex
) throws InterruptedException {
return CacheStoreCentral.getCentral().getImageById(fromGroup, illustId, quality, pageIndex);
}
/**
* 获取作品预加载数据.
* 可以获取作品的一些与用户相关的信息
* 举报某一作品
* @param fromGroup 来源群(系统提供)
* @param illustId 需要举报的作品id
* @param reason 举报原因
* @return 返回提示信息
*/
@Command
public static String report(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "$fromQQ") long fromQQ,
@Argument(name = "id") int illustId,
@Argument(name = "msg", force = false) String reason
) {
log.warn("收到作品反馈(IllustId: {}, 原因: {})", illustId, reason);
JsonObject reportJson = new JsonObject();
reportJson.addProperty("illustId", illustId);
reportJson.addProperty("reportTime", new Date().getTime());
reportJson.addProperty("fromGroup", fromGroup);
reportJson.addProperty("fromQQ", fromQQ);
reportJson.addProperty("reason", reason);
reportStore.update(String.valueOf(illustId), reportJson, 0);
return "色图姬收到了你的报告,将屏蔽该作品并对作品违规情况进行核实,感谢你的反馈!";
}
/**
* 检查某一作品是否被报告
* @param illustId 作品Id
* @param flushCache 是否刷新缓存
* @return 成功返回JsonObject对象
* @throws IOException 当Http请求处理发生异常时抛出
* @return 如果被报告了, 返回true
*/
public static JsonObject getIllustPreLoadData(int illustId, boolean flushCache) throws IOException {
String illustIdStr = buildSyncKey(Integer.toString(illustId));
JsonObject result = null;
if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) {
synchronized (illustIdStr) {
if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) {
log.info("IllustId {} 缓存失效, 正在更新...", illustId);
JsonObject preLoadDataObj = pixivDownload.getIllustPreLoadDataById(illustId)
.getAsJsonObject("illust")
.getAsJsonObject(Integer.toString(illustId));
long expire = 7200 * 1000;
String propValue = globalProp.getProperty("cache.illustPreLoadData.expire", "7200000");
log.debug("PreLoadData有效时间设定: {}", propValue);
try {
expire = Long.parseLong(propValue);
} catch (Exception e) {
log.warn("全局配置项 \"{}\" 值非法, 已使用默认值: {}", propValue, expire);
}
result = preLoadDataObj;
illustPreLoadDataCache.update(illustIdStr, preLoadDataObj, expire);
log.info("作品Id {} preLoadData缓存已更新(有效时间: {})", illustId, expire);
}
}
}
if(Objects.isNull(result)) {
result = illustPreLoadDataCache.getCache(illustIdStr).getAsJsonObject();
log.debug("作品Id {} PreLoadData缓存命中.", illustId);
}
return result;
public static boolean isReported(int illustId) {
return reportStore.exists(String.valueOf(illustId));
}
public static List<String> getIllustPages(int illustId, PixivDownload.PageQuality quality, boolean flushCache) throws IOException {
String pagesSign = buildSyncKey(Integer.toString(illustId), ".", quality.name());
List<String> result = null;
if (!pagesCache.exists(pagesSign) || flushCache) {
synchronized (pagesSign) {
if (!pagesCache.exists(pagesSign) || flushCache) {
List<String> linkList = PixivDownload.getIllustAllPageDownload(pixivDownload.getHttpClient(), pixivDownload.getCookieStore(), illustId, quality);
result = linkList;
pagesCache.update(pagesSign, linkList, null);
}
}
}
if(Objects.isNull(result)) {
result = pagesCache.getCache(pagesSign);
log.debug("作品Id {} Pages缓存命中.", illustId);
}
return result;
}
private static File getImageStoreDir() {
if(!imageStoreDir.exists() && !imageStoreDir.mkdirs()) {
log.warn("酷Q图片缓存目录失效(Path: {} )", imageStoreDir.getAbsolutePath());
throw new RuntimeException(new IOException("文件夹创建失败!"));
}
return imageStoreDir;
}
private final static Random expireTimeFloatRandom = new Random();
/**
* 获取排行榜
* @param contentType 排行榜类型
* @param mode 排行榜模式
* @param queryDate 查询时间
* @param start 开始排名, 从1开始
* @param range 取范围
* @param flushCache 是否强制刷新缓存
* @return 成功返回有值List, 失败且无异常返回空
* @throws IOException 获取异常时抛出
* Tag过滤表达式
*/
public static List<JsonObject> getRankingInfoByCache(PixivURL.RankingContentType contentType, PixivURL.RankingMode mode, Date queryDate, int start, int range, boolean flushCache) throws IOException {
if(!contentType.isSupportedMode(mode)) {
log.warn("试图获取不支持的排行榜类型已拒绝.(ContentType: {}, RankingMode: {})", contentType.name(), mode.name());
if(log.isDebugEnabled()) {
try {
Thread.dumpStack();
} catch(Exception e) {
log.debug("本次非法请求的堆栈信息如下: \n{}", Throwables.getStackTraceAsString(e));
}
}
return new ArrayList<>(0);
}
String date = new SimpleDateFormat("yyyyMMdd").format(queryDate);
String requestSign = buildSyncKey(contentType.name(), ".", mode.name(), ".", date);
List<JsonObject> result = null;
if(!rankingCache.exists(requestSign) || flushCache) {
synchronized(requestSign) {
if(!rankingCache.exists(requestSign) || flushCache) {
log.info("Ranking缓存失效, 正在更新...(RequestSign: {})", requestSign);
List<JsonObject> rankingResult = pixivDownload.getRanking(contentType, mode, queryDate, 1, 500);
if(rankingResult.size() == 0) {
log.info("数据获取失败, 将设置浮动有效时间以准备下次更新.");
}
result = new ArrayList<>(rankingResult).subList(start - 1, range);
rankingCache.update(requestSign, rankingResult,
rankingResult.size() == 0 ? 5400000 + expireTimeFloatRandom.nextInt(1800000) : 0);
log.info("Ranking缓存更新完成.(RequestSign: {})", requestSign);
private final static Pattern tagPattern = Pattern.compile(".*R-*18.*");
/**
* 检查指定作品是否为r18
* @param illustId 作品Id
* @param settingProp 配置项
* @param returnRaw 是否返回原始值
* @return 如果为true, 则不为全年龄
* @throws IOException 获取数据时发生异常时抛出
* @throws NoSuchElementException 当作品不存在时抛出
*/
public static boolean isNoSafe(int illustId, Properties settingProp, boolean returnRaw)
throws IOException, NoSuchElementException {
JsonObject illustInfo = CacheStoreCentral.getCentral().getIllustInfo(illustId, false);
JsonArray tags = illustInfo.getAsJsonArray("tags");
boolean rawValue = illustInfo.get("xRestrict").getAsInt() != 0;
if(!rawValue) {
for(JsonElement tag : tags) {
boolean current = tagPattern.matcher(tag.getAsString()).matches();
if (current) {
rawValue = true;
break;
}
}
}
if (Objects.isNull(result)) {
result = rankingCache.getCache(requestSign, start - 1, range);
log.debug("RequestSign [{}] 缓存命中.", requestSign);
}
log.debug("Result-Length: {}", result.size());
return PixivDownload.getRanking(result, start - 1, range);
return returnRaw || settingProp == null ? rawValue :
rawValue && !settingProp.getProperty("image.allowR18", "false")
.equalsIgnoreCase("true");
}
private static String buildSyncKey(String... keys) {
StringBuilder sb = new StringBuilder();
for (String string : keys) {
sb.append(string);
}
return sb.toString().intern();
}
}

View File

@ -1,27 +0,0 @@
package net.lamgc.cgj.bot;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import net.lz1998.cq.CQGlobal;
import net.lz1998.cq.EnableCQ;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
@EnableCQ
public class CQConfig {
public static void init() {
CQGlobal.pluginList.add(CQPluginMain.class);
CQGlobal.executor = new ThreadPoolExecutor(
(int) Math.ceil(Runtime.getRuntime().availableProcessors() / 2F),
Runtime.getRuntime().availableProcessors(),
25, TimeUnit.SECONDS,
new LinkedBlockingQueue<>(512),
new ThreadFactoryBuilder()
.setNameFormat("Plugin-ProcessThread-%d")
.build()
);
}
}

View File

@ -0,0 +1,83 @@
package net.lamgc.cgj.bot;
import net.lamgc.cgj.bot.event.MessageEvent;
import net.lamgc.cgj.bot.event.VirtualLoadMessageEvent;
import net.lamgc.utils.event.EventExecutor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Properties;
/**
* 消息事件处理调试器.
* <p>当启用了消息事件处理调试后, 将会根据调试器代号调用指定调试器</p>
*/
@SuppressWarnings("unused")
public enum MessageEventExecutionDebugger {
/**
* PM - 压力测试
*/
PM ((executor, event, properties, log) -> {
MessageEvent virtualLoadEvent = VirtualLoadMessageEvent.toVirtualLoadMessageEvent(event, false);
int rotation = 5;
int number = 50;
int interval = 2500;
try {
rotation = Integer.parseInt(properties.getProperty("debug.pm.rotation", "5"));
} catch(NumberFormatException e) {
log.warn("配置项 {} 值无效, 将使用默认值.({})", "debug.pm.rotation", rotation);
}
try {
number = Integer.parseInt(properties.getProperty("debug.pm.number", "50"));
} catch(NumberFormatException e) {
log.warn("配置项 {} 值无效, 将使用默认值.({})", "debug.pm.number", number);
}
try {
interval = Integer.parseInt(properties.getProperty("debug.pm.interval", "2500"));
} catch(NumberFormatException e) {
log.warn("配置项 {} 值无效, 将使用默认值.({})", "debug.pm.interval", interval);
}
boolean interrupted = false;
Thread currentThread = Thread.currentThread();
for(int rotationCount = 0; rotationCount < rotation && !interrupted; rotationCount++) {
for(int sendCount = 0; sendCount < number; sendCount++) {
if(currentThread.isInterrupted()) {
interrupted = true;
break;
}
executor.executor(virtualLoadEvent);
}
try {
Thread.sleep(interval);
} catch (InterruptedException e) {
break;
}
}
});
public final MessageExecuteDebugger debugger;
MessageEventExecutionDebugger(MessageExecuteDebugger debugger) {
this.debugger = debugger;
}
public static Logger getDebuggerLogger(MessageEventExecutionDebugger debugger) {
return LoggerFactory.getLogger(MessageEventExecutionDebugger.class.getName() + "." + debugger.name());
}
@FunctionalInterface
public interface MessageExecuteDebugger {
/**
* 接收事件并根据指定需求处理
* @param executor 事件执行器
* @param event 消息事件对象
* @param properties 配置项, 调试器应按'debug.[debuggerName].'为前缀存储相应调试信息
* @throws Exception 当抛出异常则打断调试, 并输出至日志
*/
void accept(EventExecutor executor, MessageEvent event, Properties properties, Logger logger) throws Exception;
}
}

View File

@ -1,53 +0,0 @@
package net.lamgc.cgj.bot;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.event.MiraiMessageEvent;
import net.mamoe.mirai.Bot;
import net.mamoe.mirai.japt.Events;
import net.mamoe.mirai.message.FriendMessage;
import net.mamoe.mirai.message.GroupMessage;
import net.mamoe.mirai.qqandroid.QQAndroid;
import net.mamoe.mirai.utils.BotConfiguration;
import org.apache.commons.net.util.Base64;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.*;
import java.util.Properties;
public class MiraiMain implements Closeable {
private final Logger log = LoggerFactory.getLogger(this.toString());
private Bot bot;
private final static Properties botProperties = new Properties();
public void init() {
try {
Class.forName(BotEventHandler.class.getName());
} catch (ClassNotFoundException e) {
log.error("加载BotEventHandler时发生异常", e);
return;
}
File botPropFile = new File("./bot.properties");
try (Reader reader = new BufferedReader(new FileReader(botPropFile))) {
botProperties.load(reader);
} catch (IOException e) {
log.error("机器人配置文件读取失败!", e);
return;
}
bot = QQAndroid.INSTANCE.newBot(Long.parseLong(botProperties.getProperty("bot.qq", "0")), Base64.decodeBase64(botProperties.getProperty("bot.password", "")), new BotConfiguration());
Events.subscribeAlways(GroupMessage.class, (msg) -> BotEventHandler.executor.executor(new MiraiMessageEvent(msg)));
Events.subscribeAlways(FriendMessage.class, (msg) -> BotEventHandler.executor.executor(new MiraiMessageEvent(msg)));
bot.login();
bot.join();
}
public void close() {
bot.close(null);
}
}

View File

@ -0,0 +1,170 @@
package net.lamgc.cgj.bot;
import com.google.common.base.Throwables;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.*;
import java.util.concurrent.atomic.AtomicBoolean;
/**
* 随机间隔发送器
*/
public class RandomIntervalSendTimer extends TimerTask {
private final static Timer timer = new Timer("Thread-RandomIntervalSendTimer", true);
private final static Logger log = LoggerFactory.getLogger(RandomIntervalSendTimer.class);
private final static Map<Long, RandomIntervalSendTimer> timerMap = new HashMap<>();
private final long timerId;
private final Random timeRandom = new Random();
private final AutoSender sender;
private final long time;
private final int floatTime;
private final AtomicBoolean loop = new AtomicBoolean();
private final AtomicBoolean running = new AtomicBoolean();
private final String hashId = Integer.toHexString(this.hashCode());
/**
* 创建一个随机延迟发送器
* @param timerId 该Timer的标识,
* 标识必须是唯一的, 当使用相同id创建时, 将会返回该id所属的Timer对象
* @param sender 自动发送器
* @param time 最低时间(ms)
* @param floatTime 浮动时间(ms)
* @param startNow 现在开始
* @param loop 是否循环
*/
public static RandomIntervalSendTimer createTimer(long timerId, AutoSender sender, long time, int floatTime, boolean startNow, boolean loop) {
if(timerMap.containsKey(timerId)) {
return timerMap.get(timerId);
}
RandomIntervalSendTimer timer = new RandomIntervalSendTimer(timerId, sender, time, floatTime, startNow, loop);
timerMap.put(timerId, timer);
return timer;
}
/**
* 通过Id获取Timer
* @param id 待获取Timer对应的Id
* @return 返回RandomIntervalSendTimer对象
* @throws NoSuchElementException 当不存在Timer时抛出
*/
public static RandomIntervalSendTimer getTimerById(long id) {
if(!timerMap.containsKey(id)) {
throw new NoSuchElementException("id=" + id);
}
return timerMap.get(id);
}
/**
* 获取所有id
* @return 所有TimerId的集合
*/
public static Set<Long> timerIdSet() {
return new HashSet<>(timerMap.keySet());
}
/**
* 创建一个随机延迟发送器
* @param timerId 该Timer的标识
* @param sender 自动发送器
* @param time 最低时间(ms)
* @param floatTime 浮动时间(ms)
* @param startNow 现在开始
* @param loop 是否循环
*/
private RandomIntervalSendTimer(
long timerId,
AutoSender sender,
long time,
int floatTime,
boolean startNow,
boolean loop) {
this.timerId = timerId;
this.sender = sender;
this.time = time;
this.floatTime = floatTime;
if(startNow) {
start(loop);
}
}
public void start() {
start(this.loop.get());
}
/**
* 启动定时器
* @param loop 是否循环, 如果为true, 则任务完成后会自动调用start方法继续循环, 直到被调用{@code #}或总定时器被销毁;
*/
public void start(boolean loop) {
this.loop.set(loop);
long nextDelay = time + (floatTime <= 0 ? 0 : timeRandom.nextInt(floatTime));
Date nextDate = new Date();
nextDate.setTime(nextDate.getTime() + nextDelay);
log.info("定时器 {} 下一延迟: {}ms ({})", hashId, nextDelay, nextDate);
if(running.get()) {
reset();
return;
}
running.set(true);
timer.schedule(this, nextDelay);
}
public void reset() {
timerMap.put(timerId, (RandomIntervalSendTimer) clone());
}
@Override
public void run() {
log.info("定时器 {} 开始执行...(Sender: {}@{})", this.hashId, sender.getClass().getSimpleName(), sender.hashCode());
try {
sender.send();
} catch (Exception e) {
log.error("定时器 {} 执行时发生异常:\n{}",
Integer.toHexString(this.hashCode()),
Throwables.getStackTraceAsString(e));
}
log.info("定时器 {} 执行结束.", this.hashId);
if (this.loop.get()) {
start();
}
}
/**
* 取消该定时器
* @return 取消成功返回true
*/
@Override
public boolean cancel() {
running.set(false);
loop.set(false);
return super.cancel();
}
/**
* 销毁这个定时器
*/
public void destroy() {
cancel();
timerMap.remove(this.timerId);
}
/**
* 克隆一个参数完全一样的TimerTask对象.
* @return 返回对象不同, 参数相同的TimerTask对象.
*/
@Override
@SuppressWarnings("MethodDoesntCallSuperMethod")
public Object clone() {
RandomIntervalSendTimer newTimerTask = new RandomIntervalSendTimer(
this.timerId, this.sender,
time, floatTime,
running.get(), loop.get());
this.destroy();
return newTimerTask;
}
}

View File

@ -0,0 +1,129 @@
package net.lamgc.cgj.bot;
import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.cache.CacheStoreCentral;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivURL;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Calendar;
import java.util.Date;
import java.util.List;
import java.util.Random;
/**
* 推荐作品发送器
*/
public class RandomRankingArtworksSender extends AutoSender {
private final Logger log;
private final long groupId;
private final int rankingStart;
private final int rankingStop;
private final PixivURL.RankingMode mode;
private final PixivURL.RankingContentType contentType;
private final PixivDownload.PageQuality quality;
/**
* 构造一个推荐作品发送器
* @param messageSender 消息发送器
* @param rankingStart 排行榜开始范围(从1开始, 名次)如传入0或负数则为默认值默认为1
* @param rankingStop 排名榜结束范围(包括该名次)如传入0或负数则为默认值默认为150
* @param mode 排行榜模式
* @param contentType 排行榜内容类型
* @param quality 图片质量, 详见{@link PixivDownload.PageQuality}
* @throws IndexOutOfBoundsException 当 rankingStart > rankingStop时抛出
*/
public RandomRankingArtworksSender(
MessageSender messageSender,
int rankingStart,
int rankingStop,
PixivURL.RankingMode mode,
PixivURL.RankingContentType contentType,
PixivDownload.PageQuality quality) {
this(messageSender, 0, rankingStart, rankingStop, mode, contentType, quality);
}
/**
* 构造一个推荐作品发送器
* @param messageSender 消息发送器
* @param groupId 群组Id, 如果发送目标为群组, 则可设置群组Id, 以使用群组配置.
* @param rankingStart 排行榜开始范围(从1开始, 名次)如传入0或负数则为默认值默认为1
* @param rankingStop 排名榜结束范围(包括该名次)如传入0或负数则为默认值默认为150
* @param mode 排行榜模式
* @param contentType 排行榜内容类型
* @param quality 图片质量, 详见{@link PixivDownload.PageQuality}
* @throws IndexOutOfBoundsException 当 rankingStart > rankingStop时抛出
*/
public RandomRankingArtworksSender(
MessageSender messageSender,
long groupId,
int rankingStart,
int rankingStop,
PixivURL.RankingMode mode,
PixivURL.RankingContentType contentType,
PixivDownload.PageQuality quality) {
super(messageSender);
this.groupId = groupId;
this.mode = mode;
this.contentType = contentType;
log = LoggerFactory.getLogger(this.toString());
this.rankingStart = rankingStart > 0 ? rankingStart : 1;
this.rankingStop = rankingStop > 0 ? rankingStop : 150;
if(this.rankingStart > this.rankingStop) {
throw new IndexOutOfBoundsException("rankingStart=" + this.rankingStart + ", rankingStop=" + this.rankingStop);
}
this.quality = quality == null ? PixivDownload.PageQuality.REGULAR : quality;
}
@Override
public void send() {
Date queryDate = new Date();
Calendar calendar = Calendar.getInstance();
calendar.setTime(queryDate);
if(calendar.get(Calendar.HOUR_OF_DAY) < 12) {
calendar.add(Calendar.DAY_OF_YEAR, -2);
} else {
calendar.add(Calendar.DAY_OF_YEAR, -1);
}
queryDate = calendar.getTime();
int selectRanking = rankingStart + new Random().nextInt(rankingStop - rankingStart + 1);
try {
List<JsonObject> rankingList = CacheStoreCentral.getCentral().getRankingInfoByCache(
contentType,
mode,
queryDate,
selectRanking,
1, false);
log.debug("RankingResult.size: {}", rankingList.size());
if(rankingList.size() != 1) {
log.error("排行榜选取失败!(获取到了多个结果)");
return;
}
JsonObject rankingInfo = rankingList.get(0);
int illustId = rankingInfo.get("illust_id").getAsInt();
if(BotCommandProcess.isNoSafe(illustId,
SettingProperties.getProperties(groupId), false)) {
log.warn("作品为r18作品, 取消本次发送.");
return;
} else if(BotCommandProcess.isReported(illustId)) {
log.warn("作品Id {} 被报告, 正在等待审核, 跳过该作品.", illustId);
return;
}
String message = "#美图推送 - 今日排行榜 第 " + rankingInfo.get("rank").getAsInt() + "\n" +
"标题:" + rankingInfo.get("title").getAsString() + "(" + illustId + ")\n" +
"作者:" + rankingInfo.get("user_name").getAsString() + "\n" +
CacheStoreCentral.getCentral().getImageById(0, illustId, quality, 1) +
"\n如有不当作品可使用\".cgj report -id " + illustId + "\"向色图姬反馈。";
getMessageSender().sendMessage(message);
} catch (Exception e) {
log.error("发送随机作品时发生异常", e);
}
}
}

View File

@ -1,10 +1,12 @@
package net.lamgc.cgj.bot;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.event.VirtualLoadMessageEvent;
import net.lamgc.cgj.pixiv.PixivURL;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.text.SimpleDateFormat;
import java.time.LocalDate;
import java.util.Calendar;
import java.util.Date;
@ -25,38 +27,42 @@ public class RankingUpdateTimer {
Calendar cal = Calendar.getInstance();
cal.setTime(firstRunDate == null ? new Date() : firstRunDate);
LocalDate currentLocalDate = LocalDate.now();
if(cal.get(Calendar.DAY_OF_YEAR) <= currentLocalDate.getDayOfYear() && cal.get(Calendar.HOUR_OF_DAY) >= 12) {
cal.set(Calendar.DAY_OF_YEAR, cal.get(Calendar.DAY_OF_YEAR) + 1);
if(cal.get(Calendar.DAY_OF_YEAR) < currentLocalDate.getDayOfYear() || (
cal.get(Calendar.DAY_OF_YEAR) == currentLocalDate.getDayOfYear() &&
(cal.get(Calendar.HOUR_OF_DAY) * 60 + cal.get(Calendar.MINUTE) >= 690))) {
cal.set(Calendar.DAY_OF_YEAR, currentLocalDate.getDayOfYear() + 1);
}
cal.set(Calendar.HOUR_OF_DAY, 12);
cal.set(Calendar.HOUR_OF_DAY, 11);
cal.set(Calendar.MINUTE, 30);
cal.set(Calendar.SECOND, 0);
cal.set(Calendar.MILLISECOND, 0);
log.warn("已设置排行榜定时更新, 首次运行时间: {}", cal.getTime());
long delay = cal.getTime().getTime() - System.currentTimeMillis();
log.warn("已设置排行榜定时更新, 首次运行时间: {} ({}min)", cal.getTime(), delay / 1000 / 60);
timer.schedule(new TimerTask() {
@Override
public void run() {
now(null);
}
}, cal.getTime(), 86400000); // 1 Day
}, delay, 86400000); // 1 Day
}
public void now(Date queryDate) {
log.info("当前时间 {}, 定时任务开始执行...", new Date());
log.warn("当前时间 {}, 定时任务开始执行...", new Date());
Calendar calendar = Calendar.getInstance();
calendar.setTime(queryDate == null ? new Date() : queryDate);
LocalDate currentLocalDate = LocalDate.now();
if(calendar.get(Calendar.DAY_OF_YEAR) == currentLocalDate.getDayOfYear() ||
calendar.get(Calendar.DAY_OF_YEAR) == currentLocalDate.getDayOfYear() - 1) {
if(calendar.get(Calendar.HOUR_OF_DAY) < 12) {
if(calendar.get(Calendar.HOUR_OF_DAY) < 11) {
calendar.add(Calendar.DAY_OF_YEAR, -2);
} else {
calendar.add(Calendar.DAY_OF_YEAR, -1);
}
}
String dateStr = new SimpleDateFormat("yyyy-MM-dd").format(calendar.getTime());
log.info("正在获取 {} 期排行榜数据...", calendar.getTime());
for (PixivURL.RankingMode rankingMode : PixivURL.RankingMode.values()) {
for (PixivURL.RankingContentType contentType : PixivURL.RankingContentType.values()) {
@ -64,15 +70,12 @@ public class RankingUpdateTimer {
log.debug("不支持的类型, 填空值跳过...(类型: {}.{})", rankingMode.name(), contentType.name());
}
log.info("当前排行榜类型: {}.{}, 正在更新...", rankingMode.name(), contentType.name());
try {
BotCommandProcess.getRankingInfoByCache(contentType, rankingMode, calendar.getTime(), 1, 0, true);
log.info("排行榜 {}.{} 更新完成.", rankingMode.name(), contentType.name());
} catch (IOException e) {
log.error("排行榜 {}.{} 更新时发生异常", rankingMode.name(), contentType.name());
log.error("异常信息如下", e);
}
BotEventHandler.executeMessageEvent(new VirtualLoadMessageEvent(0,0,
".cgj ranking -type=" + contentType.name() + " -mode=" + rankingMode.name() + " -force -date " + dateStr));
log.info("排行榜 {}.{} 负载指令已投递.", rankingMode.name(), contentType.name());
}
}
log.warn("定时任务更新完成.");
}
/**

View File

@ -0,0 +1,246 @@
package net.lamgc.cgj.bot;
import com.google.common.base.Throwables;
import net.lamgc.cgj.bot.boot.BotGlobal;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.*;
import java.nio.charset.StandardCharsets;
import java.util.*;
public final class SettingProperties {
private final static Logger log = LoggerFactory.getLogger(SettingProperties.class);
private final static File globalPropFile = new File(getPropertiesDir(), "global.properties");
private final static Properties globalProp = new Properties();
private final static Map<Long, Properties> groupPropMap = new HashMap<>();
private final static Set<Long> changeList = Collections.synchronizedSet(new HashSet<>());
/**
* 全局配置项
*/
public final static long GLOBAL = 0;
/**
* 清空所有Properties.
*/
public static void clearProperties() {
groupPropMap.clear();
globalProp.clear();
}
/**
* 加载配置文件
*/
public static void loadProperties() {
loadGlobalProperties();
File[] files = getPropertiesDir()
.listFiles((dir, fileName) -> fileName.startsWith("group.") && fileName.endsWith(".properties"));
if(files == null) {
log.error("检索群组配置文件失败, 可能是被拒绝访问.");
return;
}
for (File file : files) {
String name = file.getName();
long groupId;
try {
groupId = Long.parseLong(name.substring(name.indexOf("group.") + 6, name.lastIndexOf(".properties")));
if(groupId <= 0) {
log.warn("无效的群配置文件: {}", groupId);
continue;
}
} catch (NumberFormatException e) {
log.error("非法的配置文件名: {}", name);
continue;
}
if(!groupPropMap.containsKey(groupId)) {
groupPropMap.put(groupId, new Properties(globalProp));
}
loadGroupProperties(groupId, groupPropMap.get(groupId));
}
}
/**
* 保存配置项
*/
public static void saveProperties() {
log.info("正在保存所有配置...");
saveGlobalProperties();
for (Long groupId : groupPropMap.keySet()) {
if(!changeList.contains(groupId)) {
log.debug("群组 {} 配置无改动, 忽略保存.", groupId);
return;
}
log.debug("正在保存群组 {} 配置文件...", groupId);
saveGroupProperties(groupId);
}
log.info("配置保存完成.");
}
/**
* 保存指定群组的配置文件
* @param groupId 要保存配置的群组Id
*/
private static void saveGroupProperties(long groupId) {
try {
saveGroupProperties(groupId, getGroupProperties(groupId));
} catch (IOException e) {
log.error("群组 {} 配置保存失败\n{}", groupId, Throwables.getStackTraceAsString(e));
}
}
private static void saveGroupProperties(Long groupId, Properties properties) throws IOException {
File groupPropFile = new File(getPropertiesDir(), "group." + groupId + ".properties");
if(!groupPropFile.exists() && !groupPropFile.createNewFile()) {
log.error("群组 {} 配置文件创建失败!", groupId);
return;
}
saveProperties(properties, new FileOutputStream(groupPropFile));
}
private static void loadGlobalProperties() {
if(globalPropFile.exists() && globalPropFile.isFile()) {
log.info("正在加载全局配置文件...");
try (Reader reader = new InputStreamReader(new FileInputStream(globalPropFile), StandardCharsets.UTF_8)) {
globalProp.load(reader);
log.info("全局配置文件加载完成.");
} catch (IOException e) {
log.error("加载全局配置文件时发生异常", e);
}
} else {
log.info("未找到全局配置文件,跳过加载.");
}
}
/**
* 保存全局配置项
*/
private static void saveGlobalProperties() {
try {
if(!globalPropFile.exists() && !globalPropFile.createNewFile()) {
log.error("创建全局配置文件失败.");
return;
}
saveProperties(globalProp, new FileOutputStream(globalPropFile));
} catch (IOException e) {
log.error("全局配置文件保存时发生异常", e);
}
}
private static void loadGroupProperties(long groupId, Properties properties) {
File propFile = new File(getPropertiesDir(), "group." + groupId + ".properties");
Properties groupProp = Objects.requireNonNull(properties);
if(!propFile.exists() || !propFile.isFile()) {
log.warn("群组 {} 配置文件不存在, 或不是一个文件.({})", groupId, propFile.getAbsolutePath());
return;
}
try (Reader reader = new InputStreamReader(new FileInputStream(propFile), StandardCharsets.UTF_8)) {
groupProp.load(reader);
} catch (IOException e) {
log.error("读取群组 {} 群配置文件时发生异常:\n{}", groupId, Throwables.getStackTraceAsString(e));
}
}
private static void saveProperties(Properties properties, OutputStream stream) throws IOException {
properties.store(new OutputStreamWriter(stream, StandardCharsets.UTF_8), null);
}
/**
* 获取配置文件目录
* @return 返回目录File对象.
*/
private static File getPropertiesDir() {
File propDir = new File(BotGlobal.getGlobal().getDataStoreDir(), "/setting/");
if(!propDir.exists() && !propDir.mkdirs()) {
log.warn("Setting文件夹创建失败!");
}
return propDir;
}
public static String getProperty(long groupId, String key) {
return getProperty(groupId, key, null);
}
public static String getProperty(long groupId, String key, String defaultValue) {
if(groupId <= 0) {
return globalProp.getProperty(key, defaultValue);
} else {
Properties properties = groupPropMap.get(groupId);
return properties == null ? defaultValue : properties.getProperty(key, defaultValue);
}
}
/**
* 设置配置项
* @param groupId 群组Id, 如为0或负数则为全局配置
* @param key 配置项key名
* @param value 欲设置的新值, 如为null则删除该配置项
* @return 返回上一次设定值
*/
public static String setProperty(long groupId, String key, String value) {
Objects.requireNonNull(key);
Properties targetProperties;
if(groupId <= 0) {
targetProperties = globalProp;
} else {
changeList.add(groupId);
targetProperties = getGroupProperties(groupId);
}
String lastValue = targetProperties.getProperty(key);
if(value != null) {
targetProperties.setProperty(key, value);
} else {
targetProperties.remove(key);
}
return lastValue;
}
/**
* 获取GlobalProperties
* @return 全局Properties
*/
private static Properties getGlobalProperties() {
return globalProp;
}
/**
* 获取群组Properties
* @param groupId 群组Id
* @return 如果存在, 返回Properties, 不存在返回null.
* @throws IllegalArgumentException 当群组Id 小于或等于0 时抛出.
*/
private static Properties getGroupProperties(long groupId) {
if (groupId <= 0) {
throw new IllegalArgumentException("Group number cannot be 0 or negative: " + groupId);
}
if(!groupPropMap.containsKey(groupId)) {
groupPropMap.put(groupId, new Properties(globalProp));
}
return groupPropMap.get(groupId);
}
/**
* 获取群组 Properties, 如果指定群组没有 Properties, 则使用GlobalProperties.
* @param groupId 指定群组Id
* @return 如果群组存在所属Properties, 则返回群组Properties, 否则返回GlobalProperties.
*/
public static Properties getProperties(long groupId) {
if(groupId > 0 && groupPropMap.containsKey(groupId)) {
return groupPropMap.get(groupId);
}
return getGlobalProperties();
}
}

View File

@ -0,0 +1,46 @@
package net.lamgc.cgj.bot.boot;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.util.PropertiesUtils;
import net.lamgc.utils.base.ArgumentsProperties;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public final class ApplicationBoot {
private final static Logger log = LoggerFactory.getLogger(ApplicationBoot.class);
private ApplicationBoot() {}
/**
* 初始化应用.
* <p>该方法不会初始化机器人, 仅初始化应用所需的配置信息.</p>
*/
public static void initialApplication(String[] args) {
ArgumentsProperties argsProp = new ArgumentsProperties(args);
if(!PropertiesUtils.getSettingToSysProp(argsProp, "proxy", null)) {
PropertiesUtils.getEnvSettingToSysProp("CGJ_PROXY", "proxy", null);
}
if(!PropertiesUtils.getSettingToSysProp(argsProp, "botDataDir", "./") &&
!PropertiesUtils.getEnvSettingToSysProp("CGJ_BOT_DATA_DIR", "botDataDir", "./")) {
log.warn("未设置botDataDir, 当前运行目录将作为酷Q机器人所在目录.");
}
if(!PropertiesUtils.getSettingToSysProp(argsProp, "redisAddress", "127.0.0.1") &&
!PropertiesUtils.getEnvSettingToSysProp("CGJ_REDIS_URI", "redisAddress", "127.0.0.1")) {
log.warn("未设置RedisAddress, 将使用默认值连接Redis服务器(127.0.0.1:6379)");
}
// 初始化 BotGlobal
//noinspection ResultOfMethodCallIgnored 这里仅仅是加载BotGlobal而已
BotGlobal.getGlobal();
}
/**
* 初始化机器人.
* <p>本方法由框架调用.</p>
*/
public static void initialBot() {
BotEventHandler.initial();
}
}

View File

@ -0,0 +1,132 @@
package net.lamgc.cgj.bot.boot;
import com.google.common.base.Strings;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import net.lamgc.cgj.pixiv.PixivDownload;
import org.apache.http.HttpHost;
import org.apache.http.client.CookieStore;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.exceptions.JedisConnectionException;
import java.io.File;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URL;
import java.nio.file.Files;
public final class BotGlobal {
private final static Logger log = LoggerFactory.getLogger(BotGlobal.class);
private final static BotGlobal instance = new BotGlobal();
public static BotGlobal getGlobal() {
if(instance == null) {
throw new IllegalStateException("BotGlobal has not been initialized");
}
return instance;
}
private final URI redisUri;
/**
* 所有缓存共用的JedisPool
*/
private final JedisPool redisServer;
private final File dataStoreDir;
private final HttpHost proxy;
private CookieStore cookieStore;
private final Gson gson = new GsonBuilder()
.serializeNulls()
.create();
private PixivDownload pixivDownload;
private final File imageStoreDir;
private BotGlobal() {
this.redisUri = URI.create("redis://" + System.getProperty("cgj.redisAddress"));
this.redisServer = new JedisPool(
getRedisUri().getHost(),
getRedisUri().getPort() == -1 ? 6379 : getRedisUri().getPort());
try (Jedis jedis = this.redisServer.getResource()) {
log.warn("Redis连接状态(Ping): {}", jedis.ping().equalsIgnoreCase("pong"));
} catch(JedisConnectionException e) {
log.warn("Redis连接失败, 将会影响到后续功能运行.({})", e.getCause().getMessage());
}
String dataStoreDirPath = System.getProperty("cgj.botDataDir");
this.dataStoreDir = new File((!dataStoreDirPath.endsWith("/") || !dataStoreDirPath.endsWith("\\")) ?
dataStoreDirPath + System.getProperty("file.separator") : dataStoreDirPath);
this.imageStoreDir = new File(getDataStoreDir(), "data/image/cgj/");
String proxyAddress = System.getProperty("cgj.proxy");
HttpHost temp = null;
if(!Strings.isNullOrEmpty(proxyAddress)) {
try {
URL proxyUrl = new URL(proxyAddress);
temp = new HttpHost(proxyUrl.getHost(), proxyUrl.getPort());
log.info("已启用代理:{}", temp.toHostString());
} catch (MalformedURLException e) {
log.error("Proxy地址解析失败, 代理将不会启用.", e);
}
}
this.proxy = temp;
}
public URI getRedisUri() {
return redisUri;
}
public File getDataStoreDir() {
if(!dataStoreDir.exists() && !dataStoreDir.mkdirs()) {
log.error("DataStoreDir 创建失败, 数据存储可能失效!");
}
return dataStoreDir;
}
public JedisPool getRedisServer() {
return redisServer;
}
public HttpHost getProxy() {
return proxy;
}
public void setCookieStore(CookieStore cookieStore) {
if(this.cookieStore != null) {
throw new IllegalStateException("CookieStore set");
}
this.cookieStore = cookieStore;
this.pixivDownload =
new PixivDownload(cookieStore, proxy);
}
public Gson getGson() {
return gson;
}
public PixivDownload getPixivDownload() {
return pixivDownload;
}
public File getImageStoreDir() {
if(!imageStoreDir.exists() && !Files.isSymbolicLink(imageStoreDir.toPath())) {
if(!imageStoreDir.mkdirs()) {
log.warn("酷Q图片缓存目录失效(Path: {} )", imageStoreDir.getAbsolutePath());
throw new RuntimeException(new IOException("文件夹创建失败!"));
}
}
return imageStoreDir;
}
}

View File

@ -0,0 +1,52 @@
package net.lamgc.cgj.bot.cache;
import com.google.common.base.Throwables;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Set;
import java.util.Timer;
import java.util.TimerTask;
import java.util.concurrent.CopyOnWriteArraySet;
public class AutoCleanTimer extends TimerTask {
private final static Set<Cleanable> cleanSet = new CopyOnWriteArraySet<>();
private final static Timer cleanTimer = new Timer("Thread-AutoClean", true);
private final static Logger log = LoggerFactory.getLogger(AutoCleanTimer.class);
static {
cleanTimer.schedule(new AutoCleanTimer(), 100L, 100L);
}
/**
* 增加需要定时执行清理的缓存库
* @param store 已实现Cleanable的对象
*/
public static void add(Cleanable store) {
cleanSet.add(store);
}
/**
* 移除已添加的缓存库
* @param store 需要从AutoCleanTimer移除的对象
*/
public static void remove(Cleanable store) {
cleanSet.remove(store);
}
private AutoCleanTimer() {}
@Override
public void run() {
cleanSet.forEach(cleanable -> {
try {
cleanable.clean();
} catch (Exception e) {
log.error("{} 执行清理动作时发生异常:\n{}", cleanable.toString(), Throwables.getStackTraceAsString(e));
}
});
}
}

View File

@ -1,14 +1,19 @@
package net.lamgc.cgj.bot.cache;
import java.util.Date;
import java.util.Set;
/**
* 缓存库接口
* @param <T> 缓存数据类型
*/
public interface CacheStore<T> {
/**
* 更新或添加缓存项
* @param key 缓存键名
* @param value 缓存值
* @param expire 有效期, 单位为ms(毫秒), 如不过期传入0或赋值
* @param expire 有效期, 单位为ms(毫秒), 如不过期传入0或负数
*/
void update(String key, T value, long expire);
@ -64,6 +69,19 @@ public interface CacheStore<T> {
*/
boolean clear();
/**
* 获取key集合
* @return 返回存储缓存库中所有缓存项key的集合
*/
Set<String> keys();
/**
* 删除指定缓存项
* @param key 缓存项key
* @return 删除成功返回true
*/
boolean remove(String key);
/**
* 是否支持持久化
* @return 如果支持返回true

View File

@ -0,0 +1,699 @@
package net.lamgc.cgj.bot.cache;
import com.google.common.base.Strings;
import com.google.common.base.Throwables;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.BotCode;
import net.lamgc.cgj.bot.BotCommandProcess;
import net.lamgc.cgj.bot.SettingProperties;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.exception.HttpRequestException;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivSearchLinkBuilder;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.cgj.util.Locker;
import net.lamgc.cgj.util.LockerMap;
import net.lamgc.cgj.util.URLs;
import net.lamgc.utils.encrypt.MessageDigestUtils;
import net.lz1998.cq.utils.CQCode;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.util.EntityUtils;
import org.apache.tomcat.util.http.fileupload.IOUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.*;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.concurrent.atomic.AtomicInteger;
@SuppressWarnings("SynchronizationOnLocalVariableOrMethodParameter")
public final class CacheStoreCentral {
private final static Logger log = LoggerFactory.getLogger(CacheStoreCentral.class);
private static CacheStoreCentral central = new CacheStoreCentral();
public static CacheStoreCentral getCentral() {
if(central == null) {
initialCentral();
}
return central;
}
private synchronized static void initialCentral() {
if(central != null) {
return;
}
central = new CacheStoreCentral();
}
private final LockerMap<String> lockerMap = new LockerMap<>();
private CacheStoreCentral() {}
private final Hashtable<String, File> imageCache = new Hashtable<>();
private final CacheStore<JsonElement> imageChecksumCache =
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(),
"imageChecksum", BotGlobal.getGlobal().getGson());
/*
* 注意:
* 在启用了远端缓存的情况下, 不允许滥用本地缓存
* 只有在处理命令中需要短时间大量存取的缓存项才能进行本地缓存(例如PreLoadData需要在排序中大量获取);
* 如果没有短时间大量存取的需要, 切勿使用本地缓存
*/
/**
* 作品信息缓存 - 不过期
*/
private final CacheStore<JsonElement> illustInfoCache =
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(),
"illustInfo", BotGlobal.getGlobal().getGson());
/**
* 作品信息预加载数据 - 有效期 2 小时, 本地缓存有效期 0.5 ± 0.25 小时
*/
private final CacheStore<JsonElement> illustPreLoadDataCache =
CacheStoreUtils.hashLocalHotDataStore(
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(),
"illustPreLoadData", BotGlobal.getGlobal().getGson()), 600000, 120000);
/**
* 搜索内容缓存, 有效期 2 小时
*/
private final CacheStore<JsonElement> searchBodyCache =
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(),
"searchBody", BotGlobal.getGlobal().getGson());
/**
* 排行榜缓存, 不过期
*/
private final CacheStore<List<JsonObject>> rankingCache =
new JsonObjectRedisListCacheStore(BotGlobal.getGlobal().getRedisServer(),
"ranking", BotGlobal.getGlobal().getGson());
/**
* 作品页面下载链接缓存 - 不过期
*/
private final CacheStore<List<String>> pagesCache =
new StringListRedisCacheStore(BotGlobal.getGlobal().getRedisServer(), "imagePages");
/**
* 清空所有缓存
*/
public void clearCache() {
imageCache.clear();
illustInfoCache.clear();
illustPreLoadDataCache.clear();
searchBodyCache.clear();
rankingCache.clear();
pagesCache.clear();
}
/**
* 通过illustId获取作品图片
* @param fromGroup 来源群(系统提供)
* @param illustId 作品Id
* @param quality 图片质量
* @param pageIndex 指定页面索引, 从1开始
* @return 如果成功, 返回BotCode, 否则返回错误信息.
*/
public String getImageById(long fromGroup, int illustId, PixivDownload.PageQuality quality, int pageIndex) throws InterruptedException {
log.debug("IllustId: {}, Quality: {}, PageIndex: {}", illustId, quality.name(), pageIndex);
if(pageIndex <= 0) {
log.warn("指定的页数不能小于或等于0: {}", pageIndex);
return "指定的页数不能小于或等于0";
}
try {
if (BotCommandProcess.isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) {
log.warn("作品 {} 存在R-18内容且设置\"image.allowR18\"为false将屏蔽该作品不发送.", illustId);
return "(根据设置,该作品已被屏蔽!)";
} else if(BotCommandProcess.isReported(illustId)) {
log.warn("作品Id {} 被报告, 正在等待审核, 跳过该作品.", illustId);
return "(该作品已被封印)";
}
} catch (IOException e) {
log.warn("作品信息无法获取!", e);
return "(发生网络异常,无法获取图片!)";
}
List<String> pagesList;
try {
pagesList = getIllustPages(illustId, quality, false);
} catch (IOException e) {
log.error("获取下载链接列表时发生异常", e);
return "发生网络异常,无法获取图片!";
}
if(log.isDebugEnabled()) {
StringBuilder logBuilder = new StringBuilder("作品Id " + illustId + " 所有页面下载链接: \n");
AtomicInteger index = new AtomicInteger();
pagesList.forEach(item ->
logBuilder.append(index.incrementAndGet()).append(". ").append(item).append("\n"));
log.debug(logBuilder.toString());
}
if (pagesList.size() < pageIndex) {
log.warn("指定的页数超出了总页数({} / {})", pageIndex, pagesList.size());
return "指定的页数超出了范围(总共 " + pagesList.size() + " 页)";
}
String downloadLink = pagesList.get(pageIndex - 1);
String fileName = URLs.getResourceName(Strings.nullToEmpty(downloadLink));
File imageFile = new File(BotGlobal.getGlobal().getImageStoreDir(),
downloadLink.substring(downloadLink.lastIndexOf("/") + 1));
log.debug("FileName: {}, DownloadLink: {}", fileName, downloadLink);
if(!imageCache.containsKey(fileName)) {
if(imageFile.exists() && imageFile.isFile()) {
ImageChecksum imageChecksum = getImageChecksum(illustId, pageIndex);
if(imageChecksum != null) {
try {
log.trace("正在检查作品Id {} 第 {} 页图片文件 {} ...", illustId, pageIndex, imageFile.getName());
if (ImageChecksum.checkFile(imageChecksum, Files.readAllBytes(imageFile.toPath()))) {
imageCache.put(URLs.getResourceName(downloadLink), imageFile);
log.trace("作品Id {} 第 {} 页缓存已补充.", illustId, pageIndex);
return getImageToBotCode(imageFile, false).toString();
} else {
log.warn("图片文件 {} 校验失败, 重新下载图片...", imageFile.getName());
}
} catch(IOException e) {
log.error("文件检验时读取失败, 重新下载文件...(file: {})", imageFile.getPath());
}
} else {
log.warn("图片存在但校验不存在, 重新下载图片...");
}
}
try {
Throwable throwable = ImageCacheStore.executeCacheRequest(
new ImageCacheObject(imageCache, illustId, pageIndex, downloadLink, imageFile));
if(throwable != null) {
throw throwable;
}
} catch (InterruptedException e) {
log.warn("图片缓存被中断", e);
throw e;
} catch (Throwable e) {
log.error("图片 {} 获取失败:\n{}", illustId + "p" + pageIndex, Throwables.getStackTraceAsString(e));
return "(错误: 图片获取出错)";
}
} else {
log.trace("图片 {} 缓存命中.", fileName);
}
return getImageToBotCode(imageCache.get(fileName), false).toString();
}
/**
* 通过文件获取图片的BotCode代码
* @param targetFile 图片文件
* @param updateCache 是否刷新缓存(只是让机器人重新上传, 如果上传接口有重复检测的话是无法处理的)
* @return 返回设定好参数的BotCode
*/
@SuppressWarnings("SameParameterValue")
private BotCode getImageToBotCode(File targetFile, boolean updateCache) {
String fileName = Objects.requireNonNull(targetFile, "targetFile is null").getName();
BotCode code = BotCode.parse(
CQCode.image(BotGlobal.getGlobal().getImageStoreDir().getName() + "/" + fileName));
code.addParameter("absolutePath", targetFile.getAbsolutePath());
code.addParameter("imageName", fileName.substring(0, fileName.lastIndexOf(".")));
code.addParameter("updateCache", updateCache ? "true" : "false");
return code;
}
/**
* 获取作品信息
* @param illustId 作品Id
* @param flushCache 强制刷新缓存
* @return 返回作品信息
* @throws IOException 当Http请求发生异常时抛出
* @throws NoSuchElementException 当作品未找到时抛出
*/
public JsonObject getIllustInfo(int illustId, boolean flushCache)
throws IOException, NoSuchElementException {
Locker<String> locker = buildSyncKey(Integer.toString(illustId));
String illustIdStr = locker.getKey();
JsonObject illustInfoObj = null;
if (!illustInfoCache.exists(illustIdStr) || flushCache) {
try {
locker.lock();
synchronized (locker) {
if (!illustInfoCache.exists(illustIdStr) || flushCache) {
illustInfoObj = BotGlobal.getGlobal().getPixivDownload().getIllustInfoByIllustId(illustId);
illustInfoCache.update(illustIdStr, illustInfoObj, null);
}
}
} finally {
locker.unlock();
}
}
if(Objects.isNull(illustInfoObj)) {
illustInfoObj = illustInfoCache.getCache(illustIdStr).getAsJsonObject();
log.trace("作品Id {} IllustInfo缓存命中.", illustId);
}
return illustInfoObj;
}
/**
* 获取作品预加载数据.
* 可以获取作品的一些与用户相关的信息
* @param illustId 作品Id
* @param flushCache 是否刷新缓存
* @return 成功返回JsonObject对象
* @throws IOException 当Http请求处理发生异常时抛出
*/
public JsonObject getIllustPreLoadData(int illustId, boolean flushCache) throws IOException {
Locker<String> locker = buildSyncKey(Integer.toString(illustId));
String illustIdStr = locker.getKey();
JsonObject result = null;
if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) {
try {
locker.lock();
synchronized (locker) {
if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) {
log.trace("IllustId {} 缓存失效, 正在更新...", illustId);
JsonObject preLoadDataObj = BotGlobal.getGlobal().getPixivDownload()
.getIllustPreLoadDataById(illustId)
.getAsJsonObject("illust")
.getAsJsonObject(Integer.toString(illustId));
long expire = 7200 * 1000;
String propValue = SettingProperties.
getProperty(SettingProperties.GLOBAL, "cache.illustPreLoadData.expire", "7200000");
log.debug("PreLoadData有效时间设定: {}", propValue);
try {
expire = Long.parseLong(propValue);
} catch (Exception e) {
log.warn("全局配置项 \"{}\" 值非法, 已使用默认值: {}", propValue, expire);
}
result = preLoadDataObj;
illustPreLoadDataCache.update(illustIdStr, preLoadDataObj, expire);
log.trace("作品Id {} preLoadData缓存已更新(有效时间: {})", illustId, expire);
}
}
} finally {
locker.unlock();
}
}
if(Objects.isNull(result)) {
result = illustPreLoadDataCache.getCache(illustIdStr).getAsJsonObject();
log.trace("作品Id {} PreLoadData缓存命中.", illustId);
}
return result;
}
public List<String> getIllustPages(int illustId, PixivDownload.PageQuality quality, boolean flushCache)
throws IOException {
Locker<String> locker
= buildSyncKey(Integer.toString(illustId), ".", quality.name());
String pagesSign = locker.getKey();
List<String> result = null;
if (!pagesCache.exists(pagesSign) || flushCache) {
try {
locker.lock();
synchronized (locker) {
if (!pagesCache.exists(pagesSign) || flushCache) {
List<String> linkList = PixivDownload
.getIllustAllPageDownload(BotGlobal.getGlobal().getPixivDownload().getHttpClient(),
BotGlobal.getGlobal().getPixivDownload().getCookieStore(), illustId, quality);
result = linkList;
pagesCache.update(pagesSign, linkList, null);
}
}
} finally {
locker.unlock();
}
}
if(Objects.isNull(result)) {
result = pagesCache.getCache(pagesSign);
log.trace("作品Id {} Pages缓存命中.", illustId);
}
return result;
}
private final Random expireTimeFloatRandom = new Random();
/**
* 获取排行榜
* @param contentType 排行榜类型
* @param mode 排行榜模式
* @param queryDate 查询时间
* @param start 开始排名, 从1开始
* @param range 取范围
* @param flushCache 是否强制刷新缓存
* @return 成功返回有值List, 失败且无异常返回空
* @throws IOException 获取异常时抛出
*/
public List<JsonObject> getRankingInfoByCache(PixivURL.RankingContentType contentType,
PixivURL.RankingMode mode,
Date queryDate, int start, int range, boolean flushCache)
throws IOException {
if(!contentType.isSupportedMode(mode)) {
log.warn("试图获取不支持的排行榜类型已拒绝.(ContentType: {}, RankingMode: {})", contentType.name(), mode.name());
if(log.isDebugEnabled()) {
try {
Thread.dumpStack();
} catch(Exception e) {
log.debug("本次非法请求的堆栈信息如下: \n{}", Throwables.getStackTraceAsString(e));
}
}
return new ArrayList<>(0);
}
String date = new SimpleDateFormat("yyyyMMdd").format(queryDate);
Locker<String> locker
= buildSyncKey(contentType.name(), ".", mode.name(), ".", date);
String requestSign = locker.getKey();
List<JsonObject> result = null;
if(!rankingCache.exists(requestSign) || flushCache) {
try {
locker.lock();
synchronized (locker) {
if (!rankingCache.exists(requestSign) || flushCache) {
log.trace("Ranking缓存失效, 正在更新...(RequestSign: {})", requestSign);
List<JsonObject> rankingResult = BotGlobal.getGlobal().getPixivDownload()
.getRanking(contentType, mode, queryDate, 1, 500);
long expireTime = 0;
if (rankingResult.size() == 0) {
expireTime = 5400000 + expireTimeFloatRandom.nextInt(1800000);
log.warn("数据获取失败, 将设置浮动有效时间以准备下次更新. (ExpireTime: {}ms)", expireTime);
}
result = new ArrayList<>(rankingResult).subList(start - 1, start + range - 1);
rankingCache.update(requestSign, rankingResult, expireTime);
log.trace("Ranking缓存更新完成.(RequestSign: {})", requestSign);
}
}
} finally {
locker.unlock();
}
}
if (Objects.isNull(result)) {
result = rankingCache.getCache(requestSign, start - 1, range);
log.trace("RequestSign [{}] 缓存命中.", requestSign);
}
return PixivDownload.getRanking(result, start - 1, range);
}
/**
* 获取搜索结果
* @param content 搜索内容
* @param type 类型
* @param area 范围
* @param includeKeywords 包含关键词
* @param excludeKeywords 排除关键词
* @param contentOption 内容类型
* @return 返回完整搜索结果
* @throws IOException 当请求发生异常, 或接口返回异常信息时抛出.
*/
public JsonObject getSearchBody(
String content,
String type,
String area,
String includeKeywords,
String excludeKeywords,
String contentOption,
int pageIndex
) throws IOException {
PixivSearchLinkBuilder searchBuilder = new PixivSearchLinkBuilder(Strings.isNullOrEmpty(content) ? "" : content);
if (type != null) {
try {
searchBuilder.setSearchType(PixivSearchLinkBuilder.SearchType.valueOf(type.toUpperCase()));
} catch (IllegalArgumentException e) {
log.warn("不支持的SearchType: {}", type);
}
}
if (area != null) {
try {
searchBuilder.setSearchArea(PixivSearchLinkBuilder.SearchArea.valueOf(area));
} catch (IllegalArgumentException e) {
log.warn("不支持的SearchArea: {}", area);
}
}
if (contentOption != null) {
try {
searchBuilder.setSearchContentOption(
PixivSearchLinkBuilder.SearchContentOption.valueOf(contentOption.trim().toUpperCase()));
} catch (IllegalArgumentException e) {
log.warn("不支持的SearchContentOption: {}", contentOption);
}
}
if (!Strings.isNullOrEmpty(includeKeywords)) {
for (String keyword : includeKeywords.split(";")) {
searchBuilder.removeExcludeKeyword(keyword.trim());
searchBuilder.addIncludeKeyword(keyword.trim());
log.trace("已添加关键字: {}", keyword);
}
}
if (!Strings.isNullOrEmpty(excludeKeywords)) {
for (String keyword : excludeKeywords.split(";")) {
searchBuilder.removeIncludeKeyword(keyword.trim());
searchBuilder.addExcludeKeyword(keyword.trim());
log.trace("已添加排除关键字: {}", keyword);
}
}
if(pageIndex > 0) {
searchBuilder.setPage(pageIndex);
}
log.debug("正在搜索作品, 条件: {}", searchBuilder.getSearchCondition());
Locker<String> locker
= buildSyncKey(searchBuilder.buildURL());
String requestUrl = locker.getKey();
log.debug("RequestUrl: {}", requestUrl);
JsonObject resultBody = null;
if(!searchBodyCache.exists(requestUrl)) {
try {
locker.lock();
synchronized (locker) {
if (!searchBodyCache.exists(requestUrl)) {
log.trace("searchBody缓存失效, 正在更新...");
JsonObject jsonObject;
HttpGet httpGetRequest = BotGlobal.getGlobal().getPixivDownload().
createHttpGetRequest(requestUrl);
HttpResponse response = BotGlobal.getGlobal().getPixivDownload().
getHttpClient().execute(httpGetRequest);
String responseBody = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
log.trace("ResponseBody: {}", responseBody);
jsonObject = BotGlobal.getGlobal().getGson().fromJson(responseBody, JsonObject.class);
if (jsonObject.get("error").getAsBoolean()) {
log.error("接口请求错误, 错误信息: {}", jsonObject.get("message").getAsString());
throw new HttpRequestException(response.getStatusLine(), responseBody);
}
long expire = 7200 * 1000;
String propValue = SettingProperties
.getProperty(SettingProperties.GLOBAL, "cache.searchBody.expire", "7200000");
try {
expire = Long.parseLong(propValue);
} catch (Exception e) {
log.warn("全局配置项 \"{}\" 值非法, 已使用默认值: {}", propValue, expire);
}
resultBody = jsonObject.getAsJsonObject().getAsJsonObject("body");
searchBodyCache.update(requestUrl, jsonObject, expire);
log.trace("searchBody缓存已更新(有效时间: {})", expire);
} else {
log.trace("搜索缓存命中.");
}
}
} finally {
locker.unlock();
}
} else {
log.trace("搜索缓存命中.");
}
if(Objects.isNull(resultBody)) {
resultBody = searchBodyCache.getCache(requestUrl).getAsJsonObject().getAsJsonObject("body");
}
return resultBody;
}
protected ImageChecksum getImageChecksum(int illustId, int pageIndex) {
String cacheKey = illustId + ":" + pageIndex;
if(!imageChecksumCache.exists(cacheKey)) {
return null;
} else {
return ImageChecksum.fromJsonObject(imageChecksumCache.getCache(cacheKey).getAsJsonObject());
}
}
protected void setImageChecksum(ImageChecksum checksum) {
String cacheKey = checksum.getIllustId() + ":" + checksum.getPage();
imageChecksumCache.update(cacheKey, ImageChecksum.toJsonObject(checksum), 0);
}
/**
* 合并String并存取到常量池, 以保证对象一致
* @param keys String对象
* @return 合并后, 如果常量池存在合并后的结果, 则返回常量池中的对象, 否则存入常量池后返回.
*/
private Locker<String> buildSyncKey(String... keys) {
StringBuilder sb = new StringBuilder();
for (String string : keys) {
sb.append(string);
}
return lockerMap.createLocker(sb.toString(), true);
}
/**
* 图片检验信息
*/
public static class ImageChecksum implements Serializable {
private final static MessageDigestUtils.Algorithm ALGORITHM = MessageDigestUtils.Algorithm.SHA256;
private ImageChecksum() {}
private int illustId;
private int page;
private String fileName;
private long size;
private byte[] checksum;
public long getSize() {
return size;
}
public void setSize(long size) {
this.size = size;
}
public byte[] getChecksum() {
return checksum;
}
public void setChecksum(byte[] checksum) {
this.checksum = checksum;
}
public int getIllustId() {
return illustId;
}
public void setIllustId(int illustId) {
this.illustId = illustId;
}
public String getFileName() {
return fileName;
}
public void setFileName(String fileName) {
this.fileName = fileName;
}
public int getPage() {
return page;
}
public void setPage(int page) {
this.page = page;
}
public static ImageChecksum buildImageChecksumFromStream(
int illustId, int pageIndex,
String fileName, InputStream imageStream) throws IOException {
ImageChecksum checksum = new ImageChecksum();
checksum.setIllustId(illustId);
checksum.setPage(pageIndex);
checksum.setFileName(fileName);
ByteArrayOutputStream bufferStream = new ByteArrayOutputStream();
checksum.setSize(IOUtils.copyLarge(imageStream, bufferStream));
checksum.setChecksum(
MessageDigestUtils.encrypt(bufferStream.toByteArray(), ALGORITHM));
return checksum;
}
/**
* 将图片检验信息转换成JsonObject
* @param checksum 检验信息对象
* @return 转换后的JsonObject对象
*/
public static JsonObject toJsonObject(ImageChecksum checksum) {
JsonObject result = new JsonObject();
result.addProperty("illustId", checksum.getIllustId());
result.addProperty("page", checksum.getPage());
result.addProperty("fileName", checksum.getFileName());
result.addProperty("size", checksum.getSize());
result.addProperty("checksum", Base64.getEncoder().encodeToString(checksum.getChecksum()));
return result;
}
/**
* 从JsonObject转换到图片检验信息
* @param checksumObject JsonObject对象
* @return 转换后的图片检验信息对象
*/
public static ImageChecksum fromJsonObject(JsonObject checksumObject) {
ImageChecksum checksum = new ImageChecksum();
checksum.setIllustId(checksumObject.get("illustId").getAsInt());
checksum.setPage(checksumObject.get("page").getAsInt());
checksum.setFileName(checksumObject.get("fileName").getAsString());
checksum.setSize(checksumObject.get("size").getAsLong());
checksum.setChecksum(Base64.getDecoder().decode(checksumObject.get("checksum").getAsString()));
return checksum;
}
/**
* 比对图片文件是否完整.
* @param checksum 图片检验信息
* @param imageData 图片数据
* @return 如果检验成功, 则返回true
*/
public static boolean checkFile(ImageChecksum checksum, byte[] imageData) {
byte[] sha256Checksum = MessageDigestUtils.encrypt(imageData, ALGORITHM);
return checksum.getSize() == imageData.length &&
Arrays.equals(checksum.getChecksum(), sha256Checksum);
}
@Override
public String toString() {
return "ImageChecksum{" +
"illustId=" + illustId +
", page=" + page +
", fileName='" + fileName + '\'' +
", size=" + size +
", checksum=" + Base64.getEncoder().encodeToString(getChecksum()) +
'}';
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
ImageChecksum checksum1 = (ImageChecksum) o;
return illustId == checksum1.illustId &&
page == checksum1.page &&
size == checksum1.size &&
Objects.equals(fileName, checksum1.fileName) &&
Arrays.equals(checksum, checksum1.checksum);
}
@Override
public int hashCode() {
int result = Objects.hash(illustId, page, fileName, size);
result = 31 * result + Arrays.hashCode(checksum);
return result;
}
}
}

View File

@ -0,0 +1,35 @@
package net.lamgc.cgj.bot.cache;
public final class CacheStoreUtils {
private CacheStoreUtils() {}
/**
* 将 CacheStore 转换成 基于 {@link LocalHashCacheStore} 的 {@link HotDataCacheStore}
* <p>通过该方法转换, 会自动启用 自动清理</p>
* @param cacheStore 上游缓存库
* @param expireTime 热点缓存最小有效期
* @param floatRange 缓存浮动最大范围
* @param <T> 缓存库数据类型
* @return 返回 {@link HotDataCacheStore}
*/
public static <T> CacheStore<T> hashLocalHotDataStore(CacheStore<T> cacheStore, long expireTime, int floatRange) {
return hashLocalHotDataStore(cacheStore, expireTime, floatRange, true);
}
/**
* 将 CacheStore 转换成 基于 {@link LocalHashCacheStore} 的 {@link HotDataCacheStore}
* @param cacheStore 上游缓存库
* @param expireTime 热点缓存最小有效期
* @param floatRange 缓存浮动最大范围
* @param autoClean 是否启用自动清理
* @param <T> 缓存库数据类型
* @return 返回 {@link HotDataCacheStore}
*/
public static <T> CacheStore<T> hashLocalHotDataStore(CacheStore<T> cacheStore,
long expireTime, int floatRange, boolean autoClean) {
return new HotDataCacheStore<>(cacheStore, new LocalHashCacheStore<>(), expireTime, floatRange, autoClean);
}
}

View File

@ -0,0 +1,13 @@
package net.lamgc.cgj.bot.cache;
/**
* 可清理接口, 实现该接口代表该类具有清理动作.
*/
public interface Cleanable {
/**
* 该方法需要CacheStore完成对过期Entry的清除.
*/
void clean() throws Exception;
}

View File

@ -3,40 +3,45 @@ package net.lamgc.cgj.bot.cache;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Date;
import java.util.Objects;
import java.util.Random;
import java.util.*;
/**
* 具有继承性的热点数据缓存库
* @param <T> 存储类型
* @author LamGC
*/
public class HotDataCacheStore<T> implements CacheStore<T> {
public class HotDataCacheStore<T> implements CacheStore<T>, Cleanable {
private final CacheStore<T> parent;
private final CacheStore<T> current;
private final long expireTime;
private final int expireFloatRange;
private final Random random = new Random();
private final Logger log = LoggerFactory.getLogger(HotDataCacheStore.class.getSimpleName() + "@" + Integer.toHexString(this.hashCode()));
private final Logger log = LoggerFactory.getLogger(this.toString());
/**
* 构造热点缓存存储对象
* @param parent 上级缓存存储库
* @param current 热点缓存存储库, 最好使用本地缓存(例如 {@linkplain LocalHashCacheStore LocalHashCacheStore})
* @param expireTime 本地缓存库的缓存项过期时间, 单位毫秒;
* 该时间并不是所有缓存项的最终过期时间, 还需要根据expireFloatRange的设定随机设置, 公式:
* {@code expireTime + new Random().nextInt(expireFloatRange)}
* @param expireFloatRange 过期时间的浮动范围(单位毫秒), 用于防止短时间内大量缓存项失效导致的缓存雪崩
* 该时间并不是所有缓存项的最终过期时间, 还需要根据expireFloatRange的设定随机设置, 公式:
* {@code expireTime + new Random().nextInt(expireFloatRange)}
* @param expireFloatRange 过期时间的浮动范围(单位毫秒), 用于防止短时间内大量缓存项失效导致的缓存雪崩,
* 如设置为0或负数, 则不启用浮动范围.
* @param autoClean 是否交由{@link AutoCleanTimer}自动执行清理, 启用后, AutoCleanTimer会自动检查过期Key并进行删除.
*/
public HotDataCacheStore(CacheStore<T> parent, CacheStore<T> current, long expireTime, int expireFloatRange) {
public HotDataCacheStore(CacheStore<T> parent, CacheStore<T> current, long expireTime, int expireFloatRange, boolean autoClean) {
this.parent = parent;
this.current = current;
this.expireTime = expireTime;
this.expireFloatRange = expireFloatRange;
log.debug("HotDataCacheStore初始化完成. (Parent: {}, Current: {}, expireTime: {}, expireFloatRange: {})",
parent, current, expireTime, expireFloatRange);
if(autoClean) {
AutoCleanTimer.add(this);
}
log.trace("HotDataCacheStore初始化完成. " +
"(Parent: {}, Current: {}, expireTime: {}, expireFloatRange: {}, autoClean: {})",
parent, current, expireTime, expireFloatRange, autoClean);
}
@Override
@ -53,23 +58,27 @@ public class HotDataCacheStore<T> implements CacheStore<T> {
@Override
public T getCache(String key) {
if(!exists(key)) {
log.debug("查询缓存键名不存在, 直接返回null.");
log.trace("查询缓存键名不存在, 直接返回null.");
return null;
}
T result = current.getCache(key);
if(Objects.isNull(result)) {
log.debug("Current缓存库未命中, 查询Parent缓存库");
log.trace("Current缓存库未命中, 查询Parent缓存库");
T parentResult = parent.getCache(key);
if(Objects.isNull(parentResult)) {
log.debug("Parent缓存库未命中, 缓存不存在");
log.trace("Parent缓存库未命中, 缓存不存在");
return null;
}
log.debug("Parent缓存命中, 正在更新Current缓存库...");
current.update(key, parentResult, expireTime + random.nextInt(expireFloatRange));
log.debug("Current缓存库更新完成.");
log.trace("Parent缓存命中, 正在更新Current缓存库...");
current.update(key, parentResult,
expireTime + (expireFloatRange <= 0 ? 0 : random.nextInt(expireFloatRange)));
log.trace("Current缓存库更新完成.");
result = parentResult;
} else {
log.debug("Current缓存库缓存命中.");
// 更新该Key的过期时间
current.update(key, result,
expireTime + (expireFloatRange <= 0 ? 0 : random.nextInt(expireFloatRange)));
log.trace("Current缓存库缓存命中.");
}
return result;
}
@ -99,13 +108,48 @@ public class HotDataCacheStore<T> implements CacheStore<T> {
return current.clear();
}
@Override
public Set<String> keys() {
Set<String> keys = new HashSet<>();
keys.addAll(current.keys());
keys.addAll(parent.keys());
return keys;
}
@Override
public boolean remove(String key) {
parent.remove(key);
current.remove(key);
return true;
}
@Override
public boolean supportedPersistence() {
// 由于Current的缓存数据会更新到Parent上,
// 所以只要任意一边支持持久化, 那么该缓存库就支持持久化
return current.supportedPersistence() || parent.supportedPersistence();
}
@Override
public boolean supportedList() {
return false;
// 只有两边都支持List, 该缓存库才会支持持久化
return current.supportedList() && parent.supportedList();
}
/**
* 检查并清理已过期的Entry.
* <p>该方法仅清理Current缓存库, 不会对上游缓存库造成影响.</p>
*/
@Override
public void clean() throws Exception {
if(current instanceof Cleanable) {
((Cleanable) current).clean();
} else {
for(String key : this.current.keys()) {
if (!current.exists(key)) {
current.remove(key);
}
}
}
}
}

View File

@ -1,52 +1,54 @@
package net.lamgc.cgj.bot.cache;
import net.lamgc.cgj.Main;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.exception.HttpRequestException;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.cgj.util.URLs;
import net.lamgc.utils.event.EventHandler;
import org.apache.commons.io.IOUtils;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.tomcat.util.http.fileupload.util.Streams;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.*;
import java.util.Collections;
import java.util.HashSet;
import java.util.Set;
public class ImageCacheHandler implements EventHandler {
private final static Logger log = LoggerFactory.getLogger("ImageCacheHandler");
private final static Logger log = LoggerFactory.getLogger(ImageCacheHandler.class);
private final static HttpClient httpClient = HttpClientBuilder.create().setProxy(Main.proxy).build();
private final static HttpClient httpClient = HttpClientBuilder.create()
.setProxy(BotGlobal.getGlobal().getProxy())
.build();
private final static Set<ImageCacheObject> cacheQueue = Collections.synchronizedSet(new HashSet<>());
public void getImageToCache(ImageCacheObject event) {
@SuppressWarnings("unused")
public void getImageToCache(ImageCacheObject event) throws Exception {
if(cacheQueue.contains(event)) {
log.info("图片 {} 已存在相同缓存任务, 跳过.", event.getStoreFile().getName());
log.warn("图片 {} 已存在相同缓存任务, 跳过.", event.getStoreFile().getName());
return;
} else {
cacheQueue.add(event);
}
try {
log.info("图片 {} Event正在进行...({})", event.getStoreFile().getName(), Integer.toHexString(event.hashCode()));
log.debug("图片 {} Event正在进行...({})", event.getStoreFile().getName(), Integer.toHexString(event.hashCode()));
File storeFile = event.getStoreFile();
log.info("正在缓存图片 {} (Path: {})", storeFile.getName(), storeFile.getAbsolutePath());
log.debug("正在缓存图片 {} (Path: {})", storeFile.getName(), storeFile.getAbsolutePath());
try {
if(!storeFile.exists() && !storeFile.createNewFile()) {
log.error("无法创建文件(Path: {})", storeFile.getAbsolutePath());
return;
throw new IOException("Failed to create file");
}
} catch (IOException e) {
log.error("无法创建文件(Path: {})", storeFile.getAbsolutePath());
e.printStackTrace();
throw e;
}
HttpGet request = new HttpGet(event.getDownloadLink());
@ -56,23 +58,36 @@ public class ImageCacheHandler implements EventHandler {
response = httpClient.execute(request);
} catch (IOException e) {
log.error("Http请求时发生异常", e);
return;
throw e;
}
if(response.getStatusLine().getStatusCode() != 200) {
log.warn("Http请求异常{}", response.getStatusLine());
return;
HttpRequestException requestException = new HttpRequestException(response);
log.warn("Http请求异常{}", requestException.getStatusLine());
throw requestException;
}
log.info("正在下载...(Content-Length: {}KB)", response.getEntity().getContentLength() / 1024);
try(FileOutputStream fos = new FileOutputStream(storeFile)) {
IOUtils.copy(response.getEntity().getContent(), fos);
log.trace("正在下载...(Content-Length: {}KB)", response.getEntity().getContentLength() / 1024);
ByteArrayOutputStream bufferOutputStream = new ByteArrayOutputStream();
try(FileOutputStream fileOutputStream = new FileOutputStream(storeFile)) {
Streams.copy(response.getEntity().getContent(), bufferOutputStream, false);
ByteArrayInputStream bufferInputStream = new ByteArrayInputStream(bufferOutputStream.toByteArray());
CacheStoreCentral.ImageChecksum imageChecksum = CacheStoreCentral.ImageChecksum
.buildImageChecksumFromStream(
event.getIllustId(),
event.getPageIndex(),
event.getStoreFile().getName(),
bufferInputStream
);
bufferInputStream.reset();
Streams.copy(bufferInputStream, fileOutputStream, false);
CacheStoreCentral.getCentral().setImageChecksum(imageChecksum);
} catch (IOException e) {
log.error("下载图片时发生异常", e);
return;
throw e;
}
event.getImageCache().put(URLs.getResourceName(event.getDownloadLink()), storeFile);
} finally {
log.info("图片 {} Event结束({})", event.getStoreFile().getName(), Integer.toHexString(event.hashCode()));
log.debug("图片 {} Event结束({})", event.getStoreFile().getName(), Integer.toHexString(event.hashCode()));
cacheQueue.remove(event);
}
}

View File

@ -12,13 +12,16 @@ public class ImageCacheObject implements EventObject {
private final int illustId;
private final int pageIndex;
private final String downloadLink;
private final File storeFile;
public ImageCacheObject(Map<String, File> imageCache, int illustId, String downloadLink, File storeFile) {
public ImageCacheObject(Map<String, File> imageCache, int illustId, int pageIndex, String downloadLink, File storeFile) {
this.imageCache = imageCache;
this.illustId = illustId;
this.pageIndex = pageIndex;
this.downloadLink = downloadLink;
this.storeFile = storeFile;
}
@ -39,12 +42,17 @@ public class ImageCacheObject implements EventObject {
return illustId;
}
public int getPageIndex() {
return pageIndex;
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
ImageCacheObject that = (ImageCacheObject) o;
return illustId == that.illustId &&
pageIndex == that.pageIndex &&
Objects.equals(imageCache, that.imageCache) &&
Objects.equals(downloadLink, that.downloadLink) &&
Objects.equals(storeFile, that.storeFile);
@ -52,13 +60,15 @@ public class ImageCacheObject implements EventObject {
@Override
public int hashCode() {
return Objects.hash(imageCache, illustId, downloadLink, storeFile);
return Objects.hash(imageCache, illustId, pageIndex, downloadLink, storeFile);
}
@Override
public String toString() {
return "ImageCacheObject@" + Integer.toHexString(hashCode()) + "{" +
"illustId=" + illustId +
return "ImageCacheObject{" +
"imageCache=" + imageCache +
", illustId=" + illustId +
", pageIndex=" + pageIndex +
", downloadLink='" + downloadLink + '\'' +
", storeFile=" + storeFile +
'}';

View File

@ -0,0 +1,142 @@
package net.lamgc.cgj.bot.cache;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import net.lamgc.cgj.exception.HttpRequestException;
import java.util.Hashtable;
import java.util.Map;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicReference;
import java.util.concurrent.locks.ReentrantLock;
public final class ImageCacheStore {
private final static Logger log = LoggerFactory.getLogger(ImageCacheStore.class);
private final static Map<ImageCacheObject, Task> cacheMap = new Hashtable<>();
private final static ThreadPoolExecutor imageCacheExecutor = new ThreadPoolExecutor(
4, 6,
30L,
TimeUnit.SECONDS,
new LinkedBlockingQueue<>(100),
new ThreadFactoryBuilder()
.setNameFormat("ImageCacheThread-%d")
.build()
);
private final static ImageCacheHandler handler = new ImageCacheHandler();
static {
Thread shutdownThread = new Thread(imageCacheExecutor::shutdownNow);
shutdownThread.setName("Thread-ImageCacheShutdown");
Runtime.getRuntime().addShutdownHook(shutdownThread);
}
private ImageCacheStore() {}
/**
* 传递图片缓存任务, 并等待缓存完成.
* @param cacheObject 缓存任务组
*/
public static Throwable executeCacheRequest(ImageCacheObject cacheObject) throws InterruptedException {
Task task = getTaskState(cacheObject);
if(task.taskState.get() == TaskState.COMPLETE) {
return null;
}
boolean locked = false;
try {
if(task.taskState.get() == TaskState.COMPLETE) {
return null;
}
task.lock.lock();
locked = true;
// 双重检查
if(task.taskState.get() == TaskState.COMPLETE) {
return null;
}
// 置任务状态
task.taskState.set(TaskState.RUNNING);
Future<Throwable> future = imageCacheExecutor.submit(() -> {
try {
handler.getImageToCache(cacheObject);
} catch (Throwable e) {
return e;
}
return null;
});
Throwable throwable;
try {
throwable = future.get();
if(throwable == null) {
task.taskState.set(TaskState.COMPLETE);
} else {
task.taskState.set(TaskState.ERROR);
}
} catch (ExecutionException e) {
log.error("执行图片缓存任务时发生异常", e);
task.taskState.set(TaskState.ERROR);
return e.getCause();
} catch (InterruptedException e) {
future.cancel(true);
task.taskState.set(TaskState.ERROR);
throw e;
}
return throwable;
} finally {
if(locked) {
task.lock.unlock();
}
}
}
private static Task getTaskState(ImageCacheObject cacheObject) {
if(!cacheMap.containsKey(cacheObject)) {
cacheMap.put(cacheObject, new Task());
}
return cacheMap.get(cacheObject);
}
/**
* 获取错误信息
*/
public static String getErrorMessageFromThrowable(Throwable throwable, boolean onlyRequestException) {
if(throwable == null) {
return "";
} else if(!(throwable instanceof HttpRequestException)) {
if(onlyRequestException) {
return "";
}
return throwable.getMessage();
}
JsonObject result = new Gson()
.fromJson(((HttpRequestException) throwable).getContent(), JsonObject.class);
return result.get("msg").getAsString();
}
/**
* 任务状态
*/
private enum TaskState {
READY, RUNNING, COMPLETE, ERROR
}
private static class Task {
public final ReentrantLock lock = new ReentrantLock(true);
public final AtomicReference<TaskState> taskState = new AtomicReference<>(TaskState.READY);
}
}

View File

@ -2,24 +2,52 @@ package net.lamgc.cgj.bot.cache;
import org.jetbrains.annotations.NotNull;
import java.util.Date;
import java.util.Hashtable;
import java.util.Objects;
import java.util.*;
import java.util.concurrent.atomic.AtomicReference;
public class LocalHashCacheStore<T> implements CacheStore<T> {
/**
* 基于Hashtable的本地缓存库
* @param <T> 缓存类型
*/
public class LocalHashCacheStore<T> implements CacheStore<T>, Cleanable {
private final Hashtable<String, CacheObject<T>> cache;
/**
* 构造一个基于Hashtable的本地缓存库
* @see Hashtable
*/
public LocalHashCacheStore() {
this(0);
}
/**
* 构造一个基于Hashtable的本地缓存库
* @param initialCapacity 初始容量
* @see Hashtable
*/
public LocalHashCacheStore(int initialCapacity) {
this(initialCapacity, 0F);
}
/**
* 构造一个基于Hashtable的本地缓存库
* @param initialCapacity 初始容量
* @param loadFactor 重载因子
* @see Hashtable
*/
public LocalHashCacheStore(int initialCapacity, float loadFactor) {
this(initialCapacity, loadFactor, false);
}
/**
* 构造一个基于Hashtable的本地缓存库
* @param initialCapacity 初始容量
* @param loadFactor 重载因子
* @param autoClean 是否自动清理
* @see Hashtable
*/
public LocalHashCacheStore(int initialCapacity, float loadFactor, boolean autoClean) {
if(initialCapacity != 0) {
if(loadFactor <= 0F) {
cache = new Hashtable<>(initialCapacity);
@ -29,6 +57,10 @@ public class LocalHashCacheStore<T> implements CacheStore<T> {
} else {
cache = new Hashtable<>();
}
if(autoClean) {
AutoCleanTimer.add(this);
}
}
@Override
@ -93,6 +125,16 @@ public class LocalHashCacheStore<T> implements CacheStore<T> {
return true;
}
@Override
public Set<String> keys() {
return cache.keySet();
}
@Override
public boolean remove(String key) {
return cache.remove(key) != null;
}
@Override
public boolean supportedPersistence() {
return false;
@ -103,11 +145,23 @@ public class LocalHashCacheStore<T> implements CacheStore<T> {
return false;
}
@Override
public void clean() {
Date currentDate = new Date();
Set<String> expireKeySet = new HashSet<>();
cache.forEach((key, value) -> {
if(value.isExpire(currentDate)) {
expireKeySet.add(key);
}
});
public static class CacheObject<T> implements Comparable<CacheObject<T>> {
expireKeySet.forEach(cache::remove);
}
private AtomicReference<T> value;
private AtomicReference<Date> expire;
private static class CacheObject<T> implements Comparable<CacheObject<T>> {
private final AtomicReference<T> value;
private final AtomicReference<Date> expire;
public CacheObject(T value, Date expire) {
this.value = new AtomicReference<>(value);

View File

@ -9,10 +9,11 @@ import redis.clients.jedis.*;
import java.net.URI;
import java.util.Date;
import java.util.Objects;
import java.util.Set;
import java.util.function.Consumer;
import java.util.function.Function;
public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
private final JedisPool jedisPool;
private final String keyPrefix;
@ -35,7 +36,7 @@ public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
if(jedisPool.isClosed()) {
throw new IllegalStateException("JedisPool is closed");
}
log = LoggerFactory.getLogger(this.getClass().getSimpleName() + "@" + Integer.toHexString(jedisPool.hashCode()));
log = LoggerFactory.getLogger(this.getClass().getName() + "@" + Integer.toHexString(jedisPool.hashCode()));
if(!Strings.isNullOrEmpty(keyPrefix)) {
this.keyPrefix = keyPrefix.endsWith(".") ? keyPrefix : keyPrefix + ".";
} else {
@ -50,29 +51,27 @@ public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
@Override
public void update(String key, T value, Date expire) {
Jedis jedis = jedisPool.getResource();
jedis.set(keyPrefix + key, parse(value));
if(expire != null) {
jedis.pexpireAt(keyPrefix + key, expire.getTime());
log.debug("已设置Key {} 的过期时间(Expire: {})", key, expire.getTime());
}
jedis.close();
executeJedisCommand(jedis -> {
jedis.set(keyPrefix + key, parse(value));
if(expire != null) {
jedis.pexpireAt(keyPrefix + key, expire.getTime());
log.debug("已设置Key {} 的过期时间(Expire: {})", key, expire.getTime());
}
});
}
@Override
public T getCache(String key) {
Jedis jedis = jedisPool.getResource();
T result = analysis(jedis.get(keyPrefix + key));
jedis.close();
return result;
return executeJedisCommand(jedis -> {
return analysis(jedis.get(keyPrefix + key));
});
}
@Override
public boolean exists(String key) {
Jedis jedis = jedisPool.getResource();
boolean result = jedis.exists(keyPrefix + key);
jedis.close();
return result;
return executeJedisCommand(jedis -> {
return jedis.exists(keyPrefix + key);
});
}
@Override
@ -82,11 +81,21 @@ public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
@Override
public boolean clear() {
Jedis jedis = jedisPool.getResource();
String result = jedis.flushDB();
jedis.close();
log.info("flushDB返回结果: {}", result);
return true;
return executeJedisCommand(jedis -> {
return jedis.flushDB().equalsIgnoreCase("ok");
});
}
@Override
public Set<String> keys() {
return executeJedisCommand(jedis -> {
return jedis.keys(keyPrefix + "*");
});
}
@Override
public boolean remove(String key) {
return executeJedisCommand(jedis -> jedis.del(keyPrefix + key) == 1);
}
/**
@ -145,13 +154,4 @@ public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
return false;
}
/**
* 替换原本的分隔符(.)为(:).<br/>
* 即将启用
* @param key 要处理的键名
* @return 处理后的键名
*/
public static String replaceKey(String key) {
return key.replaceAll("\\.", ":");
}
}

View File

@ -0,0 +1,30 @@
package net.lamgc.cgj.bot.cache;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.JedisPoolConfig;
import java.net.URI;
public class StringListRedisCacheStore extends RedisListCacheStore<String> {
public StringListRedisCacheStore(URI redisServerUri, String prefix) {
super(redisServerUri, prefix);
}
public StringListRedisCacheStore(URI redisServerUri, JedisPoolConfig config, int timeout, String password, String prefix) {
super(redisServerUri, config, timeout, password, prefix);
}
public StringListRedisCacheStore(JedisPool pool, String prefix) {
super(pool, prefix);
}
@Override
public String parseData(String dataObj) {
return dataObj;
}
@Override
public String analysisData(String str) {
return str;
}
}

View File

@ -5,30 +5,26 @@ import com.google.common.base.Throwables;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import net.lamgc.cgj.bot.BotAdminCommandProcess;
import net.lamgc.cgj.bot.BotCommandProcess;
import net.lamgc.cgj.bot.MessageEventExecutionDebugger;
import net.lamgc.cgj.bot.SettingProperties;
import net.lamgc.cgj.util.DateParser;
import net.lamgc.cgj.util.PagesQualityParser;
import net.lamgc.cgj.util.TimeLimitThreadPoolExecutor;
import net.lamgc.utils.base.runner.ArgumentsRunner;
import net.lamgc.utils.base.runner.ArgumentsRunnerConfig;
import net.lamgc.utils.base.runner.exception.DeveloperRunnerException;
import net.lamgc.utils.base.runner.exception.NoSuchCommandException;
import net.lamgc.utils.base.runner.exception.ParameterNoFoundException;
import net.lamgc.utils.event.EventExecutor;
import net.lamgc.utils.event.EventHandler;
import net.lamgc.utils.event.*;
import net.lamgc.utils.event.EventObject;
import net.lamgc.utils.event.EventUncaughtExceptionHandler;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.JedisPool;
import java.lang.reflect.Method;
import java.net.URI;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Date;
import java.util.Objects;
import java.util.*;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
@ -36,26 +32,22 @@ import java.util.regex.Pattern;
public class BotEventHandler implements EventHandler {
public final static String COMMAND_PREFIX = ".cgj";
public final static String ADMIN_COMMAND_PREFIX = ".cgjadmin ";
private final ArgumentsRunner processRunner;
private final ArgumentsRunner adminRunner;
private final Logger log = LoggerFactory.getLogger("BotEventHandler@" + Integer.toHexString(this.hashCode()));
/**
* 所有缓存共用的JedisPool
*/
private final static URI redisServerUri = URI.create("redis://" + System.getProperty("cgj.redisAddress"));
public final static JedisPool redisServer = new JedisPool(redisServerUri.getHost(), redisServerUri.getPort() == -1 ? 6379 : redisServerUri.getPort());
private final static Logger log = LoggerFactory.getLogger(BotEventHandler.class);
/**
* 消息事件执行器
*/
public final static EventExecutor executor = new EventExecutor(new ThreadPoolExecutor(
(int) Math.ceil(Runtime.getRuntime().availableProcessors() / 2F),
Runtime.getRuntime().availableProcessors(),
private final static EventExecutor executor = new EventExecutor(new TimeLimitThreadPoolExecutor(
180000, // 3min limit
Math.max(Runtime.getRuntime().availableProcessors(), 4), // 4 ~ processors
Math.min(Math.max(Runtime.getRuntime().availableProcessors() * 2, 8), 32),// (8 ~ processors * 2) ~ 32
30L,
TimeUnit.SECONDS,
TimeUnit.SECONDS,
new LinkedBlockingQueue<>(1536),
new ThreadFactoryBuilder()
.setNameFormat("CommandProcess-%d")
@ -63,23 +55,26 @@ public class BotEventHandler implements EventHandler {
));
private static boolean initialled = false;
static {
initial();
}
/**
* 初始化BotEventHandler
*/
public synchronized static void initial() {
if(initialled) {
Logger logger = LoggerFactory.getLogger("BotEventHandler@<init>");
logger.warn("BotEventHandler已经执行过初始化方法, 可能存在多次执行的问题, 堆栈信息: \n {}",
log.warn("BotEventHandler已经执行过初始化方法, 可能存在多次执行的问题, 堆栈信息: \n {}",
Throwables.getStackTraceAsString(new Exception()));
return;
}
executor.setEnableEventResend(true);
executor.setEventUncaughtExceptionHandler(new EventUncaughtExceptionHandler() {
private final Logger log = LoggerFactory.getLogger("EventUncaughtExceptionHandler");
private final Logger log = LoggerFactory.getLogger(this.getClass());
@Override
public void exceptionHandler(Thread executeThread, EventHandler handler, Method handlerMethod, EventObject event, Throwable cause) {
log.error("发生未捕获异常:\nThread:{}, EventHandler: {}, HandlerMethod: {}, EventObject: {}\n{}",
log.error("EventExecutor@{} 发生未捕获异常:\n\t" +
"Thread:{}\n\tEventHandler: {}\n\tHandlerMethod: {}\n\tEventObject: {}\n" +
"------------------ Stack Trace ------------------\n{}",
executor.hashCode(),
executeThread.getName(),
handler.toString(),
handlerMethod.getName(),
@ -89,8 +84,17 @@ public class BotEventHandler implements EventHandler {
});
try {
executor.addHandler(new BotEventHandler());
Thread shutdownThread = new Thread(() -> executor.shutdown(true));
shutdownThread.setName("Thread-EventHandlerShutdown");
Runtime.getRuntime().addShutdownHook(shutdownThread);
} catch (IllegalAccessException e) {
LoggerFactory.getLogger("BotEventHandler@Static").error("添加Handler时发生异常", e);
log.error("添加Handler时发生异常", e);
}
try {
BotAdminCommandProcess.loadPushList();
} catch(Throwable e) {
log.error("加载推送列表失败", e);
}
initialled = true;
}
@ -102,24 +106,69 @@ public class BotEventHandler implements EventHandler {
runnerConfig.addStringParameterParser(new DateParser(new SimpleDateFormat("yyyy-MM-dd")));
runnerConfig.addStringParameterParser(new PagesQualityParser());
log.debug("DateParser添加情况: {}", runnerConfig.hasStringParameterParser(Date.class));
processRunner = new ArgumentsRunner(BotCommandProcess.class, runnerConfig);
adminRunner = new ArgumentsRunner(BotAdminCommandProcess.class, runnerConfig);
BotCommandProcess.initialize();
}
/**
* 投递消息事件
* @param event 事件对象
*/
@NotAccepted
public static void executeMessageEvent(MessageEvent event) {
try {
executeMessageEvent(event, false);
} catch (InterruptedException e) {
log.error("执行时发生异常", e);
throw new RuntimeException(e);
}
}
/**
* 投递消息事件
* @param event 事件对象
* @param sync 是否同步执行事件
*/
@NotAccepted
public static void executeMessageEvent(MessageEvent event, boolean sync) throws InterruptedException {
String debuggerName = SettingProperties.getProperty(0, "debug.debugger");
if(!event.getMessage().startsWith(ADMIN_COMMAND_PREFIX) &&
!Strings.isNullOrEmpty(debuggerName)) {
try {
MessageEventExecutionDebugger debugger = MessageEventExecutionDebugger.valueOf(debuggerName.toUpperCase());
debugger.debugger.accept(executor, event, SettingProperties.getProperties(SettingProperties.GLOBAL),
MessageEventExecutionDebugger.getDebuggerLogger(debugger));
} catch(IllegalArgumentException e) {
log.warn("未找到指定调试器: '{}'", debuggerName);
} catch (Exception e) {
log.error("事件调试处理时发生异常", e);
}
} else {
if(sync) {
BotEventHandler.executor.executorSync(event);
} else {
BotEventHandler.executor.executor(event);
}
}
}
/**
* 以事件形式处理消息事件
* @param event 消息事件对象
*/
@SuppressWarnings("unused")
public void processMessage(MessageEvent event) {
String msg = event.getMessage();
log.debug(event.toString());
if(!match(msg)) {
if(mismatch(msg)) {
return;
}
Pattern pattern = Pattern.compile("/\\s*(\".+?\"|[^:\\s])+((\\s*:\\s*(\".+?\"|[^\\s])+)|)|(\".+?\"|[^\"\\s])+");
Matcher matcher = pattern.matcher(Strings.nullToEmpty(msg));
ArrayList<String> argsList = new ArrayList<>();
List<String> argsList = new ArrayList<>();
while (matcher.find()) {
String arg = matcher.group();
int startIndex = 0;
@ -141,17 +190,23 @@ public class BotEventHandler implements EventHandler {
String[] args = new String[argsList.size()];
argsList.toArray(args);
log.debug("传入参数: {}", Arrays.toString(args));
argsList.add("-$fromGroup");
argsList.add(String.valueOf(event.getFromGroup()));
argsList.add("-$fromQQ");
argsList.add(String.valueOf(event.getFromQQ()));
args = Arrays.copyOf(args, args.length + 4);
argsList.toArray(args);
log.info("正在处理命令...");
long time = System.currentTimeMillis();
Object result;
try {
if(msg.toLowerCase().startsWith(COMMAND_PREFIX + "admin")) {
if(!String.valueOf(event.getFromQQ()).equals(BotCommandProcess.globalProp.getProperty("admin.adminId"))) {
event.sendMessage("你没有执行该命令的权限!");
return;
if(msg.toLowerCase().startsWith(ADMIN_COMMAND_PREFIX)) {
if(!String.valueOf(event.getFromQQ())
.equals(SettingProperties.getProperty(0, "admin.adminId"))) {
result = "你没有执行该命令的权限!";
} else {
result = adminRunner.run(new BotAdminCommandProcess(), args.length <= 1 ? new String[0] : Arrays.copyOfRange(args, 1, args.length));
result = adminRunner.run(args.length <= 1 ? new String[0] : Arrays.copyOfRange(args, 1, args.length));
}
} else {
result = processRunner.run(args.length <= 1 ? new String[0] : Arrays.copyOfRange(args, 1, args.length));
@ -161,18 +216,37 @@ public class BotEventHandler implements EventHandler {
} catch(ParameterNoFoundException e) {
result = "命令缺少参数: " + e.getParameterName();
} catch(DeveloperRunnerException e) {
log.error("执行命令时发生异常", e);
result = "命令执行时发生错误,无法完成!";
Throwable cause = e.getCause();
if (cause instanceof InterruptedException) {
log.error("命令执行超时, 终止执行.", cause);
result = "色图姬查阅图库太久,被赶出来了!";
} else if(cause instanceof NoSuchElementException && cause.getMessage().startsWith("No work found: ")) {
String message = cause.getMessage();
log.error("指定作品不存在.(Id: {})", message.substring(message.lastIndexOf(": ") + 2));
result = "色图姬找不到这个作品!";
} else {
log.error("执行命令时发生异常", e);
result = "色图姬在执行命令时遇到了一个错误!";
}
}
log.info("命令处理完成.(耗时: {}ms)", System.currentTimeMillis() - time);
if(Objects.requireNonNull(result) instanceof String) {
long processTime = System.currentTimeMillis() - time;
if(!Objects.isNull(result) && result instanceof String) {
try {
event.sendMessage((String) result);
int sendResult = event.sendMessage((String) result);
if (sendResult < 0) {
log.warn("消息发送失败, Sender {} 返回错误代码: {}", event.getClass().getName(), sendResult);
}
} catch(InterruptedException e) {
log.info("事件在发送消息时超时, 重新投递该事件.(Event: {})", event);
EventExecutor.resendCurrentEvent();
} catch (Exception e) {
log.error("发送消息时发生异常", e);
}
}
log.info("命令反馈完成.(耗时: {}ms)", System.currentTimeMillis() - time);
long totalTime = System.currentTimeMillis() - time;
log.info("命令反馈完成.(事件耗时: {}ms, P: {}%({}ms), R: {}%({}ms))", totalTime,
String.format("%.3f", ((double) processTime / (double)totalTime) * 100F), processTime,
String.format("%.3f", ((double) (totalTime - processTime) / (double)totalTime) * 100F), totalTime - processTime);
}
/**
@ -180,8 +254,8 @@ public class BotEventHandler implements EventHandler {
* @param message 要检查的消息
* @return 如果为true则提交
*/
public static boolean match(String message) {
return message.startsWith(COMMAND_PREFIX);
public static boolean mismatch(String message) {
return !message.startsWith(COMMAND_PREFIX) && !message.startsWith(ADMIN_COMMAND_PREFIX);
}
}

View File

@ -0,0 +1,23 @@
package net.lamgc.cgj.bot.event;
import net.lamgc.cgj.bot.message.MessageSender;
public class BufferedMessageSender implements MessageSender {
private final StringBuffer buffer = new StringBuffer();
@Override
public int sendMessage(String message) {
buffer.append(message);
return 0;
}
/**
* 从缓冲区中取出消息内容.
* @return 返回事件发送的消息内容.
*/
public String getBufferContent() {
return buffer.toString();
}
}

View File

@ -1,8 +1,9 @@
package net.lamgc.cgj.bot.event;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.utils.event.EventObject;
public abstract class MessageEvent implements EventObject {
public abstract class MessageEvent implements EventObject, MessageSender {
private final long fromGroup;
private final long fromQQ;
@ -11,15 +12,17 @@ public abstract class MessageEvent implements EventObject {
public MessageEvent(long fromGroup, long fromQQ, String message) {
this.fromGroup = fromGroup;
this.fromQQ = fromQQ;
this.message = message;
this.message = message.trim();
}
/**
* 发送消息
* @param message 消息内容
* @return 成功返回MessageId, 如没有MessageId则返回0, 失败返回负数错误码
* @throws Exception 该方法根据不同实现, 可能会抛出不同异常, 详见实现所标识的文档内容.
*/
public abstract int sendMessage(final String message);
@Override
public abstract int sendMessage(final String message) throws Exception;
/**
* 获取图片下载地址.
@ -28,12 +31,6 @@ public abstract class MessageEvent implements EventObject {
*/
public abstract String getImageUrl(String image);
/**
* 获取原始消息对象.(不推荐使用)
* @return 消息对象
*/
public abstract Object getRawMessage();
/**
* 获取来源群组号
* @return 如非群组消息, 返回0
@ -60,7 +57,7 @@ public abstract class MessageEvent implements EventObject {
@Override
public String toString() {
return "MessageEvent{" +
return this.getClass().getSimpleName() + "@" + Integer.toHexString(this.hashCode()) + "{" +
"fromGroup=" + getFromGroup() +
", fromQQ=" + getFromQQ() +
", message='" + getMessage() + '\'' +

View File

@ -0,0 +1,44 @@
package net.lamgc.cgj.bot.event;
/**
* 假负载消息事件, 一般用于预处理某个命令使用,可以增强在高峰期来临时的处理速度.
*/
public class VirtualLoadMessageEvent extends MessageEvent {
/**
* 将任意消息事件转换为假负载消息事件.
* <p>转换之后, 除了fromGroup, fromQQ, message外其他信息不会保留</p>
* @param event 待转换的消息事件
* @param inheritImpl 是否继承除 sendMessage 外的其他 MessageEvent 实现
* @return 转换后的消息事件
*/
public static VirtualLoadMessageEvent toVirtualLoadMessageEvent(MessageEvent event, boolean inheritImpl) {
if(event instanceof VirtualLoadMessageEvent) {
return (VirtualLoadMessageEvent) event;
} else if(!inheritImpl) {
return new VirtualLoadMessageEvent(event.getFromGroup(), event.getFromQQ(), event.getMessage());
} else {
return new VirtualLoadMessageEvent(event.getFromGroup(), event.getFromQQ(), event.getMessage()) {
@Override
public String getImageUrl(String image) {
return event.getImageUrl(image);
}
};
}
}
public VirtualLoadMessageEvent(long fromGroup, long fromQQ, String message) {
super(fromGroup, fromQQ, message);
}
@Override
public int sendMessage(String message) {
return 0;
}
@Override
public String getImageUrl(String image) {
return null;
}
}

View File

@ -0,0 +1,41 @@
package net.lamgc.cgj.bot.framework;
public interface Framework {
/**
* 框架初始化方法
* @param resources 框架所分配到的资源.
* @throws Exception 当框架抛出异常时, 将不会继续运行框架.
* @see FrameworkResources
*/
void init(FrameworkResources resources) throws Exception;
/**
* 框架运行方法
* @throws Exception 当框架抛出异常时, 将会终止框架的所有活动.
*/
void run() throws Exception;
/**
* 关闭框架
* @throws Exception 即使该方法抛出异常, {@link FrameworkManager}依然会尝试向框架所属的线程发起中断, 以试图清除框架资源.
*/
void close() throws Exception;
/**
* 获取框架标识名.
* <p>可根据需要自行调整框架标识名.</p>
* @return 返回标识名.
*/
default String getIdentify() {
return this.getClass().getSimpleName() + "@" + Integer.toHexString(this.hashCode());
}
/**
* 获取框架名称.
* <p>框架名称不可更改.</p>
* @return 返回框架名称.
*/
String getFrameworkName();
}

View File

@ -0,0 +1,75 @@
package net.lamgc.cgj.bot.framework;
import org.slf4j.Logger;
import java.util.*;
import java.util.regex.Pattern;
public final class FrameworkManager {
private FrameworkManager() {}
private final static Map<Framework, FrameworkResources> resourcesMap = new HashMap<>();
private final static ThreadGroup frameworkRootGroup = new ThreadGroup("FrameworkRootGroup");
static {
Runtime.getRuntime()
.addShutdownHook(new Thread(FrameworkManager::shutdownAllFramework, "FrameworkManager-Shutdown"));
}
public static Thread registerFramework(Framework framework) {
checkFramework(framework);
FrameworkResources resources = new FrameworkResources(framework);
resourcesMap.put(framework, resources);
Thread frameworkThread = new Thread(resources.getFrameworkThreadGroup(),
() -> FrameworkManager.runFramework(framework), "FrameworkThread-" + framework.getIdentify());
frameworkThread.start();
return frameworkThread;
}
private static final Pattern FRAMEWORK_NAME_CHECK_PATTERN = Pattern.compile("^[A-Za-z0-9_\\-$]+$");
private static void checkFramework(Framework framework) {
if(!FRAMEWORK_NAME_CHECK_PATTERN.matcher(framework.getFrameworkName()).matches()) {
throw new IllegalStateException("Invalid Framework Name: " + framework.getFrameworkName());
}
}
public static Set<Framework> frameworkSet() {
return new HashSet<>(resourcesMap.keySet());
}
public static void shutdownAllFramework() {
for (Framework framework : resourcesMap.keySet()) {
FrameworkResources frameworkResources = resourcesMap.get(framework);
Logger frameworkLogger = frameworkResources.getLogger();
try {
frameworkLogger.info("正在关闭框架...");
framework.close();
frameworkLogger.info("框架已关闭.");
frameworkResources.getFrameworkThreadGroup().interrupt();
resourcesMap.remove(framework);
} catch(Throwable e) {
frameworkLogger.error("退出框架时发生异常", e);
}
}
}
static ThreadGroup getFrameworkRootGroup() {
return frameworkRootGroup;
}
private static void runFramework(Framework framework) {
FrameworkResources frameworkResources = resourcesMap.get(framework);
try {
framework.init(frameworkResources);
framework.run();
} catch(Throwable e) {
frameworkResources.getLogger().error("框架未捕获异常, 导致异常退出.", e);
} finally {
frameworkResources.getFrameworkThreadGroup().interrupt();
}
}
}

View File

@ -0,0 +1,35 @@
package net.lamgc.cgj.bot.framework;
import net.lamgc.cgj.bot.boot.BotGlobal;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
public class FrameworkResources {
private final static File frameworkDataStoreRootDir = new File(BotGlobal.getGlobal().getDataStoreDir(),
"frameworks/");
private final ThreadGroup frameworkThreadGroup;
private final Logger logger;
private final File frameworkDataStoreDir;
public FrameworkResources(Framework framework) {
frameworkThreadGroup = new ThreadGroup(FrameworkManager.getFrameworkRootGroup(),
"Framework-" + framework.getIdentify());
frameworkDataStoreDir = new File(frameworkDataStoreRootDir, framework.getClass().getSimpleName());
logger = LoggerFactory.getLogger("Framework-" + framework.getIdentify());
}
ThreadGroup getFrameworkThreadGroup() {
return frameworkThreadGroup;
}
public Logger getLogger() {
return logger;
}
}

View File

@ -0,0 +1,74 @@
package net.lamgc.cgj.bot.framework.cli;
import net.lamgc.cgj.bot.boot.ApplicationBoot;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.framework.Framework;
import net.lamgc.cgj.bot.framework.FrameworkManager;
import net.lamgc.cgj.bot.framework.FrameworkResources;
import net.lamgc.cgj.bot.framework.cli.message.ConsoleMessageEvent;
import net.lamgc.cgj.bot.framework.cli.message.ConsoleMessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import org.jline.reader.LineReader;
import org.jline.reader.LineReaderBuilder;
import org.jline.reader.impl.history.DefaultHistory;
import org.jline.terminal.TerminalBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.concurrent.atomic.AtomicBoolean;
public class ConsoleMain implements Framework {
private final static Logger log = LoggerFactory.getLogger(ConsoleMain.class);
private final AtomicBoolean quitState = new AtomicBoolean();
@Override
public void init(FrameworkResources resources) { }
@Override
public void run() throws Exception {
MessageSenderBuilder.setCurrentMessageSenderFactory(new ConsoleMessageSenderFactory());
ApplicationBoot.initialBot();
LineReader lineReader = LineReaderBuilder.builder()
.appName("CGJ")
.history(new DefaultHistory())
.terminal(TerminalBuilder.terminal())
.build();
long qqId = Long.parseLong(lineReader.readLine("会话QQ: "));
long groupId = Long.parseLong(lineReader.readLine("会话群组号:"));
boolean isGroup = false;
do {
String input = lineReader.readLine("App " + qqId + (isGroup ? "@" + groupId : "#private") + " >");
if(input.equalsIgnoreCase("#exit")) {
System.out.println("退出应用...");
break;
} else if(input.equalsIgnoreCase("#setgroup")) {
isGroup = !isGroup;
System.out.println("System: 群模式状态已变更: " + isGroup);
continue;
}
try {
BotEventHandler.executeMessageEvent(new ConsoleMessageEvent(isGroup ? groupId : 0, qqId, input), true);
} catch (InterruptedException e) {
log.error("执行时发生中断", e);
}
} while(!quitState.get());
}
@Override
public void close() {
quitState.set(true);
Thread.currentThread().getThreadGroup().interrupt();
}
@Override
public String getIdentify() {
return this.toString();
}
@Override
public String getFrameworkName() {
return "console";
}
}

View File

@ -0,0 +1,29 @@
package net.lamgc.cgj.bot.framework.cli.message;
import net.lamgc.cgj.bot.event.MessageEvent;
import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import net.lamgc.cgj.bot.message.MessageSource;
public class ConsoleMessageEvent extends MessageEvent {
public ConsoleMessageEvent(long groupId, long qqId, String message) {
super(groupId, qqId, message);
}
@Override
public int sendMessage(String message) throws Exception {
if(getFromGroup() <= 0) {
return MessageSenderBuilder
.getMessageSender(MessageSource.PRIVATE, getFromQQ()).sendMessage(message);
} else {
return MessageSenderBuilder
.getMessageSender(MessageSource.GROUP, getFromQQ()).sendMessage(message);
}
}
@Override
public String getImageUrl(String image) {
return null;
}
}

View File

@ -0,0 +1,26 @@
package net.lamgc.cgj.bot.framework.cli.message;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import java.text.SimpleDateFormat;
import java.util.Date;
public class ConsoleMessageSender implements MessageSender {
private final static SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS");
private final MessageSource source;
private final long id;
ConsoleMessageSender(MessageSource source, long id) {
this.source = source;
this.id = id;
}
@Override
public synchronized int sendMessage(String message) {
System.out.println(dateFormat.format(new Date()) + " Bot -> " +
(source == MessageSource.PRIVATE ? "#" : "@") + id + ": " + message);
return 0;
}
}

View File

@ -0,0 +1,13 @@
package net.lamgc.cgj.bot.framework.cli.message;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSource;
public class ConsoleMessageSenderFactory implements MessageSenderFactory {
@Override
public MessageSender createMessageSender(MessageSource source, long id) {
return new ConsoleMessageSender(source, id);
}
}

View File

@ -1,7 +1,9 @@
package net.lamgc.cgj.bot;
package net.lamgc.cgj.bot.framework.coolq;
import net.lamgc.cgj.bot.boot.ApplicationBoot;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.event.SpringCQMessageEvent;
import net.lamgc.cgj.bot.framework.coolq.message.SpringCQMessageEvent;
import net.lamgc.cgj.bot.framework.coolq.message.SpringCQMessageSenderFactory;
import net.lamgc.utils.event.EventHandler;
import net.lz1998.cq.event.message.CQDiscussMessageEvent;
import net.lz1998.cq.event.message.CQGroupMessageEvent;
@ -12,37 +14,52 @@ import net.lz1998.cq.robot.CoolQ;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
import java.util.concurrent.atomic.AtomicBoolean;
@Component
@SuppressWarnings("unused")
public class CQPluginMain extends CQPlugin implements EventHandler {
private final static AtomicBoolean initialState = new AtomicBoolean();
public CQPluginMain() {
LoggerFactory.getLogger(this.toString())
LoggerFactory.getLogger(CQPluginMain.class)
.info("BotEventHandler.COMMAND_PREFIX = {}", BotEventHandler.COMMAND_PREFIX);
}
@Override
public int onPrivateMessage(CoolQ cq, CQPrivateMessageEvent event) {
//log.info("私聊消息到达: 发送者[{}], 消息内容: {}", event.getSender().getUserId(), event.getMessage());
return processMessage(cq, event);
}
@Override
public int onGroupMessage(CoolQ cq, CQGroupMessageEvent event) {
//log.info("群消息到达: 群[{}], 发送者[{}], 消息内容: {}", event.getGroupId(), event.getSender().getUserId(), event.getMessage());
return processMessage(cq, event);
}
@Override
public int onDiscussMessage(CoolQ cq, CQDiscussMessageEvent event) {
//log.info("讨论组消息到达: 群[{}], 发送者[{}], 消息内容: {}", event.getDiscussId(), event.getSender().getUserId(), event.getMessage());
return processMessage(cq, event);
}
public int processMessage(CoolQ cq, CQMessageEvent event) {
if(!BotEventHandler.match(event.getMessage())) {
/**
* 处理消息
* @param cq CoolQ机器人对象
* @param event 消息事件
* @return 是否拦截消息
*/
private static int processMessage(CoolQ cq, CQMessageEvent event) {
SpringCQMessageSenderFactory.setCoolQ(cq);
synchronized (initialState) {
if(!initialState.get()) {
ApplicationBoot.initialBot();
initialState.set(true);
}
}
if(BotEventHandler.mismatch(event.getMessage())) {
return MESSAGE_IGNORE;
}
BotEventHandler.executor.executor(new SpringCQMessageEvent(cq, event));
BotEventHandler.executeMessageEvent(new SpringCQMessageEvent(cq, event));
return MESSAGE_BLOCK;
}

View File

@ -0,0 +1,64 @@
package net.lamgc.cgj.bot.framework.coolq;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.framework.Framework;
import net.lamgc.cgj.bot.framework.FrameworkResources;
import org.slf4j.Logger;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.context.event.ApplicationFailedEvent;
import org.springframework.context.ApplicationListener;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.context.event.ContextClosedEvent;
import org.springframework.context.event.ContextStoppedEvent;
@SpringBootApplication
public class SpringCQApplication implements Framework {
private Logger log;
private final Object quitLock = new Object();
@Override
public void init(FrameworkResources resources) {
this.log = resources.getLogger();
}
public void run() {
log.info("酷Q机器人根目录: {}", BotGlobal.getGlobal().getDataStoreDir().getPath());
ConfigurableApplicationContext context = SpringApplication.run(SpringCQApplication.class);
registerShutdownHook(context);
try {
synchronized (quitLock) {
quitLock.wait();
}
} catch (InterruptedException e) {
log.warn("发生中断, 退出SpringCQ...", e);
}
context.stop();
context.close();
}
private void registerShutdownHook(ConfigurableApplicationContext context) {
context.addApplicationListener((ApplicationListener<ApplicationFailedEvent>)
event -> close());
context.addApplicationListener((ApplicationListener<ContextClosedEvent>)
event -> close());
context.addApplicationListener((ApplicationListener<ContextStoppedEvent>)
event -> close());
Runtime.getRuntime().addShutdownHook(new Thread(this::close));
}
public void close() {
synchronized (quitLock) {
quitLock.notify();
}
}
@Override
public String getFrameworkName() {
return "SpringCoolQ";
}
}

View File

@ -1,6 +1,9 @@
package net.lamgc.cgj.bot.event;
package net.lamgc.cgj.bot.framework.coolq.message;
import net.lamgc.cgj.bot.BotCode;
import net.lamgc.cgj.bot.event.MessageEvent;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lz1998.cq.event.message.CQDiscussMessageEvent;
import net.lz1998.cq.event.message.CQGroupMessageEvent;
import net.lz1998.cq.event.message.CQMessageEvent;
@ -11,13 +14,8 @@ import java.util.Objects;
public class SpringCQMessageEvent extends MessageEvent {
private final static int TYPE_PRIVATE = 0;
private final static int TYPE_GROUP = 1;
private final static int TYPE_DISCUSS = 2;
private final CoolQ cq;
private final int type;
private final CQMessageEvent messageEvent;
private final MessageSender messageSender;
public SpringCQMessageEvent(CoolQ cq, CQMessageEvent messageEvent) {
super(messageEvent instanceof CQGroupMessageEvent ? (
@ -26,27 +24,20 @@ public class SpringCQMessageEvent extends MessageEvent {
((CQDiscussMessageEvent) messageEvent).getDiscussId() : 0,
messageEvent.getUserId(), messageEvent.getMessage());
this.cq = Objects.requireNonNull(cq);
MessageSource source;
if(messageEvent instanceof CQGroupMessageEvent) {
type = TYPE_GROUP;
source = MessageSource.GROUP;
} else if (messageEvent instanceof CQDiscussMessageEvent) {
type = TYPE_DISCUSS;
source = MessageSource.DISCUSS;
} else {
type = TYPE_PRIVATE;
source = MessageSource.PRIVATE;
}
this.messageEvent = messageEvent;
messageSender = new SpringCQMessageSender(cq, source, source == MessageSource.PRIVATE ? getFromQQ() : getFromGroup());
}
@Override
public int sendMessage(final String message) {
switch(type) {
case TYPE_PRIVATE:
return cq.sendPrivateMsg(getFromQQ(), message, false).getData().getMessageId();
case TYPE_GROUP:
case TYPE_DISCUSS:
return cq.sendGroupMsg(getFromGroup(), message, false).getData().getMessageId();
default:
return -1;
}
public int sendMessage(final String message) throws Exception {
return messageSender.sendMessage(message);
}
/**
@ -70,8 +61,4 @@ public class SpringCQMessageEvent extends MessageEvent {
}
}
@Override
public Object getRawMessage() {
return messageEvent;
}
}

View File

@ -0,0 +1,32 @@
package net.lamgc.cgj.bot.framework.coolq.message;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lz1998.cq.robot.CoolQ;
public class SpringCQMessageSender implements MessageSender {
private final CoolQ coolQ;
private final MessageSource source;
private final long target;
public SpringCQMessageSender(CoolQ coolQ, MessageSource source, long target) {
this.coolQ = coolQ;
this.source = source;
this.target = target;
}
@Override
public int sendMessage(String message) {
switch (source) {
case PRIVATE:
return coolQ.sendPrivateMsg(target, message, false).getData().getMessageId();
case GROUP:
return coolQ.sendGroupMsg(target, message, false).getData().getMessageId();
case DISCUSS:
return coolQ.sendDiscussMsg(target, message, false).getData().getMessageId();
default:
return -1;
}
}
}

View File

@ -0,0 +1,31 @@
package net.lamgc.cgj.bot.framework.coolq.message;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSource;
import net.lz1998.cq.robot.CoolQ;
import java.util.Objects;
import java.util.concurrent.atomic.AtomicReference;
public class SpringCQMessageSenderFactory implements MessageSenderFactory {
private final static AtomicReference<CoolQ> coolQ = new AtomicReference<>();
/**
* 设置CoolQ对象.
* <p>该方法仅接受第一次设置的CoolQ对象, 其他对象将会忽略.</p>
* @param coolQObj CoolQ对象
*/
public static void setCoolQ(CoolQ coolQObj) {
if(coolQ.get() == null) {
coolQ.set(coolQObj);
}
}
@Override
public MessageSender createMessageSender(MessageSource source, long id) {
return new SpringCQMessageSender(
Objects.requireNonNull(coolQ.get(), "CoolQ object not ready"), source, id);
}
}

View File

@ -0,0 +1,136 @@
package net.lamgc.cgj.bot.framework.mirai;
import net.lamgc.cgj.bot.boot.ApplicationBoot;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.framework.Framework;
import net.lamgc.cgj.bot.framework.FrameworkResources;
import net.lamgc.cgj.bot.framework.mirai.message.MiraiMessageEvent;
import net.lamgc.cgj.bot.framework.mirai.message.MiraiMessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import net.lamgc.cgj.bot.util.GroupMuteManager;
import net.mamoe.mirai.Bot;
import net.mamoe.mirai.BotFactoryJvm;
import net.mamoe.mirai.event.events.BotMuteEvent;
import net.mamoe.mirai.event.events.BotUnmuteEvent;
import net.mamoe.mirai.japt.Events;
import net.mamoe.mirai.message.FriendMessageEvent;
import net.mamoe.mirai.message.GroupMessageEvent;
import net.mamoe.mirai.message.MessageEvent;
import net.mamoe.mirai.message.TempMessageEvent;
import net.mamoe.mirai.utils.BotConfiguration;
import net.mamoe.mirai.utils.Utils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.*;
import java.util.Base64;
import java.util.Properties;
public class MiraiMain implements Framework {
private final Logger log = LoggerFactory.getLogger(MiraiMain.class);
private Bot bot;
private final Properties botProperties = new Properties();
private final GroupMuteManager muteManager = new GroupMuteManager();
@Override
public void init(FrameworkResources resources) {
Runtime.getRuntime().addShutdownHook(new Thread(this::close));
try {
Class.forName(BotEventHandler.class.getName());
} catch (ClassNotFoundException e) {
log.error("加载BotEventHandler时发生异常", e);
return;
}
File botPropFile = new File(BotGlobal.getGlobal().getDataStoreDir(), "./bot.properties");
try (Reader reader = new BufferedReader(new FileReader(botPropFile))) {
botProperties.load(reader);
} catch (IOException e) {
log.error("机器人配置文件读取失败!", e);
return;
}
Utils.setDefaultLogger(MiraiToSlf4jLoggerAdapter::new);
BotConfiguration configuration = new BotConfiguration();
configuration.randomDeviceInfo();
configuration.setProtocol(BotConfiguration.MiraiProtocol.ANDROID_PAD);
// 心跳包周期间隔 (ms)
configuration.setHeartbeatPeriodMillis(
Long.parseLong(botProperties.getProperty("network.heartbeatPeriodMillis", "60000")));
// 心跳包超时时间 (ms)
configuration.setHeartbeatTimeoutMillis(
Long.parseLong(botProperties.getProperty("network.heartbeatTimeoutMillis", "5000")));
// 重连间隔时间
configuration.setReconnectPeriodMillis(
Integer.parseInt(botProperties.getProperty("network.reconnectPeriodMillis", "5")));
// 重连最大次数
configuration.setReconnectionRetryTimes(
Integer.parseInt(botProperties.getProperty("network.reconnectionRetryTimes", "10")));
bot = BotFactoryJvm.newBot(Long.parseLong(botProperties.getProperty("bot.qq", "0")),
Base64.getDecoder().decode(botProperties.getProperty("bot.password", "")), configuration);
// TODO: 看看能不能单独订阅某个Bot?
Events.subscribeAlways(GroupMessageEvent.class, this::executeMessageEvent);
Events.subscribeAlways(FriendMessageEvent.class, this::executeMessageEvent);
Events.subscribeAlways(TempMessageEvent.class, this::executeMessageEvent);
Events.subscribeAlways(BotMuteEvent.class,
event -> muteManager.setMuteState(event.getGroup().getId(), true));
Events.subscribeAlways(BotUnmuteEvent.class,
event -> muteManager.setMuteState(event.getGroup().getId(), false));
bot.login();
MessageSenderBuilder.setCurrentMessageSenderFactory(new MiraiMessageSenderFactory(bot));
ApplicationBoot.initialBot();
bot.join();
}
@Override
public void run() {
bot.login();
bot.join();
}
/**
* 处理消息事件
* @param message 消息事件对象
*/
private void executeMessageEvent(MessageEvent message) {
log.debug("Mirai Message: {}", message);
if(message instanceof GroupMessageEvent) {
GroupMessageEvent GroupMessageEvent = (GroupMessageEvent) message;
Boolean muteState = muteManager.isMute(GroupMessageEvent.getGroup().getId(), true);
if(muteState == null) {
muteManager.setMuteState(GroupMessageEvent.getGroup().getId(),
((GroupMessageEvent) message).getGroup().getBotMuteRemaining() != 0);
} else if(muteState) {
return;
}
}
BotEventHandler.executeMessageEvent(MiraiMessageEvent.covertEventObject(message));
}
/**
* 关闭机器人
*/
@Override
public synchronized void close() {
if(bot == null) {
return;
}
log.warn("正在关闭机器人...");
bot.close(null);
bot = null;
log.warn("机器人已关闭.");
}
@Override
public String getFrameworkName() {
return "MiraiQQ";
}
}

View File

@ -0,0 +1,89 @@
package net.lamgc.cgj.bot.framework.mirai;
import net.mamoe.mirai.utils.MiraiLogger;
import net.mamoe.mirai.utils.MiraiLoggerPlatformBase;
import org.jetbrains.annotations.Nullable;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.Marker;
import org.slf4j.MarkerFactory;
/**
* MiraiLoggerToSlf4jLogger适配器
* <p>该Logger通过Slf4j的Marker进行标识, loggerName为{@code mirai.[identity]}</p>
* <p>由于适配器适配方式的原因, 日志输出的调用信息将不可用(调用指向了适配器内的方法);</p>
*/
public class MiraiToSlf4jLoggerAdapter extends MiraiLoggerPlatformBase {
private final static Marker marker = MarkerFactory.getMarker("mirai");
private final Logger logger;
private final String identity;
public MiraiToSlf4jLoggerAdapter(String identity) {
this.identity = identity;
this.logger = LoggerFactory.getLogger("mirai." + identity);
}
@Override
protected void debug0(@Nullable String s) {
logger.debug(marker, s);
}
@Override
protected void debug0(@Nullable String s, @Nullable Throwable throwable) {
logger.debug(marker, s, throwable);
}
@Override
protected void error0(@Nullable String s) {
logger.error(marker, s);
}
@Override
protected void error0(@Nullable String s, @Nullable Throwable throwable) {
logger.error(marker, s, throwable);
}
@Override
protected void info0(@Nullable String s) {
logger.info(marker, s);
}
@Override
protected void info0(@Nullable String s, @Nullable Throwable throwable) {
logger.info(marker, s, throwable);
}
@Override
protected void verbose0(@Nullable String s) {
logger.trace(marker, s);
}
@Override
protected void verbose0(@Nullable String s, @Nullable Throwable throwable) {
logger.trace(marker, s, throwable);
}
@Override
protected void warning0(@Nullable String s) {
logger.warn(marker, s);
}
@Override
protected void warning0(@Nullable String s, @Nullable Throwable throwable) {
logger.warn(marker, s, throwable);
}
@Nullable
@Override
public String getIdentity() {
if(identity == null) {
MiraiLogger followerLogger = getFollower();
return followerLogger == null ? null : followerLogger.getIdentity();
} else {
return identity;
}
}
}

View File

@ -0,0 +1,98 @@
package net.lamgc.cgj.bot.framework.mirai.message;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.message.FriendMessageEvent;
import net.mamoe.mirai.message.GroupMessageEvent;
import net.mamoe.mirai.message.MessageEvent;
import net.mamoe.mirai.message.TempMessageEvent;
import net.mamoe.mirai.message.data.MessageChain;
import net.mamoe.mirai.message.data.MessageUtils;
import java.util.Objects;
public class MiraiMessageEvent extends net.lamgc.cgj.bot.event.MessageEvent {
private final MessageEvent messageObject;
private final MessageSender messageSender;
/**
* 通过Mirai的MessageEvent转换成应用支持的MessageEvent.
* @deprecated 请使用 {@link #covertEventObject(MessageEvent)}方法转换.
* @param message 消息对象
* @see #covertEventObject(MessageEvent)
*/
@Deprecated
public MiraiMessageEvent(MessageEvent message) {
super(message instanceof GroupMessageEvent ? ((GroupMessageEvent) message).getGroup().getId() : 0,
message.getSender().getId(), getMessageBodyWithoutSource(message.getMessage().toString()));
this.messageObject = Objects.requireNonNull(message);
if(message instanceof GroupMessageEvent) {
messageSender = new MiraiMessageSender(((GroupMessageEvent) message).getGroup(), MessageSource.GROUP);
} else {
messageSender = new MiraiMessageSender(message.getSender(), MessageSource.PRIVATE);
}
}
/**
* 通过解析好的信息构造MessageEvent
* @param messageObject 消息原始对象
* @param groupId 群组Id, 非群聊或无需使用群聊时, 该参数为0
* @param qqId 发送者Id, 不能为0
* @param message 原始消息内容对象, 由构造方法内部解析
*/
private MiraiMessageEvent(MessageEvent messageObject, long groupId, long qqId, MessageChain message) {
super(groupId, qqId, getMessageBodyWithoutSource(message.toString()));
this.messageObject = Objects.requireNonNull(messageObject, "messageObject is null");
if(groupId != 0) {
this.messageSender = new MiraiMessageSender(((GroupMessageEvent) messageObject).getGroup(), MessageSource.GROUP);
} else {
this.messageSender = new MiraiMessageSender(messageObject.getSender(), MessageSource.GROUP);
}
}
/**
* 将Mirai原始MessageEvent转换成应用支持的MessageEvent对象
* @param event 原始消息对象
* @return 原始消息对象所对应的应用MessageEvent对象.
* @throws IllegalArgumentException 当出现不支持的Mirai {@link MessageEvent}实现时将抛出异常.
* @see MessageEvent 原始消息对象
* @see net.lamgc.cgj.bot.event.MessageEvent 应用消息对象
*/
public static MiraiMessageEvent covertEventObject(MessageEvent event) throws IllegalArgumentException {
if(event instanceof GroupMessageEvent) {
return new MiraiMessageEvent(event,
((GroupMessageEvent) event).getGroup().getId(), event.getSender().getId(), event.getMessage());
} else if(event instanceof FriendMessageEvent) {
return new MiraiMessageEvent(event, 0, event.getSender().getId(), event.getMessage());
} else if(event instanceof TempMessageEvent) {
return new MiraiMessageEvent(event, 0, event.getSender().getId(), event.getMessage());
} else {
throw new IllegalArgumentException("Unsupported event type: " + event.toString());
}
}
/**
* 将ContactMessage获得的消息内容删除 Mirai:source 并返回.
* <p>该做法比较保守, 防止Mirai:source位置出现变动.</p>
* @param message ContactMessage的消息内容;
* @return 返回删除了Mirai:source的消息
*/
private static String getMessageBodyWithoutSource(String message) {
StringBuilder builder = new StringBuilder(message);
int startIndex = builder.indexOf("[mirai:source:");
int endIndex = builder.indexOf("]", startIndex) + 1;
return builder.delete(startIndex, endIndex).toString();
}
@Override
public int sendMessage(final String message) throws Exception {
return messageSender.sendMessage(message);
}
@Override
public String getImageUrl(String imageId) {
return messageObject.getBot().queryImageUrl(MessageUtils.newImage(imageId));
}
}

View File

@ -1,70 +1,68 @@
package net.lamgc.cgj.bot.event;
package net.lamgc.cgj.bot.framework.mirai.message;
import com.google.common.base.Strings;
import net.lamgc.cgj.bot.BotCode;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.cache.CacheStore;
import net.lamgc.cgj.bot.cache.HotDataCacheStore;
import net.lamgc.cgj.bot.cache.LocalHashCacheStore;
import net.lamgc.cgj.bot.cache.StringRedisCacheStore;
import net.mamoe.mirai.message.ContactMessage;
import net.mamoe.mirai.message.FriendMessage;
import net.mamoe.mirai.message.GroupMessage;
import net.mamoe.mirai.message.data.CombinedMessage;
import net.mamoe.mirai.message.data.Image;
import net.mamoe.mirai.message.data.Message;
import net.mamoe.mirai.message.data.MessageUtils;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.Bot;
import net.mamoe.mirai.contact.Contact;
import net.mamoe.mirai.message.data.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
import java.util.ArrayList;
import java.util.NoSuchElementException;
import java.util.Objects;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class MiraiMessageEvent extends MessageEvent {
public class MiraiMessageSender implements MessageSender {
private final ContactMessage messageObject;
private final Logger log = LoggerFactory.getLogger(this.getClass().getSimpleName() + "@" + Integer.toHexString(this.hashCode()));
private final Contact member;
private final MessageSource source;
private final static Logger log = LoggerFactory.getLogger(MiraiMessageSender.class);
private final static CacheStore<String> imageIdCache = new HotDataCacheStore<>(
new StringRedisCacheStore(BotEventHandler.redisServer, "mirai.imageId"),
new StringRedisCacheStore(BotGlobal.getGlobal().getRedisServer(), "mirai.imageId"),
new LocalHashCacheStore<>(),
5400000, 1800000);
5400000, 1800000, true);
/**
* 使用id构造发送器
* @param bot 机器人对象
* @param source 消息源类型
* @param id id, 将会根据消息源类型判断为什么号(QQ号或群号)
* @throws NoSuchElementException 当在机器人好友列表或群列表里没有这个好友或群的时候抛出
*/
public MiraiMessageSender(Bot bot, MessageSource source, long id) {
this(source == MessageSource.PRIVATE ? bot.getFriend(id) : bot.getGroup(id), source);
}
public MiraiMessageEvent(ContactMessage message) {
super(message instanceof GroupMessage ? ((GroupMessage) message).getGroup().getId() : 0,
message.getSender().getId(), message.getMessage().toString());
this.messageObject = Objects.requireNonNull(message);
/**
* 通过联系人对象构造发送器
* @param contact 联系人
* @param source 消息源类型
*/
public MiraiMessageSender(Contact contact, MessageSource source) {
this.member = contact;
this.source = source;
}
@Override
public int sendMessage(final String message) {
log.debug("处理前的消息内容:\n{}", message);
Message msgBody = processMessage(Objects.requireNonNull(message));
log.debug("处理后的消息内容(可能出现乱序的情况, 但实际上顺序是没问题的):\n{}", msgBody);
if(getFromGroup() == 0) {
FriendMessage msgObject = (FriendMessage) messageObject;
//FIXME(LamGC, 2020.04.10): 当前 Mirai 不支持私聊长文本, 所以发生异常是正常情况...
msgObject.getSender().sendMessage(msgBody);
} else {
GroupMessage msgObject = (GroupMessage) messageObject;
msgObject.getGroup().sendMessage(msgBody);
}
log.debug("处理后的消息内容(可能出现乱序的情况, 但实际上顺序是没问题的):\n{}", msgBody.contentToString());
member.sendMessage(msgBody);
return 0;
}
@Override
public String getImageUrl(String imageId) {
return messageObject.getBot().queryImageUrl(MessageUtils.newImage(imageId));
}
@Override
public Object getRawMessage() {
return messageObject;
}
private final static Pattern cqCodePattern = Pattern.compile("\\[.*?:.*?]");
private final static Pattern cqCodePattern = BotCode.getCodePattern();
private Message processMessage(final String message) {
Matcher matcher = cqCodePattern.matcher(message);
ArrayList<String> cqCode = new ArrayList<>();
@ -79,7 +77,7 @@ public class MiraiMessageEvent extends MessageEvent {
.replaceAll("&38", "&")
.split("\\|");
CombinedMessage chain = MessageUtils.newChain().plus("");
MessageChain messages = MessageUtils.newChain();
int codeIndex = 0;
for(String text : texts) {
if(text.equals("{BotCode}")) {
@ -90,32 +88,54 @@ public class MiraiMessageEvent extends MessageEvent {
log.warn("解析待发送消息内的BotCode时发生异常, 请检查错误格式BotCode的来源并尽快排错!", e);
continue;
}
chain = chain.plus(processBotCode(code));
messages = messages.plus(processBotCode(code));
} else {
chain = chain.plus(text);
messages = messages.plus(text);
}
}
return chain;
return messages;
}
private Message processBotCode(BotCode code) {
switch(code.getFunctionName().toLowerCase()) {
case "image":
Image img;
if(code.containsParameter("id")) {
return MessageUtils.newImage(code.getParameter("id"));
img = MessageUtils.newImage(code.getParameter("id"));
} else if(code.containsParameter("absolutePath")) {
return uploadImage(code);
img = uploadImage(code);
} else {
return MessageUtils.newChain("(参数不存在)");
}
if(Strings.nullToEmpty(code.getParameter("flashImage"))
.equalsIgnoreCase("true")) {
return MessageUtils.flash(img);
} else {
return img;
}
case "face":
if(!code.containsParameter("id")) {
return MessageUtils.newChain("(无效的表情Id)");
}
int faceId = Integer.parseInt(code.getParameter("id"));
if(faceId <= 0) {
return MessageUtils.newChain("(无效的表情Id)");
}
return new Face(faceId);
default:
log.warn("解析到不支持的BotCode: {}", code);
return MessageUtils.newChain("(不支持的BotCode)");
}
}
private Image uploadImage(BotCode code) {
/**
* 存在缓存的上传图片.
* @param code 图片BotCode
* @return Image对象
*/
@SuppressWarnings("SynchronizationOnLocalVariableOrMethodParameter")
public Image uploadImage(BotCode code) {
log.debug("传入BotCode信息:\n{}", code);
String absolutePath = code.getParameter("absolutePath");
if(Strings.isNullOrEmpty(absolutePath)) {
@ -125,20 +145,16 @@ public class MiraiMessageEvent extends MessageEvent {
String imageName = code.getParameter("imageName");
if(!Strings.isNullOrEmpty(imageName)) {
Image image = null;
imageName = (getMessageSource() + "." + imageName).intern();
imageName = (source + "." + imageName).intern();
if(!imageIdCache.exists(imageName) ||
Strings.nullToEmpty(code.getParameter("updateCache")).equalsIgnoreCase("true")) {
Strings.nullToEmpty(code.getParameter("updateCache")).equalsIgnoreCase("true")) {
synchronized (imageName) {
if(!imageIdCache.exists(imageName) ||
Strings.nullToEmpty(code.getParameter("updateCache")) .equalsIgnoreCase("true")) {
log.debug("imageName [{}] 缓存失效或强制更新, 正在更新缓存...", imageName);
Strings.nullToEmpty(code.getParameter("updateCache")) .equalsIgnoreCase("true")) {
log.trace("imageName [{}] 缓存失效或强制更新, 正在更新缓存...", imageName);
image = uploadImage0(new File(absolutePath));
if(Objects.isNull(image)) {
return null;
}
String cacheExpireAt;
long expireTime = 0;
long expireTime = 864000000; // 10d
if(!Strings.isNullOrEmpty(cacheExpireAt = code.getParameter("cacheExpireAt"))) {
try {
expireTime = Integer.parseInt(cacheExpireAt);
@ -147,13 +163,13 @@ public class MiraiMessageEvent extends MessageEvent {
}
}
imageIdCache.update(imageName, image.getImageId(), expireTime);
log.info("imageName [{}] 缓存更新完成.(有效时间: {})", imageName, expireTime);
log.trace("imageName [{}] 缓存更新完成.(有效时间: {})", imageName, expireTime);
} else {
log.debug("ImageName: [{}] 缓存命中.", imageName);
log.trace("ImageName: [{}] 缓存命中.", imageName);
}
}
} else {
log.debug("ImageName: [{}] 缓存命中.", imageName);
log.trace("ImageName: [{}] 缓存命中.", imageName);
}
if(image == null) {
@ -169,25 +185,7 @@ public class MiraiMessageEvent extends MessageEvent {
}
private Image uploadImage0(File imageFile) {
if(messageObject instanceof FriendMessage) {
return messageObject.getSender().uploadImage(imageFile);
} else if(messageObject instanceof GroupMessage) {
return ((GroupMessage) messageObject).getGroup().uploadImage(imageFile);
} else {
log.warn("未知的ContactMessage类型: " + messageObject.toString());
return null;
}
}
private String getMessageSource() {
if(messageObject instanceof FriendMessage) {
return "Private";
} else if(messageObject instanceof GroupMessage) {
return "Group";
} else {
log.warn("未知的ContactMessage类型: " + messageObject.toString());
return "Unknown";
}
return member.uploadImage(imageFile);
}
}

View File

@ -0,0 +1,26 @@
package net.lamgc.cgj.bot.framework.mirai.message;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.Bot;
import java.util.Objects;
public class MiraiMessageSenderFactory implements MessageSenderFactory {
private final Bot bot;
public MiraiMessageSenderFactory(Bot bot) {
this.bot = bot;
}
@Override
public MessageSender createMessageSender(MessageSource source, long id) {
Objects.requireNonNull(source);
if(id <= 0) {
throw new IllegalArgumentException("id cannot be 0 or negative: " + id);
}
return new MiraiMessageSender(bot, source, id);
}
}

View File

@ -0,0 +1,16 @@
package net.lamgc.cgj.bot.message;
public interface MessageSender {
/**
* 发送消息并返回消息id
* @param message 消息内容
* @return 返回非负数则发送成功,
* 返回0则发送器不支持消息Id,
* 返回非0正整数则为消息Id,
* 返回负数则为错误.
* @throws Exception 该方法根据不同实现, 可能会抛出不同异常, 详见实现所标识的文档内容.
*/
int sendMessage(final String message) throws Exception;
}

View File

@ -0,0 +1,43 @@
package net.lamgc.cgj.bot.message;
import java.util.concurrent.atomic.AtomicReference;
/**
* 消息发送器构造
*/
public final class MessageSenderBuilder {
private final static AtomicReference<MessageSenderFactory> currentFactory = new AtomicReference<>();
private MessageSenderBuilder() {}
/**
* 获取消息发送器
* @param source 消息源类型
* @param id 消息源Id
* @return 返回新建的发送器
*/
public static MessageSender getMessageSender(MessageSource source, long id) {
MessageSenderFactory messageSenderFactory = currentFactory.get();
if(messageSenderFactory == null) {
throw new IllegalStateException("The factory is not ready");
}
try {
return messageSenderFactory.createMessageSender(source, id);
} catch (Exception e) {
throw new IllegalStateException(e);
}
}
/**
* 设置消息发送器工厂
* @param factory 工厂对象
*/
public static void setCurrentMessageSenderFactory(MessageSenderFactory factory) {
if(currentFactory.get() != null) {
throw new IllegalStateException("Factory already exists");
}
currentFactory.set(factory);
}
}

View File

@ -0,0 +1,13 @@
package net.lamgc.cgj.bot.message;
public interface MessageSenderFactory {
/**
* 通过Id创建发送器
* @param source 消息源
* @param id 消息源id
* @return 如果成功返回MessageSender
*/
MessageSender createMessageSender(MessageSource source, long id) throws Exception;
}

View File

@ -0,0 +1,23 @@
package net.lamgc.cgj.bot.message;
/**
* 消息来源
*/
public enum MessageSource {
/**
* 私聊消息
*/
PRIVATE,
/**
* 群组消息
*/
GROUP,
/**
* 讨论组消息
*/
DISCUSS,
/**
* 未知来源
*/
UNKNOWN
}

View File

@ -0,0 +1,41 @@
package net.lamgc.cgj.bot.sort;
@SuppressWarnings("unused")
public enum PreLoadDataAttribute {
/**
* 按点赞数排序
*/
LIKE("likeCount"),
/**
* 按页面数排序
*/
PAGE("pageCount"),
/**
* 按收藏数排序
*/
BOOKMARK("bookmarkCount"),
/**
* 按评论数排序
*/
COMMENT("commentCount"),
/**
* 不明
*/
RESPONSE("responseCount"),
/**
* 按查看次数排序
*/
VIEW("viewCount"),
;
public final String attrName;
PreLoadDataAttribute(String attrName) {
this.attrName = attrName;
}
}

View File

@ -0,0 +1,59 @@
package net.lamgc.cgj.bot.sort;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.cache.CacheStoreCentral;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.util.Comparator;
/**
* 收藏数比较器
*/
public class PreLoadDataAttributeComparator implements Comparator<JsonElement> {
private final static Logger log = LoggerFactory.getLogger(PreLoadDataAttributeComparator.class);
private final PreLoadDataAttribute attribute;
public PreLoadDataAttributeComparator(PreLoadDataAttribute attribute) {
this.attribute = attribute;
}
@Override
public int compare(JsonElement o1, JsonElement o2) {
if(!o1.isJsonObject() || !o2.isJsonObject()) {
if(o1.isJsonObject()) {
return 1;
} else if(o2.isJsonObject()) {
return -1;
} else {
return 0;
}
}
if(!o1.getAsJsonObject().has("illustId") || !o2.getAsJsonObject().has("illustId")) {
if(o1.getAsJsonObject().has("illustId")) {
return 1;
} else if(o2.getAsJsonObject().has("illustId")) {
return -1;
} else {
return 0;
}
}
try {
JsonObject illustPreLoadData1 = CacheStoreCentral.getCentral()
.getIllustPreLoadData(o1.getAsJsonObject().get("illustId").getAsInt(), false);
JsonObject illustPreLoadData2 = CacheStoreCentral.getCentral()
.getIllustPreLoadData(o2.getAsJsonObject().get("illustId").getAsInt(), false);
return Integer.compare(
illustPreLoadData2.get(attribute.attrName).getAsInt(),
illustPreLoadData1.get(attribute.attrName).getAsInt());
} catch (IOException e) {
log.error("获取预加载数据失败", e);
return 0;
}
}
}

View File

@ -1,82 +0,0 @@
package net.lamgc.cgj.bot.sort;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.BotCommandProcess;
import java.io.IOException;
import java.util.Comparator;
/**
* 收藏数比较器
*/
public class PreLoadDataComparator implements Comparator<JsonElement> {
private final Attribute attribute;
public PreLoadDataComparator(Attribute attribute) {
this.attribute = attribute;
}
@Override
public int compare(JsonElement o1, JsonElement o2) {
if(!o1.getAsJsonObject().has("illustId") || !o2.getAsJsonObject().has("illustId")) {
if(o1.getAsJsonObject().has("illustId")) {
return 1;
} else if(o2.getAsJsonObject().has("illustId")) {
return -1;
} else {
return 0;
}
}
try {
JsonObject illustPreLoadData1 = BotCommandProcess.getIllustPreLoadData(o1.getAsJsonObject().get("illustId").getAsInt(), false);
JsonObject illustPreLoadData2 = BotCommandProcess.getIllustPreLoadData(o2.getAsJsonObject().get("illustId").getAsInt(), false);
return Integer.compare(illustPreLoadData2.get(attribute.attrName).getAsInt(), illustPreLoadData1.get(attribute.attrName).getAsInt());
} catch (IOException e) {
e.printStackTrace();
return 0;
}
}
public enum Attribute {
/**
* 按点赞数排序
*/
LIKE("likeCount"),
/**
* 按页面数排序
*/
PAGE("pageCount"),
/**
* 按收藏数排序
*/
BOOKMARK("bookmarkCount"),
/**
* 按评论数排序
*/
COMMENT("commentCount"),
/**
* 不明
*/
RESPONSE("responseCount"),
/**
* 按查看次数排序
*/
VIEW("viewCount"),
;
final String attrName;
Attribute(String attrName) {
this.attrName = attrName;
}
}
}

View File

@ -0,0 +1,48 @@
package net.lamgc.cgj.bot.util;
import java.util.Hashtable;
import java.util.Map;
import java.util.concurrent.atomic.AtomicBoolean;
/**
* 群禁言管理器.
* <p>该管理器用于存取群组禁言状态.</p>
*/
public class GroupMuteManager {
private final Map<Long, AtomicBoolean> muteStateMap = new Hashtable<>();
/**
* 查询某群是否被禁言.
* @param groupId 群组Id
* @param rawValue 是否返回原始值(当没有该群状态, 且本参数为true时, 将返回null)
* @return 返回状态值, 如无该群禁言记录且rawValue = true, 则返回null
*/
public Boolean isMute(long groupId, boolean rawValue) {
if(groupId <= 0) {
return false;
}
AtomicBoolean state = muteStateMap.get(groupId);
if(state == null && rawValue) {
return null;
}
return state != null && state.get();
}
/**
* 设置机器人禁言状态.
* <p>设置该项可防止因机器人在禁言期间反馈请求导致被封号.</p>
* @param mute 如果被禁言, 传入true
*/
public void setMuteState(long groupId, boolean mute) {
if(groupId <= 0) {
return;
}
if(!muteStateMap.containsKey(groupId)) {
muteStateMap.put(groupId, new AtomicBoolean(mute));
} else {
muteStateMap.get(groupId).set(mute);
}
}
}

View File

@ -0,0 +1,44 @@
package net.lamgc.cgj.exception;
import java.io.IOException;
import java.util.Objects;
import org.apache.http.HttpResponse;
import org.apache.http.StatusLine;
import org.apache.http.util.EntityUtils;
public class HttpRequestException extends IOException {
private static final long serialVersionUID = -2229221075943552798L;
private final StatusLine statusLine;
private final String content;
public HttpRequestException(HttpResponse response) throws IOException {
this(response.getStatusLine(), EntityUtils.toString(response.getEntity()));
}
public HttpRequestException(StatusLine statusLine, String content) {
super("Http Response Error: " + Objects.requireNonNull(statusLine, "statusLine is null") +
", Response Content: " + (content == null ? "null" : '\'' + content + '\''));
this.statusLine = statusLine;
this.content = content;
}
/**
* 获取Http状态行
*/
public StatusLine getStatusLine() {
return statusLine;
}
/**
* 获取Response内容
* @return 如果没有返回, 则返回null
*/
public String getContent() {
return content;
}
}

View File

@ -0,0 +1,33 @@
package net.lamgc.cgj.pixiv;
public interface AdultContentDetector {
/**
* 检查某一作品的成人内容判断指数
* @param illustId 作品Id
* @param isUgoira 是否为动图
* @param pageIndex 指定页数, 设为0或负数则视为单页面作品
* @return 返回成人作品判断指数(0 ~ 1), 需按照情况设置阀值.
*/
double detect(int illustId, boolean isUgoira, int pageIndex) throws Exception;
/**
* 检查某一作品是否为成人内容
* @param illustId 作品Id
* @param isUgoira 是否为动图
* @param pageIndex 指定页数, 设为0或负数则视为单页面作品
* @return 如果为true则为成人作品, 该方法将由检测器决定如何为成人作品.
*/
boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex) throws Exception;
/**
* 检查某一作品是否为成人内容
* @param illustId 作品Id
* @param isUgoira 是否为动图
* @param pageIndex 指定页数, 设为0或负数则视为单页面作品
* @param threshold 指数阀值, 当等于或大于该阀值时返回true.
* @return 如果为true则为成人作品, 该方法将由 threshold 参数决定是否为成人作品(如果超过阈值, 则为true).
*/
boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex, double threshold) throws Exception;
}

View File

@ -1,69 +0,0 @@
package net.lamgc.cgj.pixiv;
/**
* 插图(集)的信息
*/
public class IllustInfo{
/**
* 插图(集)ID
*/
public final int illustID;
/**
* 第几页
*/
public final int page;
/**
* 插图标题
*/
public final String title;
/**
* 插图说明
*/
public final String description;
/**
* 插图标签
*/
public final String[] tags;
/**
* 插图图片长度
*/
public final int width;
/**
* 插图图片高度
*/
public final int height;
/**
* 作者名
*/
public final String authorName;
/**
* 作者用户ID
*/
public final int authorUserID;
public IllustInfo(int illustID, String title, String description, String[] tags, int width, int height, String authorName, int authorUserID){
this(illustID, 0, title, description, tags, width, height, authorName, authorUserID);
}
public IllustInfo(int illustID, int p, String title, String description, String[] tags, int width, int height, String authorName, int authorUserID){
this.illustID = illustID;
this.page = p;
this.title = title;
this.description = description;
this.tags = tags;
this.width = width;
this.height = height;
this.authorName = authorName;
this.authorUserID = authorUserID;
}
}

View File

@ -0,0 +1,65 @@
package net.lamgc.cgj.pixiv;
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
import java.io.IOException;
/**
* 使用ModerateContent服务开发的检测器.<br/>
* ModerateContent: www.moderatecontent.com
*/
public class ModerateContentDetector implements AdultContentDetector {
private final static HttpClient httpClient = HttpClientBuilder.create().build();
private final static Gson gson = new Gson();
private final String requestUrl;
private final static String API_URL = "https://www.moderatecontent.com/api/v2?key={key}&url=https://pixiv.cat/";
/**
* 创建一个使用ModerateContent鉴黄服务的检测器
* @param apiKey API密钥
*/
public ModerateContentDetector(String apiKey) {
requestUrl = API_URL.replace("{key}", apiKey);
}
private JsonObject accessInterface(int illustId, boolean isUgoira, int pageIndex) throws IOException {
HttpResponse response;
if(pageIndex <= 0) {
response = httpClient.execute(new HttpGet(requestUrl + illustId + (isUgoira ? ".gif" : ".jpg")));
} else {
response = httpClient.execute(new HttpGet(requestUrl + illustId + "-" + pageIndex + (isUgoira ? ".gif" : ".jpg")));
}
if(response.getStatusLine().getStatusCode() != 200) {
throw new IOException("Http response error: " + response.getStatusLine());
}
JsonObject result = gson.fromJson(EntityUtils.toString(response.getEntity()), JsonObject.class);
if (result.get("error_code").getAsInt() != 0) {
throw new IOException("Interface result error: " + (result.has("error") ? result.get("error").getAsString() : "(error message is empty)"));
}
return result;
}
@Override
public double detect(int illustId, boolean isUgoira, int pageIndex) throws IOException {
return accessInterface(illustId, isUgoira, pageIndex).getAsJsonObject("predictions").get("adult").getAsDouble();
}
@Override
public boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex) throws IOException {
return accessInterface(illustId, isUgoira, pageIndex).get("rating_index").getAsInt() == 3;
}
@Override
public boolean isAdultContent(int illustId, boolean isUgoira, int pageIndex, double threshold) throws IOException {
return detect(illustId, isUgoira, pageIndex) >= threshold;
}
}

View File

@ -1,267 +0,0 @@
package net.lamgc.cgj.pixiv;
import com.google.gson.JsonArray;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.conn.ConnectionPoolTimeoutException;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.*;
public class Pixiv {
/**
* illust Link
*/
public final static String ATTR_LINK = "link";
/**
* illust Id
*/
public final static String ATTR_ILLUST_ID = "illustId";
/**
* illust Title
*/
public final static String ATTR_TITLE = "title";
/**
* illust Author Name
*/
public final static String ATTR_AUTHOR_NAME = "authorName";
/**
* illust Author UserID
*/
public final static String ATTR_AUTHOR_ID = "authorId";
private final HttpClient httpClient;
public Pixiv(HttpClient client){
this.httpClient = client;
}
/**
* 使用帐号密码登录Pixiv
* @param PixivID Pixiv账户登录名
* @param Password Pixiv帐号密码
* @throws IOException 当登录连接出现异常时抛出
* @deprecated {@link PixivSession#Login(String, String)} 已经废除, 故本方法不可用
*/
public Pixiv(String PixivID, String Password) throws IOException {
this(PixivID, Password, null);
}
/**
* 使用帐号密码登录Pixiv
* @param PixivID Pixiv账户登录名
* @param Password Pixiv帐号密码
* @param proxy 代理设置
* @throws IOException 当登录连接出现异常时抛出
* @deprecated {@link PixivSession#Login(String, String)} 已经废除, 故本方法不可用
*/
public Pixiv(String PixivID, String Password, HttpHost proxy) throws IOException {
PixivSession pixivSession = new PixivSession(proxy, null);
if(pixivSession.Login(PixivID, Password)){
System.out.println("P站登录成功!");
}else{
System.out.println("P站登录失败!错误信息: " + pixivSession.getErrMsg());
throw new RuntimeException(pixivSession.getErrMsg());
}
//httpClient = pixivSession.getHttpClient();
httpClient = HttpClientBuilder.create()
.setDefaultCookieStore(pixivSession.getCookieStore())
.build();
}
/**
* 获取首页推荐列表
* @return 首页推荐列表, 一个Map对应一个推荐项, 使用<code>ATTR_</code>开头常量访问即可
* @throws IOException
*/
public List<Map<String, String>> getRecommend() throws IOException {
HttpGet getRequest = new HttpGet(PixivURL.PIXIV_INDEX_URL);
HttpResponse response = httpClient.execute(getRequest);
String pageAsXML = EntityUtils.toString(response.getEntity(),"utf-8");
//获取推荐图列表(li)
//System.out.println(pageAsXML);
Document document = Jsoup.parse(pageAsXML);
List<String> links = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-thumbnail-link").eachAttr("href");
List<String> illustId = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-thumbnail-link").eachAttr("data-gtm-recommend-illust-id");
List<String> title = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-title>h1").eachAttr("title");
List<String> authorName = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-user-name").eachText();
List<String> authorId = document.select("._image-items.gtm-illust-recommend-zone>li>.gtm-illust-recommend-user-name").eachAttr("data-user_id");
List<Map<String, String>> recommendList = new ArrayList<>();
for(int i = 0; i < links.size(); i++){
//System.out.println(links.get(i));
Map<String, String> info = new HashMap<>();
info.put(ATTR_LINK, PixivURL.PIXIV_INDEX_URL + links.get(i));
info.put(ATTR_ILLUST_ID, illustId.get(i));
info.put(ATTR_TITLE, title.get(i));
info.put(ATTR_AUTHOR_NAME, authorName.get(i));
info.put(ATTR_AUTHOR_ID, authorId.get(i));
recommendList.add(info);
}
return recommendList;
}
public String[] getAllDownloadLink(int illustID) throws IOException {
HttpGet illustPage = new HttpGet(PixivURL.PIXIV_ILLUST_API_URL.replaceAll("\\{illustId}", String.valueOf(illustID)));
HttpResponse response = httpClient.execute(illustPage);
String pageAsXML = EntityUtils.toString(response.getEntity(),"utf-8");
//System.out.println(pageAsXML);
JsonObject resultObj = (JsonObject) new JsonParser().parse(pageAsXML);
if(!resultObj.get("error").getAsBoolean()){
JsonArray bodyArray = resultObj.get("body").getAsJsonArray();
int length = bodyArray.size();
String[] result = new String[length];
for(int i = 0; i < length; i++){
JsonObject childObj = bodyArray.get(i).getAsJsonObject();
result[i] = childObj.get("urls").getAsJsonObject().get("original").getAsString();
}
return result;
}else{
return null;
}
}
/**
* 下载P站图片
* @param illustID 插图ID
* @return 成功返回图片输入流,失败或为多图则返回null
*/
public InputStream[] downloadIllustImage(int illustID) throws IOException {
String[] links = getAllDownloadLink(illustID);
List<InputStream> inputStreamList = new ArrayList<>();
int count = 1;
boolean retry = false;
for(int i = 0; i < links.length; i++){
try {
long sleepTime = (new Random().nextInt(4) + 2) * 1000;
System.out.println("nextTime: " + (float)(sleepTime / 1000));
Thread.sleep(sleepTime);
} catch (InterruptedException ignored) {}
String link = links[i];
System.out.print("page:" + count++ + "/" + links.length + " ...");
HttpGet imgGet = new HttpGet(link);
//关键!如果不加上Referer的话,会返回403
imgGet.setHeader("Referer", PixivURL.PIXIV_ILLUST_MEDIUM_URL.replaceAll("\\{illustId}", String.valueOf(illustID)));
RequestConfig config = RequestConfig.custom()
.setConnectTimeout(20 * 1000)
.setConnectionRequestTimeout(20 * 1000)
.setSocketTimeout(30 * 1000)
.build();
imgGet.setConfig(config);
HttpResponse response;
try {
response = httpClient.execute(imgGet);
}catch(ConnectionPoolTimeoutException e){
if(retry){
retry = false;
System.out.println("获取失败,跳过...");
continue;
}
System.out.println("连接超时,重新获取...");
retry = true;
i--;
continue;
}
retry = false;
ByteArrayOutputStream cacheOutputStream = new ByteArrayOutputStream((int)response.getEntity().getContentLength());
InputStream content = response.getEntity().getContent();
int readLength;
byte[] cache = new byte[4096];
while((readLength = content.read(cache)) != -1){
cacheOutputStream.write(cache, 0, readLength);
}
byte[] data = cacheOutputStream.toByteArray();
//System.out.println("读到数据: " + data.length);
inputStreamList.add(new ByteArrayInputStream(data));
System.out.println("done!length: " + response.getEntity().getContentLength() + ")");
}
return inputStreamList.toArray(new InputStream[0]);
}
/**
* 下载P站图片
* @param illustID 插图ID
* @return 成功返回图片输入流,失败或为多图则返回null
*/
public InputStream downloadIllustImages(int illustID){
throw new UnsupportedOperationException();
}
/**
* 通过解析插图详情页获取
* - 插图标题
* - 插图作者(及其UserId)
* - 插图上传时间
* - 插图标签(原始标签)
* ...
* @return 成功返回IllustInfo对象,失败返回null
*/
public IllustInfo[] getIllustInfo(int[] illustIDs) throws IOException {
//获取Api
HttpGet apiRequest = new HttpGet(PixivURL.getPixivIllustInfoAPI(illustIDs));
HttpResponse response = httpClient.execute(apiRequest);
String resultText = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
System.out.println(resultText);
JsonObject resultObj = ((JsonObject) new JsonParser().parse(resultText));
if(resultObj.get("error").getAsBoolean()){
System.err.println("获取失败!");
return null;
}
List<IllustInfo> illustInfoList = new ArrayList<>();
JsonArray illustArray = resultObj.get("body").getAsJsonObject().get("illusts").getAsJsonArray();
illustArray.forEach(jsonElement -> {
JsonObject illustInfoObj = jsonElement.getAsJsonObject();
JsonArray tagsArray = illustInfoObj.get("tags").getAsJsonArray();
String[] tags = new String[tagsArray.size()];
for(int i = 0; i < tags.length; i++){
tags[i] = tagsArray.get(i).getAsString();
}
//TODO: 通过不需要作者id就能获取图片信息的api无法获取图片尺寸
IllustInfo illustInfo = new IllustInfo(
illustInfoObj.get("workId").getAsInt(),
illustInfoObj.get("title").getAsString(),
null,
tags,
-1,
-1,
illustInfoObj.get("userName").getAsString(),
illustInfoObj.get("userId").getAsInt()
);
});
return null;
}
/**
* 获取指定用户的所有插画
*/
public void getUserAllIllustTest() {
}
}

View File

@ -4,7 +4,9 @@ import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import com.google.gson.reflect.TypeToken;
import io.netty.handler.codec.http.HttpHeaderNames;
import net.lamgc.cgj.exception.HttpRequestException;
import org.apache.http.Header;
import org.apache.http.HttpHost;
import org.apache.http.HttpRequest;
@ -30,9 +32,10 @@ import java.util.*;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.BiConsumer;
@SuppressWarnings("ALL")
public class PixivDownload {
private final static Logger log = LoggerFactory.getLogger("PixivDownload");
private final static Logger log = LoggerFactory.getLogger(PixivDownload.class);
private final HttpClient httpClient;
@ -114,10 +117,10 @@ public class PixivDownload {
}
}
} while(!document.select(".pager-container>.next").isEmpty());
log.debug("获取完成.");
log.trace("获取完成.");
AtomicInteger count = new AtomicInteger(1);
linkList.forEach(link -> {
log.debug("Next Link [{}]: {}", count.getAndIncrement(), link);
log.trace("Next Link [{}]: {}", count.getAndIncrement(), link);
InputStream imageInputStream = null;
int tryCount = 0;
do {
@ -132,9 +135,9 @@ public class PixivDownload {
} while(imageInputStream == null);
try(InputStream imageInput = new BufferedInputStream(imageInputStream, 256 * 1024)) {
log.debug("调用回调方法...");
log.trace("调用回调方法...");
fn.accept(link, imageInput);
log.debug("回调方法调用完成.");
log.trace("回调方法调用完成.");
} catch (IOException e) {
log.error("图片获取失败", e);
}
@ -218,8 +221,8 @@ public class PixivDownload {
int authorId = rankInfo.get("user_id").getAsInt();
String authorName = rankInfo.get("user_name").getAsString();
String title = rankInfo.get("title").getAsString();
log.debug("当前到第 {}/{} 名(总共 {} 名), IllustID: {}, Author: ({}) {}, Title: {}", rank, rankStart + range - 1, range, illustId, authorId, authorName, title);
log.debug("正在获取PagesLink...");
log.trace("当前到第 {}/{} 名(总共 {} 名), IllustID: {}, Author: ({}) {}, Title: {}", rank, rankStart + range - 1, range, illustId, authorId, authorName, title);
log.trace("正在获取PagesLink...");
List<String> linkList;
try {
linkList = getIllustAllPageDownload(httpClient, this.cookieStore, illustId, quality);
@ -234,14 +237,14 @@ public class PixivDownload {
log.debug("PagesLink 获取完成, 总数: {}", linkList.size());
for (int pageIndex = 0; pageIndex < linkList.size(); pageIndex++) {
String downloadLink = linkList.get(pageIndex);
log.debug("当前Page: {}/{}", pageIndex + 1, linkList.size());
log.trace("当前Page: {}/{}", pageIndex + 1, linkList.size());
try(InputStream imageInputStream = new BufferedInputStream(getImageAsInputStream(HttpClientBuilder.create().build(), downloadLink), 256 * 1024)) {
fn.download(rank, downloadLink, rankInfo.deepCopy(), imageInputStream);
} catch(IOException e) {
log.error("下载插画时发生异常", e);
return;
}
log.debug("完成.");
log.trace("完成.");
}
});
}
@ -256,15 +259,15 @@ public class PixivDownload {
public static List<JsonObject> getRanking(List<JsonObject> rankingList, int rankStart, int range) {
log.debug("正在读取JsonArray...(rankStart: {}, range: {})", rankStart, range);
ArrayList<JsonObject> results = new ArrayList<>(rankingList.size());
for (int rankIndex = rankStart; rankIndex < rankingList.size() && rankIndex < range; rankIndex++) {
JsonElement jsonElement = rankingList.get(rankIndex);
for (int rankIndex = rankStart; rankIndex < rankStart + range; rankIndex++) {
JsonElement jsonElement = rankingList.get(rankIndex - rankStart);
JsonObject rankInfo = jsonElement.getAsJsonObject();
int rank = rankInfo.get("rank").getAsInt();
int illustId = rankInfo.get("illust_id").getAsInt();
int authorId = rankInfo.get("user_id").getAsInt();
String authorName = rankInfo.get("user_name").getAsString();
String title = rankInfo.get("title").getAsString();
log.debug("Array-当前到第 {}/{} 名(总共 {} 名), IllustID: {}, Author: ({}) {}, Title: {}", rank, rankStart + range, range, illustId, authorId, authorName, title);
log.trace("Array-当前到第 {}/{} 名(总共 {} 名), IllustID: {}, Author: ({}) {}, Title: {}", rank, rankStart + range, range, illustId, authorId, authorName, title);
results.add(rankInfo);
}
log.debug("JsonArray读取完成.");
@ -279,90 +282,64 @@ public class PixivDownload {
* @return 返回List对象
*/
public static List<JsonObject> getRanking(JsonArray rankingArray, int rankStart, int range) {
log.debug("正在读取JsonArray...(rankStart: {}, range: {})", rankStart, range);
ArrayList<JsonObject> results = new ArrayList<>(rankingArray.size());
for (int rankIndex = rankStart; rankIndex < rankingArray.size() && rankIndex < range; rankIndex++) {
JsonElement jsonElement = rankingArray.get(rankIndex);
JsonObject rankInfo = jsonElement.getAsJsonObject();
int rank = rankInfo.get("rank").getAsInt();
int illustId = rankInfo.get("illust_id").getAsInt();
int authorId = rankInfo.get("user_id").getAsInt();
String authorName = rankInfo.get("user_name").getAsString();
String title = rankInfo.get("title").getAsString();
log.debug("Array-当前到第 {}/{} 名(总共 {} 名), IllustID: {}, Author: ({}) {}, Title: {}", rank, rankStart + range, range, illustId, authorId, authorName, title);
results.add(rankInfo);
}
log.debug("JsonArray读取完成.");
return results;
List<JsonObject> list = new Gson().fromJson(rankingArray, new TypeToken<List<JsonObject>>(){}.getType());
return getRanking(list, rankStart, range);
}
/**
* 获取排行榜
* 获取排行榜.
* <p>注意: 如果范围实际上没超出, 但返回排行榜不足, 会导致与实际请求的数量不符, 需要检查</p>
* @param contentType 排行榜类型
* @param mode 排行榜模式
* @param time 查询时间
* @param rankStart 开始排名, 从1开始
* @param range 取范围
* @return 成功返回有值List, 失败且无异常返回空
* @throws IOException 获取异常时抛出
* @throws IllegalArgumentException 当{@linkplain net.lamgc.cgj.pixiv.PixivURL.RankingContentType RankingContentType}
* 与{@linkplain net.lamgc.cgj.pixiv.PixivURL.RankingMode RankingMode}互不兼容时抛出
* @throws IndexOutOfBoundsException 当排行榜选取范围超出排行榜范围时抛出(排行榜范围为 1 ~ 500 名)
* @throws IOException 当Http请求发生异常时抛出, 或Http请求响应码非200时抛出
*/
public List<JsonObject> getRanking(PixivURL.RankingContentType contentType, PixivURL.RankingMode mode,
Date time, int rankStart, int range) throws IOException {
if(rankStart <= 0) {
throw new IllegalArgumentException("rankStart cannot be less than or equal to zero");
}
if(range <= 0) {
throw new IllegalArgumentException("range cannot be less than or equal to zero");
}
if(!contentType.isSupportedMode(mode)) {
Date time, int rankStart, int range) throws IOException {
Objects.requireNonNull(time);
if(!Objects.requireNonNull(contentType).isSupportedMode(Objects.requireNonNull(mode))) {
throw new IllegalArgumentException("ContentType不支持指定的RankingMode: ContentType: " + contentType.name() + ", Mode: " + mode.name());
} else if(rankStart <= 0) {
throw new IndexOutOfBoundsException("rankStart cannot be less than or equal to zero: " + rankStart);
} else if(range <= 0) {
throw new IndexOutOfBoundsException("range cannot be less than or equal to zero:" + range);
} else if(rankStart + range - 1 > 500) {
throw new IndexOutOfBoundsException("排名选取范围超出了排行榜范围: rankStart=" + rankStart + ", range=" + range + ", length:" + (rankStart + range - 1));
}
int startPage = (int) Math.ceil(rankStart / 50F);
int requestFrequency = (int) Math.ceil((rankStart + (range - 1)) / 50F);
int surplusQuantity = range;
boolean firstRequest = true;
int startPages = (int) Math.max(1, Math.floor(rankStart / 50F));
int endPages = (int) Math.min(10, Math.ceil((rankStart + range) / 50F));
int startIndex = rankStart - 1;
int count = 0;
Gson gson = new Gson();
ArrayList<JsonObject> results = new ArrayList<>();
for (int requestCount = startPage; requestCount <= requestFrequency && requestCount <= 10; requestCount++) {
int rangeStart = (requestCount - 1) * 50 + 1;
log.debug("正在请求第 {} 到 {} 位排名榜 (第{}次请求, 共 {} 次)", rangeStart, rangeStart + 49, requestCount - startPage + 1, requestFrequency - startPage + 1);
HttpGet request = createHttpGetRequest(PixivURL.getRankingLink(contentType, mode, time, requestCount, true));
log.debug("Request URL: {}", request.getURI());
ArrayList<JsonObject> results = new ArrayList<>(range);
boolean canNext = true;
for (int pageIndex = startPages; canNext && pageIndex <= endPages && count < range; pageIndex++) {
HttpGet request = createHttpGetRequest(PixivURL.getRankingLink(contentType, mode, time, pageIndex, true));
log.trace("RequestUri: {}", request.getURI());
HttpResponse response = httpClient.execute(request);
String content = EntityUtils.toString(response.getEntity());
log.debug("Content: " + content);
JsonObject contentObject = gson.fromJson(content, JsonObject.class);
if(contentObject.has("error")) {
log.warn("接口报错, 返回信息: {}", contentObject.get("error").getAsString());
break;
String responseBody = EntityUtils.toString(response.getEntity());
log.trace("ResponseBody: {}", responseBody);
if(response.getStatusLine().getStatusCode() != 200) {
throw new HttpRequestException(response.getStatusLine(), responseBody);
}
JsonArray rankingArray = contentObject.getAsJsonArray("contents");
log.debug("正在解析数据...");
//需要添加一个总量, 否则会完整跑完一次.
//检查是否为最后一次请求,和剩余量有多少
int firstRequestStartIndex = (rankStart % 50) - 1;
for (int rankIndex = firstRequest ? firstRequestStartIndex : 0; rankIndex < rankingArray.size() && surplusQuantity > 0; rankIndex++, surplusQuantity--) {
JsonElement jsonElement = rankingArray.get(rankIndex);
JsonObject rankInfo = jsonElement.getAsJsonObject();
int rank = rankInfo.get("rank").getAsInt();
int illustId = rankInfo.get("illust_id").getAsInt();
int authorId = rankInfo.get("user_id").getAsInt();
String authorName = rankInfo.get("user_name").getAsString();
String title = rankInfo.get("title").getAsString();
log.debug("Download-当前到第 {}/{} 名(总共 {} 名), IllustID: {}, Author: ({}) {}, Title: {}", rank, rankStart + range - 1, range, illustId, authorId, authorName, title);
results.add(rankInfo);
JsonObject resultObject = gson.fromJson(responseBody, JsonObject.class);
canNext = resultObject.get("next").getAsJsonPrimitive().isNumber();
JsonArray resultArray = resultObject.getAsJsonArray("contents");
for (int resultIndex = startIndex; resultIndex < resultArray.size() && count < range; resultIndex++, count++) {
results.add(resultArray.get(resultIndex).getAsJsonObject());
}
firstRequest = false;
log.debug("第 {} 到 {} 位排名榜完成. (第{}次请求)", rangeStart, rangeStart + 49, requestCount);
}
if(requestFrequency > 10) {
log.warn("请求的排名榜范围超出所支持的范围, 已终止请求.");
// 重置索引
startIndex = 0;
}
return results;
}
@ -377,7 +354,7 @@ public class PixivDownload {
HttpResponse response = httpClient.execute(request);
if(response.getStatusLine().getStatusCode() != 200) {
throw new IOException("Http响应码非200: " + response.getStatusLine());
throw new HttpRequestException(response);
}
Document document = Jsoup.parse(EntityUtils.toString(response.getEntity()));
@ -424,8 +401,8 @@ public class PixivDownload {
if(resultObject.get("error").getAsBoolean()) {
String message = resultObject.get("message").getAsString();
log.debug("请求错误, 错误信息: {}", message);
throw new IOException(message);
log.warn("作品页面接口请求错误, 错误信息: {}", message);
throw new HttpRequestException(response.getStatusLine(), resultObject.toString());
}
JsonArray linkArray = resultObject.getAsJsonArray("body");
@ -463,13 +440,14 @@ public class PixivDownload {
THUMB_MINI
}
/**
* 获取帐号所有的收藏插画,并以输入流形式提供
* @return 获取所有链接的InputStream, 请注意关闭InputStream
* @throws IOException 当获取时发生异常则直接抛出
* @deprecated 该方法可能会导致已经打开的InputStream超时, 使图片获取失败,
* 请直接使用{@linkplain #getCollectionAsInputStream(PageQuality, BiConsumer)}
*/
@Deprecated
public Set<Map.Entry<String, InputStream>> getCollectionAsInputStream(PageQuality quality) throws IOException {
HashSet<Map.Entry<String, InputStream>> illustInputStreamSet = new HashSet<>();
getCollectionAsInputStream(quality, (link, inputStream) -> illustInputStreamSet.add(new AbstractMap.SimpleEntry<>(link, inputStream)));
@ -497,9 +475,11 @@ public class PixivDownload {
request.addHeader(HttpHeaderNames.REFERER.toString(), referer);
HttpResponse response = httpClient.execute(request);
log.debug("response: {}", response);
log.debug("Content Length: {}KB", Float.parseFloat(response.getFirstHeader(HttpHeaderNames.CONTENT_LENGTH.toString()).getValue()) / 1024F);
log.debug("{}", response.getFirstHeader(HttpHeaderNames.CONTENT_TYPE.toString()));
log.trace("response: {}", response);
log.trace("Content Length: {}KB",
Float.parseFloat(response.getFirstHeader(HttpHeaderNames.CONTENT_LENGTH.toString()).getValue()) / 1024F
);
log.trace(response.getFirstHeader(HttpHeaderNames.CONTENT_TYPE.toString()).toString());
return response.getEntity().getContent();
}
@ -520,69 +500,31 @@ public class PixivDownload {
} else {
return false;
}
}
/**
* 获取作品信息
* @param illustId 作品ID
* @return 成功获取返回JsonObject, 失败返回null, <br/>
* Json示例: <br/>
* <pre>
* {
* "illustId": "79584670",
* "illustTitle": "このヤンキーはウブすぎる",
* "id": "79584670",
* "title": "このヤンキーはウブすぎる",
* "illustType": 1,
* "xRestrict": 0,
* "restrict": 0,
* "sl": 2,
* "url": "https://i.pximg.net/c/360x360_70/img-master/img/2020/02/19/00/38/23/79584670_p0_square1200.jpg",
* "description": "",
* "tags": [
* "漫画",
* "オリジナル",
* "創作",
* "創作男女",
* "コロさん、ポリさん此方です!",
* "恋の予感",
* "あまずっぺー",
* "交換日記",
* "続編希望!!",
* "オリジナル10000users入り"
* ],
* "userId": "4778293",
* "userName": "隈浪さえ",
* "width": 3288,
* "height": 4564,
* "pageCount": 4,
* "isBookmarkable": true,
* "bookmarkData": null,
* "alt": "#オリジナル このヤンキーはウブすぎる - 隈浪さえ的漫画",
* "isAdContainer": false,
* "profileImageUrl": "https://i.pximg.net/user-profile/img/2019/12/04/18/56/19/16639046_fea29ce38ea89b0cb2313b40b3a72f9a_50.jpg",
* "type": "illust"
* }
* </pre>
* @return 成功获取返回JsonObject, 失败返回null.
* @throws IOException 当请求发生异常, 或接口返回错误信息时抛出.
* @throws NoSuchElementException 当该作品不存在时抛出异常
*/
public JsonObject getIllustInfoByIllustId(int illustId) throws IOException {
public JsonObject getIllustInfoByIllustId(int illustId) throws IOException, NoSuchElementException {
HttpGet request = createHttpGetRequest(PixivURL.getPixivIllustInfoAPI(illustId));
HttpResponse response = httpClient.execute(request);
String responseStr = EntityUtils.toString(response.getEntity());
log.debug("Response Content: {}", responseStr);
log.trace("Response Content: {}", responseStr);
JsonObject responseObj = new Gson().fromJson(responseStr, JsonObject.class);
if(responseObj.get("error").getAsBoolean()) {
throw new IOException(responseObj.get("message").getAsString());
throw new HttpRequestException(response.getStatusLine(), responseStr);
}
JsonArray illustsArray = responseObj.getAsJsonObject("body").getAsJsonArray("illusts");
if(illustsArray.size() == 1) {
return illustsArray.get(0).getAsJsonObject();
} else {
return null;
throw new NoSuchElementException("No work found: " + illustId);
}
}
@ -630,12 +572,29 @@ public class PixivDownload {
return null;
}
/**
* 通过预加载数据获取作品类型
* @param illustId 作品Id
* @param preLoadDataObject 预加载数据(IllustInfo也可以)
* @return 如果存在illustType属性, 则返回对应项, 如没有, 或数据内不存在指定作品id的数据, 返回null
*/
public static PixivIllustType getIllustTypeByPreLoadData(int illustId, JsonObject preLoadDataObject) {
JsonObject illustData;
JsonObject illustData = null;
if(preLoadDataObject.has("illust")) {
illustData = preLoadDataObject.getAsJsonObject("illust").getAsJsonObject(String.valueOf(illustId));
} else if(preLoadDataObject.has(String.valueOf(illustId))) {
illustData = preLoadDataObject.getAsJsonObject(String.valueOf(illustId));
} else if(preLoadDataObject.has("body")) { // 解析IllustInfo
for (JsonElement jsonElement : preLoadDataObject.getAsJsonObject("body").getAsJsonArray("illusts")) {
JsonObject illustInfo = jsonElement.getAsJsonObject();
if (illustInfo.get("illustId").getAsInt() == illustId) {
illustData = illustInfo;
break;
}
}
if(illustData == null) {
return null;
}
} else {
illustData = preLoadDataObject;
}

View File

@ -9,7 +9,14 @@ import java.util.Date;
import java.util.HashSet;
import java.util.Objects;
public class PixivSearchBuilder {
/**
* Pixiv搜索URL构造器
* <p>该构造器通过分析Pixiv搜索链接可用的参数而开发, 对搜索链接的构造有高度自定义能力.</p>
* @author LamGC
* @see PixivURL#PIXIV_SEARCH_CONTENT_URL
*/
@SuppressWarnings("unused")
public class PixivSearchLinkBuilder {
private final String content;
@ -19,8 +26,8 @@ public class PixivSearchBuilder {
private SearchOrder searchOrder = SearchOrder.DATE_D;
private SearchContentOption searchContentOption = SearchContentOption.ALL;
private HashSet<String> includeKeywords = new HashSet<>(0);
private HashSet<String> excludeKeywords = new HashSet<>(0);
private final HashSet<String> includeKeywords = new HashSet<>(0);
private final HashSet<String> excludeKeywords = new HashSet<>(0);
private int page = 1;
@ -35,7 +42,7 @@ public class PixivSearchBuilder {
private Date startDate = null;
private Date endDate = null;
public PixivSearchBuilder(String searchContent) {
public PixivSearchLinkBuilder(String searchContent) {
this.content = Objects.requireNonNull(searchContent);
}
@ -92,7 +99,7 @@ public class PixivSearchBuilder {
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
PixivSearchBuilder that = (PixivSearchBuilder) o;
PixivSearchLinkBuilder that = (PixivSearchLinkBuilder) o;
return page == that.page &&
wgt == that.wgt &&
hgt == that.hgt &&
@ -134,7 +141,7 @@ public class PixivSearchBuilder {
@Override
public String toString() {
return "PixivSearchBuilder{" +
return "PixivSearchLinkBuilder{" +
"content='" + content + '\'' +
", searchArea=" + searchArea +
", searchMode=" + searchMode +
@ -154,19 +161,11 @@ public class PixivSearchBuilder {
'}';
}
public PixivSearchBuilder setSearchArea(SearchArea searchArea) {
public PixivSearchLinkBuilder setSearchArea(SearchArea searchArea) {
this.searchArea = Objects.requireNonNull(searchArea);
return this;
}
/**
* 获取搜索区域
* @return 返回搜索区域对象
*/
public SearchArea getSearchArea() {
return searchArea;
}
/**
* 获取搜索条件.
* @return 搜索条件内容
@ -195,50 +194,50 @@ public class PixivSearchBuilder {
return searchContent.toString();
}
public PixivSearchBuilder setSearchMode(SearchMode searchMode) {
public PixivSearchLinkBuilder setSearchMode(SearchMode searchMode) {
this.searchMode = Objects.requireNonNull(searchMode);
return this;
}
public PixivSearchBuilder setSearchType(SearchType searchType) {
public PixivSearchLinkBuilder setSearchType(SearchType searchType) {
this.searchType = Objects.requireNonNull(searchType);
return this;
}
public PixivSearchBuilder setSearchOrder(SearchOrder searchOrder) {
public PixivSearchLinkBuilder setSearchOrder(SearchOrder searchOrder) {
this.searchOrder = Objects.requireNonNull(searchOrder);
return this;
}
public PixivSearchBuilder setSearchContentOption(SearchContentOption searchContentOption) {
public PixivSearchLinkBuilder setSearchContentOption(SearchContentOption searchContentOption) {
this.searchContentOption = Objects.requireNonNull(searchContentOption);
return this;
}
public PixivSearchBuilder setRatioOption(RatioOption ratioOption) {
public PixivSearchLinkBuilder setRatioOption(RatioOption ratioOption) {
this.ratioOption = Objects.requireNonNull(ratioOption);
return this;
}
public PixivSearchBuilder setDateRange(Date startDate, Date endDate) {
public PixivSearchLinkBuilder setDateRange(Date startDate, Date endDate) {
this.startDate = startDate;
this.endDate = endDate;
return this;
}
public PixivSearchBuilder setMaxSize(int width, int height) {
public PixivSearchLinkBuilder setMaxSize(int width, int height) {
this.wgt = width;
this.hgt = height;
return this;
}
public PixivSearchBuilder setMinSize(int width, int height) {
public PixivSearchLinkBuilder setMinSize(int width, int height) {
this.wlt = width;
this.hlt = height;
return this;
}
public PixivSearchBuilder setPage(int pageIndex) {
public PixivSearchLinkBuilder setPage(int pageIndex) {
if (pageIndex <= 0) {
throw new IllegalArgumentException("Invalid pageIndex: " + pageIndex);
}
@ -246,22 +245,22 @@ public class PixivSearchBuilder {
return this;
}
public PixivSearchBuilder addExcludeKeyword(String keyword) {
public PixivSearchLinkBuilder addExcludeKeyword(String keyword) {
excludeKeywords.add(keyword);
return this;
}
public PixivSearchBuilder removeExcludeKeyword(String keyword) {
public PixivSearchLinkBuilder removeExcludeKeyword(String keyword) {
excludeKeywords.remove(keyword);
return this;
}
public PixivSearchBuilder addIncludeKeyword(String keyword) {
public PixivSearchLinkBuilder addIncludeKeyword(String keyword) {
includeKeywords.add(keyword);
return this;
}
public PixivSearchBuilder removeIncludeKeyword(String keyword) {
public PixivSearchLinkBuilder removeIncludeKeyword(String keyword) {
includeKeywords.remove(keyword);
return this;
}

View File

@ -1,171 +0,0 @@
package net.lamgc.cgj.pixiv;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import org.apache.http.Header;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.CookieStore;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicHeader;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
public class PixivSession {
public static final String USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36";
/**
* 全登陆过程的关键,
* 保存会话用的cookieStore!
*/
private CookieStore cookieStore = new BasicCookieStore();
/**
* 可以直接使用的HttpClient对象
*/
private HttpClient httpClient;
/**
* 最后一次登录的错误信息
*/
private String errMsg;
public PixivSession(){
this(null);
}
public PixivSession(CookieStore cookieStore){
this(null, cookieStore);
}
/**
* 创建一个Pixiv登录会话
*/
public PixivSession(HttpHost proxy, CookieStore cookieStore) {
if(cookieStore != null){
this.cookieStore = cookieStore;
}
List<Header> defaultHeader = new ArrayList<>();
defaultHeader.add(new BasicHeader("accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"));
defaultHeader.add(new BasicHeader("user-agent", PixivSession.USER_AGENT));
defaultHeader.add(new BasicHeader("accept-encoding", "gzip, deflate, br"));
defaultHeader.add(new BasicHeader("accept-language", "zh-CN,zh;q=0.9"));
/*defaultHeader.add(new BasicHeader("sec-fetch-mode", "navigate"));
defaultHeader.add(new BasicHeader("sec-fetch-site", "same-origin"));
defaultHeader.add(new BasicHeader("upgrade-insecure-requests", "1"));*/
//创建一个Http访问器
httpClient = HttpClients.custom()
.setDefaultCookieStore(cookieStore)
.setDefaultHeaders(defaultHeader)
.setProxy(proxy)
.build();
}
/**
* 程序自行通过帐号密码登录Pixiv.
* @param PixivID Pixiv帐号
* @param Password Pixiv密码
* @return 登录成功返回true
* @throws IOException 当登录抛出异常时返回
* @deprecated Pixiv已经新增Google人机验证, 程序已无法自行登录Pixiv
*/
public boolean Login(String PixivID, String Password) throws IOException {
// 获取登录接口所需的PostKey
String post_key = getPostKey();
HttpPost postRequest = new HttpPost(PixivURL.PIXIV_LOGIN_URL); //https://accounts.pixiv.net/api/login?lang=zh
List<NameValuePair> params = new ArrayList<>();
params.add(new BasicNameValuePair("pixiv_id", PixivID));
params.add(new BasicNameValuePair("password", Password));
params.add(new BasicNameValuePair("post_key", post_key));
//Form编码表单,作为Post的数据
postRequest.setEntity(new UrlEncodedFormEntity(params, StandardCharsets.UTF_8));
//启动访问
HttpResponse response = httpClient.execute(postRequest);
//获取接口返回数据
String httpXML = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
System.out.println(httpXML);
JsonObject responseJson = (JsonObject) new JsonParser().parse(httpXML);
if(!responseJson.get("error").getAsBoolean() && !responseJson.get("body").getAsJsonObject().has("validation_errors")){
errMsg = null;
return true;
}else{
errMsg = responseJson.get("body").getAsJsonObject().get("validation_errors").toString();
//System.err.println("登录失败MSG: " + errMsg);
return false;
}
}
/**
* 登录前准备, 获取PostKey
* @return Post_Key
*/
private String getPostKey() throws IOException {
//创建请求,获取PostKey
HttpGet getRequest = new HttpGet(PixivURL.PIXIV_LOGIN_PAGE_URL);
//设置请求
//getRequest.setConfig(config);
getRequest.setHeader("User-Agent", USER_AGENT);
//启动访问
HttpResponse response = httpClient.execute(getRequest);
//获取网页内容
String pageAsXML = EntityUtils.toString(response.getEntity(),"utf-8");
//创建Http解析器
Document document = Jsoup.parse(pageAsXML);
//获取init-config内容
String init_config = document.getElementById("init-config").val();
//System.out.println(init_config);
//创建Json解析器解析init-config
JsonObject initConfigObj = (JsonObject) new JsonParser().parse(init_config);
//检查是否有postKey
if(!initConfigObj.has("pixivAccount.postKey")){
throw new RuntimeException("postKey获取失败!可能是Pixiv修改了登录过程!");
}
//获取postKey
return initConfigObj.get("pixivAccount.postKey").getAsString();
}
/**
* 获取CookieStore
* @return CookieStore
*/
public CookieStore getCookieStore(){
return cookieStore;
}
/**
* 获取可直接使用的HttpClient对象
* @return 已配置好的HttpClient对象
*/
public HttpClient getHttpClient(){
return this.httpClient;
}
public boolean hasError(){
return errMsg == null;
}
/**
* 获取错误信息
* @return 返回登录错误信息
* @deprecated {@link #Login(String, String)}已经废除, 故本接口废除
*/
public String getErrMsg(){
return errMsg;
}
}

View File

@ -1,16 +0,0 @@
package net.lamgc.cgj.pixiv;
public class PixivTag {
public static PixivTag create(String tagName) {
return null;
}
private PixivTag() {
}
}

View File

@ -9,90 +9,101 @@ import java.util.GregorianCalendar;
/**
* 目前已整理的一些Pixiv接口列表
*/
public class PixivURL {
@SuppressWarnings("unused")
public final class PixivURL {
private PixivURL() {}
public static final String PIXIV_INDEX_URL = "https://www.pixiv.net";
public final static String PIXIV_INDEX_URL = "https://www.pixiv.net";
/**
* P站预登陆url
*/
public static final String PIXIV_LOGIN_PAGE_URL = "https://accounts.pixiv.net/login?lang=zh&source=pc&view_type=page&ref=wwwtop_accounts_index";
public final static String PIXIV_LOGIN_PAGE_URL = "https://accounts.pixiv.net/login?lang=zh&source=pc&view_type=page&ref=wwwtop_accounts_index";
/**
* P站登录请求url
*/
public static final String PIXIV_LOGIN_URL = "https://accounts.pixiv.net/api/login?lang=zh";
public final static String PIXIV_LOGIN_URL = "https://accounts.pixiv.net/api/login?lang=zh";
/**
* P站搜索请求url
* @deprecated 该接口已被替换, 请使用{@link PixivSearchLinkBuilder}构造搜索Url
* @see PixivSearchLinkBuilder
*/
private static final String PIXIV_SEARCH_URL = "https://www.pixiv.net/search.php";
@Deprecated
private final static String PIXIV_SEARCH_URL = "https://www.pixiv.net/search.php";
/**
* P站搜索用户url
* 需要替换的参数:
* {nick} - 用户昵称、部分名称
* @deprecated 该接口已被替换, 请使用{@link PixivSearchLinkBuilder}构造搜索Url
* @see PixivSearchLinkBuilder
*/
public static final String PIXIV_SEARCH_USER_URL = PIXIV_SEARCH_URL + "?s_mode=s_usr&nick={nick}";
@Deprecated
public final static String PIXIV_SEARCH_USER_URL = PIXIV_SEARCH_URL + "?s_mode=s_usr&nick={nick}";
/**
* P站搜索插画url
* 需要替换的参数:
* {word} - 插画相关文本
* @deprecated 该接口已被替换, 请使用{@link PixivSearchLinkBuilder}构造搜索Url
* @see PixivSearchLinkBuilder
*/
public static final String PIXIV_SEARCH_TAG_URL = PIXIV_SEARCH_URL + "?s_mode=s_tag&word={word}";
@Deprecated
public final static String PIXIV_SEARCH_TAG_URL = PIXIV_SEARCH_URL + "?s_mode=s_tag&word={word}";
/**
* P站插图下载链接获取url
* 需要替换的文本:
* {illustId} - 插画ID
*/
public static final String PIXIV_ILLUST_API_URL = "https://www.pixiv.net/ajax/illust/{illustId}/pages";
public final static String PIXIV_ILLUST_API_URL = "https://www.pixiv.net/ajax/illust/{illustId}/pages";
/**
* P站用户插图列表获取API
* <p>所需数据在 body属性内的 illusts(属性名,属性值不重要), manga(多图) pickup(精选)</p>
* 需要替换的文本:
* {userId} - 用户ID
*/
//TODO: 所需数据在 body属性内的 illusts(属性名,属性值不重要), manga(多图) pickup(精选)
//{"error":false,"message":"","body":{"illusts":{"74369837":null,"70990542":null,"70608653":null,"69755191":null,"69729450":null,"69729416":null,"69503608":null,"69288766":null,"69083882":null,"69051458":null,"68484200":null,"68216927":null,"68216866":null,"68192333":null,"67915106":null,"67914932":null,"67854803":null,"67854745":null,"67854670":null,"67787211":null,"67772199":null,"67770637":null,"67754861":null,"67754804":null,"67754726":null,"67740486":null,"67740480":null,"67740450":null,"67740434":null,"67726337":null,"67499196":null,"67499163":null,"67499145":null,"67499111":null,"67499085":null,"67499038":null,"67498987":null,"67473178":null,"66271465":null,"63682753":null,"63682697":null,"59385148":null,"59383265":null,"59383240":null,"59383227":null,"59383173":null},"manga":[],"novels":[],"mangaSeries":[],"novelSeries":[],"pickup":[],"bookmarkCount":{"public":{"illust":1,"novel":0},"private":{"illust":0,"novel":0}}}}
public static final String PIXIV_USER_ILLUST_LIST_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/all";
public final static String PIXIV_USER_ILLUST_LIST_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/all";
/**
* 能够同时获取插图信息的用户插图列表获取API
* 需要替换的文本:
* {userId} - 用户ID
*/
public static final String PIXIV_USER_TOP_ILLUST_LIST_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/top";
public final static String PIXIV_USER_TOP_ILLUST_LIST_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/top";
/**
* P站单图详情页url
* 需要替换的文本:
* {illustId} - 插画ID
*/
public static final String PIXIV_ILLUST_MEDIUM_URL = "https://www.pixiv.net/member_illust.php?mode=medium&illust_id={illustId}";
public final static String PIXIV_ILLUST_MEDIUM_URL = "https://www.pixiv.net/member_illust.php?mode=medium&illust_id={illustId}";
/**
* P站多图详情页url
* 需要替换的文本:
* {illustId} - 插画ID
*/
public static final String PIXIV_ILLUST_MANGA_URL = "https://www.pixiv.net/member_illust.php?mode=manga&illust_id={illustId}";
public final static String PIXIV_ILLUST_MANGA_URL = "https://www.pixiv.net/member_illust.php?mode=manga&illust_id={illustId}";
/**
* P站用户页面url
* 需要替换的文本:
* {userId} - 用户ID
*/
public static final String PIXIV_USER_URL = "https://www.pixiv.net/member.php?id={userId}";
public final static String PIXIV_USER_URL = "https://www.pixiv.net/member.php?id={userId}";
/**
* P站插图信息获取API
* 这个API能获取插图基本信息但不能获取大小
* 请使用{@link #getPixivIllustInfoAPI(int[])}获取URL
*/
private static final String PIXIV_GET_ILLUST_INFO_URL = "https://www.pixiv.net/ajax/illust/recommend/illusts?";
private final static String PIXIV_GET_ILLUST_INFO_URL = "https://www.pixiv.net/ajax/illust/recommend/illusts?";
/**
* P站获取用户所有插图ID的Api
@ -100,7 +111,15 @@ public class PixivURL {
* 需要替换的文本:
* {userId} - 用户ID
*/
public static final String PIXIV_GET_USER_ALL_ILLUST_ID_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/all";
public final static String PIXIV_GET_USER_ALL_ILLUST_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/all";
/**
* P站获取用户推荐插画及用户基本数据
* 这个API能获得作者的部分(推荐)作品, 每个作品有详细数据, 还能获取作者主页信息(比如主页说明, 看板图)
* 需要替换的文本:
* {userId} - 用户ID
*/
public final static String PIXIV_GET_USER_TOP_ILLUST_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/top";
/**
* P站标签搜索URL
@ -108,7 +127,7 @@ public class PixivURL {
* 需要替换的文本:
* {content} - 大致tag内容
*/
public static final String PIXIV_TAG_SEARCH_URL = "https://www.pixiv.net/ajax/search/tags/{content}";
public final static String PIXIV_TAG_SEARCH_URL = "https://www.pixiv.net/ajax/search/tags/{content}";
/**
* 获取动图下载链接和拼接数据.
@ -118,11 +137,17 @@ public class PixivURL {
*/
public final static String PIXIV_GET_UGOIRA_META_URL = "https://www.pixiv.net/ajax/illust/{illustId}/ugoira_meta";
/**
* 获取自己帐号的部分数据(目前仅能获取: 关注数, 粉丝数和看板图)
* 需要登录.
*/
public final static String PIXIV_GET_USER_EXTRA_URL = "https://www.pixiv.net/ajax/user/extra";
/**
* 请求时带上需要退出的Cookies
* 无论成功与否都会返回302重定向到{@linkplain #PIXIV_LOGIN_PAGE_URL 登录页面}
*/
public static final String PIXIV_LOGOUT_URL = "https://www.pixiv.net/logout.php";
public final static String PIXIV_LOGOUT_URL = "https://www.pixiv.net/logout.php";
/**
* 构造P站获取插图信息的Api Url
@ -163,6 +188,7 @@ public class PixivURL {
/**
* 查询用户收藏.<br/>
* 该URL返回HTML页面需要进行解析.<br/>
* <p>注意: 该接口需要登陆</p>
* 需要替换的文本:<br/>
* {pageIndex} - 页数, 超出了则结果为空<br/>
*/
@ -186,7 +212,7 @@ public class PixivURL {
public static String getRankingLink(RankingContentType contentType, RankingMode mode, Date time, int pageIndex, boolean json){
StringBuilder linkBuilder = new StringBuilder(PIXIV_RANKING_LINK);
linkBuilder.append("mode=").append(mode == null ? RankingMode.MODE_DAILY.modeParam : mode.modeParam);
if(contentType != null && !contentType.equals(RankingContentType.ALL)){
if(contentType != null && !contentType.equals(RankingContentType.TYPE_ALL)){
linkBuilder.append("&content=").append(contentType.typeName);
}
SimpleDateFormat format = new SimpleDateFormat("yyyyMMdd");
@ -268,7 +294,7 @@ public class PixivURL {
/**
* Pixiv搜索接口.<br/>
* 要使用该链接请使用{@link PixivSearchBuilder}构造链接.<br/>
* 要使用该链接请使用{@link PixivSearchLinkBuilder}构造链接.<br/>
* 需要替换的参数: <br/>
* content - 搜索内容
*/
@ -278,7 +304,7 @@ public class PixivURL {
* 排名榜类型
*/
public enum RankingContentType{
ALL("", RankingMode.values()),
TYPE_ALL("", RankingMode.values()),
/**
* 插画
* 支持的时间类型: 每天, 每周, 每月, 新人
@ -335,8 +361,9 @@ public class PixivURL {
* @param mode 要检查的RankingMode项
* @return 如果支持返回true
*/
@SuppressWarnings("BooleanMethodIsAlwaysInverted")
public boolean isSupportedMode(RankingMode mode) {
return Arrays.binarySearch(supportedMode, mode) != -1;
return Arrays.binarySearch(supportedMode, mode) >= 0;
}
}

View File

@ -1,5 +1,6 @@
package net.lamgc.cgj.pixiv;
import com.google.common.io.ByteStreams;
import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonObject;
@ -7,7 +8,7 @@ import com.squareup.gifencoder.GifEncoder;
import com.squareup.gifencoder.Image;
import com.squareup.gifencoder.ImageOptions;
import io.netty.handler.codec.http.HttpHeaderNames;
import org.apache.commons.io.IOUtils;
import net.lamgc.cgj.exception.HttpRequestException;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
@ -26,12 +27,11 @@ import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
/**
* Pixiv动图构建器.
* 可便捷的接收并处理动图.
* Pixiv动图构建器
*/
public final class PixivUgoiraBuilder {
private final Logger log = LoggerFactory.getLogger(PixivUgoiraBuilder.class.getSimpleName() + "@" + Integer.toHexString(this.hashCode()));
private final Logger log = LoggerFactory.getLogger(this.toString());
private final HttpClient httpClient;
private final JsonObject ugoiraMeta;
@ -47,12 +47,12 @@ public final class PixivUgoiraBuilder {
log.debug("Request Url: {}", request.getURI());
HttpResponse response = httpClient.execute(request);
String bodyStr = EntityUtils.toString(response.getEntity());
log.debug("JsonBodyStr: {}", bodyStr);
log.trace("JsonBodyStr: {}", bodyStr);
JsonObject resultObject = new Gson().fromJson(bodyStr, JsonObject.class);
if(resultObject.get("error").getAsBoolean()) {
String message = resultObject.get("message").getAsString();
log.error("获取动图元数据失败!(接口报错: {})", message);
throw new IOException(message);
throw new HttpRequestException(response.getStatusLine(), bodyStr);
} else if(!resultObject.has("body")) {
String message = "接口返回数据不存在body属性, 可能接口发生改变!";
log.error(message);
@ -83,7 +83,33 @@ public final class PixivUgoiraBuilder {
log.debug("IllustId: {}, UgoiraMeta: {}", this.illustId, this.ugoiraMeta);
}
/**
* 获取动图元数据
* @return 动图元数据, 返回的对象不影响Builder中的meta对象
*/
public JsonObject getUgoiraMeta() {
return this.ugoiraMeta.deepCopy();
}
/**
* 构建动图
* @param original 是否为原图画质
* @return 返回动图数据输入流
* @throws IOException 当获取数据发生异常时抛出
*/
public InputStream buildUgoira(boolean original) throws IOException {
ByteArrayOutputStream bufferOutput = new ByteArrayOutputStream();
buildUgoira(bufferOutput, original);
return new ByteArrayInputStream(bufferOutput.toByteArray());
}
/**
* 构建动图
* @param outputStream 动图输出流
* @param original 是否为原图画质
* @throws IOException 当获取数据发生异常时抛出
*/
public void buildUgoira(OutputStream outputStream, boolean original) throws IOException {
getUgoiraImageSize();
log.debug("动图尺寸信息: Height: {}, Width: {}", height, width);
@ -96,20 +122,18 @@ public final class PixivUgoiraBuilder {
HttpResponse response = httpClient.execute(request);
log.trace("请求已发送, 正在处理响应...");
ZipInputStream zipInputStream = new ZipInputStream(new BufferedInputStream(response.getEntity().getContent(), 64 * 1024));
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
ZipEntry entry;
ByteArrayOutputStream cacheOutputStream = new ByteArrayOutputStream(512);
HashMap<String, InputStream> frameMap = new HashMap<>(frames.size());
while((entry = zipInputStream.getNextEntry()) != null) {
log.trace("ZipEntry {} 正在接收...", entry);
IOUtils.copy(zipInputStream, cacheOutputStream);
ByteStreams.copy(zipInputStream, cacheOutputStream);
frameMap.put(entry.getName(), new ByteArrayInputStream(cacheOutputStream.toByteArray()));
log.trace("ZipEntry {} 已接收完成.", entry);
cacheOutputStream.reset();
}
InputStream firstFrameInput = frameMap.get("000000.jpg");
InputStream firstFrameInput = frameMap.get(frames.get(0).getAsJsonObject().get("file").getAsString());
BufferedImage firstFrame = ImageIO.read(firstFrameInput);
firstFrameInput.reset();
if(width != firstFrame.getWidth() || height != firstFrame.getHeight()) {
@ -135,13 +159,12 @@ public final class PixivUgoiraBuilder {
image.getRGB(0, 0, image.getWidth(), image.getHeight(), rgb, 0, image.getWidth());
log.trace("帧解析完成, 正在插入...");
encoder.addImage(Image.fromRgb(rgb, image.getWidth()), new ImageOptions().setDelay(frameInfo.get("delay").getAsLong(), TimeUnit.MILLISECONDS));
log.debug("帧 {} 插入完成.", frameFileName);
log.trace("帧 {} 插入完成.", frameFileName);
} catch (IOException e) {
log.error("解析帧图片数据时发生异常", e);
}
});
encoder.finishEncoding();
return new ByteArrayInputStream(outputStream.toByteArray());
}
/**
@ -150,15 +173,15 @@ public final class PixivUgoiraBuilder {
private void getUgoiraImageSize() throws IOException {
log.debug("正在从Pixiv获取动图尺寸...");
HttpGet request = new HttpGet(PixivURL.getPixivIllustInfoAPI(illustId));
log.debug("Request Url: {}", request.getURI());
log.trace("Request Url: {}", request.getURI());
HttpResponse response = httpClient.execute(request);
String responseBody = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
log.debug("ResponseBody: {}", responseBody);
log.trace("ResponseBody: {}", responseBody);
JsonObject resultObject = new Gson().fromJson(responseBody, JsonObject.class);
if(resultObject.get("error").getAsBoolean()) {
String message = resultObject.get("message").getAsString();
log.error("接口返回错误: {}", message);
throw new IOException(message);
throw new HttpRequestException(response.getStatusLine(), responseBody);
}
JsonArray illustsArray = resultObject.getAsJsonObject("body").getAsJsonArray("illusts");

View File

@ -1,149 +0,0 @@
package net.lamgc.cgj.proxy;
import com.github.monkeywie.proxyee.intercept.HttpProxyIntercept;
import com.github.monkeywie.proxyee.intercept.HttpProxyInterceptInitializer;
import com.github.monkeywie.proxyee.intercept.HttpProxyInterceptPipeline;
import com.github.monkeywie.proxyee.intercept.common.CertDownIntercept;
import com.github.monkeywie.proxyee.proxy.ProxyConfig;
import com.github.monkeywie.proxyee.server.HttpProxyServer;
import com.github.monkeywie.proxyee.server.HttpProxyServerConfig;
import io.netty.channel.Channel;
import io.netty.handler.codec.http.HttpHeaderNames;
import io.netty.handler.codec.http.HttpHeaders;
import io.netty.handler.codec.http.HttpRequest;
import io.netty.handler.codec.http.HttpResponse;
import org.apache.http.client.CookieStore;
import org.apache.http.impl.cookie.BasicClientCookie;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.net.HttpCookie;
import java.util.Date;
import java.util.List;
/**
* 登录成功后提供CookieStore, 然后由程序自动登录Pixiv
* @author LamGC
*/
public class PixivAccessProxyServer {
private final Logger log = LoggerFactory.getLogger(this.getClass().getSimpleName());
private final HttpProxyServer proxyServer;
private final CookieStore cookieStore;
public PixivAccessProxyServer(CookieStore cookieStore){
this(cookieStore, null);
}
public PixivAccessProxyServer(CookieStore cookieStore, ProxyConfig proxyConfig){
HttpProxyServerConfig config = new HttpProxyServerConfig();
this.cookieStore = cookieStore;
config.setHandleSsl(true);
this.proxyServer = new HttpProxyServer();
this.proxyServer
.serverConfig(config)
.proxyConfig(proxyConfig)
.proxyInterceptInitializer(new HttpProxyInterceptInitializer(){
@Override
public void init(HttpProxyInterceptPipeline pipeline) {
pipeline.addLast(new CertDownIntercept());
pipeline.addLast(new HttpProxyIntercept(){
private boolean match(HttpRequest request){
String host = request.headers().get(HttpHeaderNames.HOST);
return host.equalsIgnoreCase("pixiv.net") || host.contains(".pixiv.net");
}
@Override
public void beforeRequest(Channel clientChannel, HttpRequest httpRequest, HttpProxyInterceptPipeline pipeline) throws Exception {
log.info("URL: " + httpRequest.headers().get(HttpHeaderNames.HOST) + httpRequest.uri());
if(!match(httpRequest)){
super.beforeRequest(clientChannel, httpRequest, pipeline);
return;
}
log.info("正在注入Cookies...");
HttpHeaders requestHeaders = httpRequest.headers();
if(requestHeaders.contains(HttpHeaderNames.COOKIE)){
log.info("原请求存在自带Cookies, 正在清除Cookies...");
log.debug("原Cookies: {}", requestHeaders.getAsString(HttpHeaderNames.COOKIE));
requestHeaders.remove(HttpHeaderNames.COOKIE);
}
StringBuilder cookieBuilder = new StringBuilder();
cookieStore.getCookies().forEach(cookie -> {
if(cookie.isExpired(new Date())){
return;
}
cookieBuilder.append(cookie.getName()).append("=").append(cookie.getValue()).append("; ");
});
log.info("Cookies构造完成, 结果: " + cookieBuilder.toString());
requestHeaders.add(HttpHeaderNames.COOKIE, cookieBuilder.toString());
log.info("Cookies注入完成.");
super.beforeRequest(clientChannel, httpRequest, pipeline);
}
@Override
public void afterResponse(Channel clientChannel, Channel proxyChannel, HttpResponse httpResponse, HttpProxyInterceptPipeline pipeline) throws Exception {
if(!match(pipeline.getHttpRequest())){
super.afterResponse(clientChannel, proxyChannel, httpResponse, pipeline);
return;
}
log.info("正在更新Response Cookie...(Header Name: " + HttpHeaderNames.SET_COOKIE + ")");
List<String> responseCookies = httpResponse.headers().getAll(HttpHeaderNames.SET_COOKIE);
responseCookies.forEach(value -> {
/*if(check(value)){
log.info("黑名单Cookie, 已忽略: " + value);
return;
}*/
log.info("Response Cookie: " + value);
BasicClientCookie cookie = parseRawCookie(value);
cookieStore.addCookie(null);
});
httpResponse.headers().remove(HttpHeaderNames.SET_COOKIE);
super.afterResponse(clientChannel, proxyChannel, httpResponse, pipeline);
}
protected BasicClientCookie parseRawCookie(String rawCookie) {
List<HttpCookie> cookies = HttpCookie.parse(rawCookie);
if (cookies.size() < 1)
return null;
HttpCookie httpCookie = cookies.get(0);
BasicClientCookie cookie = new BasicClientCookie(httpCookie.getName(), httpCookie.getValue());
if (httpCookie.getMaxAge() >= 0) {
Date expiryDate = new Date(System.currentTimeMillis() + httpCookie.getMaxAge() * 1000);
cookie.setExpiryDate(expiryDate);
}
if (httpCookie.getDomain() != null)
cookie.setDomain(httpCookie.getDomain());
if (httpCookie.getPath() != null)
cookie.setPath(httpCookie.getPath());
if (httpCookie.getComment() != null)
cookie.setComment(httpCookie.getComment());
cookie.setSecure(httpCookie.getSecure());
return cookie;
}
});
}
});
}
public void start(int port){
this.proxyServer.start(port);
}
public void close(){
this.proxyServer.close();
}
/**
* 导出CookieStore.
* 注意!该方法导出的CookieStore不适用于ApacheHttpClient, 如需使用则需要进行转换.
* @return CookieStore对象
*/
public CookieStore getCookieStore(){
return this.cookieStore;
}
}

View File

@ -1,31 +0,0 @@
package net.lamgc.cgj.util;
import org.apache.http.client.CookieStore;
import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.cookie.BasicClientCookie;
import java.util.Date;
public class CookieUtil {
/**
* 将{@link java.net.CookieStore}转换到{@link CookieStore}
* @param netCookieStore java.net.CookieStore
* @return org.apache.http.client.CookieStore
*/
public static CookieStore NAParse(java.net.CookieStore netCookieStore){
CookieStore apacheCookieStore = new BasicCookieStore();
netCookieStore.getCookies().forEach(netCookie -> {
BasicClientCookie aCookie = new BasicClientCookie(netCookie.getName(), netCookie.getValue());
aCookie.setComment(netCookie.getComment());
aCookie.setDomain(netCookie.getDomain());
aCookie.setExpiryDate(new Date(netCookie.getMaxAge()));
aCookie.setPath(netCookie.getPath());
aCookie.setSecure(netCookie.getSecure());
aCookie.setVersion(netCookie.getVersion());
apacheCookieStore.addCookie(aCookie);
});
return apacheCookieStore;
}
}

View File

@ -0,0 +1,65 @@
package net.lamgc.cgj.util;
import java.util.concurrent.atomic.AtomicInteger;
public final class Locker<K> {
private final LockerMap<K> fromMap;
private final K key;
private final boolean autoDestroy;
private final AtomicInteger lockCount = new AtomicInteger(0);
/**
* 构造一个锁对象
* @param map 所属LockerMap
* @param key 所属Key
*/
Locker(LockerMap<K> map, K key, boolean autoDestroy) {
this.fromMap = map;
this.key = key;
this.autoDestroy = autoDestroy;
}
/**
* 上锁
*/
public void lock() {
lockCount.incrementAndGet();
}
/**
* 解锁
*/
public void unlock() {
int newValue = lockCount.decrementAndGet();
if(newValue <= 0 && autoDestroy) {
destroy();
}
}
/**
* 获取锁对象所属Key
*/
public K getKey() {
return key;
}
/**
* 销毁锁对象
*/
public void destroy() {
fromMap.destroyLocker(this);
}
@Override
public String toString() {
return "Locker@" + this.hashCode() + "{" +
"fromMap=" + fromMap +
", key=" + key +
'}';
}
}

View File

@ -0,0 +1,31 @@
package net.lamgc.cgj.util;
import java.util.HashMap;
public class LockerMap<K> {
private final HashMap<K, Locker<K>> lockerHashMap = new HashMap<>();
/**
* 创建锁
* @param key Key
* @return 如果Key所属锁存在, 则返回对应锁, 否则返回新锁
*/
public Locker<K> createLocker(K key, boolean autoDestroy) {
if(lockerHashMap.containsKey(key)) {
return lockerHashMap.get(key);
}
Locker<K> newLocker = new Locker<>(this, key, autoDestroy);
lockerHashMap.put(key, newLocker);
return newLocker;
}
/**
* 销毁锁
* @param locker 锁对象
*/
public void destroyLocker(Locker<K> locker) {
lockerHashMap.remove(locker.getKey());
}
}

View File

@ -1,13 +1,12 @@
package net.lamgc.cgj.util;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.utils.base.runner.StringParameterParser;
public class PagesQualityParser implements StringParameterParser<PixivDownload.PageQuality> {
@Override
public PixivDownload.PageQuality parse(String strValue) throws Exception {
public PixivDownload.PageQuality parse(String strValue) {
return PixivDownload.PageQuality.valueOf(strValue.toUpperCase());
}

View File

@ -0,0 +1,51 @@
package net.lamgc.cgj.util;
import net.lamgc.utils.base.ArgumentsProperties;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public final class PropertiesUtils {
private final static Logger log = LoggerFactory.getLogger(PropertiesUtils.class);
private PropertiesUtils() {}
/**
* 从ArgumentsProperties获取设置项到System Properties
* @param prop ArgumentsProperties对象
* @param key 设置项key
* @param defaultValue 默认值
* @return 如果成功从ArgumentsProperties获得设置项, 返回true, 如未找到(使用了defaultValue或null), 返回false;
*/
public static boolean getSettingToSysProp(ArgumentsProperties prop, String key, String defaultValue) {
if(prop.containsKey(key)) {
log.info("{}: {}", key, prop.getValue(key));
System.setProperty("cgj." + key, prop.getValue(key));
return true;
} else {
if(defaultValue != null) {
System.setProperty("cgj." + key, defaultValue);
}
return false;
}
}
/**
* 将环境变量的值读取并存入System Properties.
* @param envKey 待获取的环境变量Key
* @param sysPropKey 要设置的System Properties Key值
* @param defaultValue 默认值, 可选
* @return 如果设置成功, 返回true, 否则返回false
*/
public static boolean getEnvSettingToSysProp(String envKey, String sysPropKey, String defaultValue) {
String env = System.getenv(envKey);
if(env != null) {
System.setProperty("cgj." + sysPropKey, env);
return true;
} else if(defaultValue != null) {
System.setProperty("cgj." + sysPropKey, defaultValue);
}
return false;
}
}

View File

@ -0,0 +1,192 @@
package net.lamgc.cgj.util;
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicLong;
/**
* 带有时间限制的线程池.
* 当线程超出了限制时间时, 将会对该线程发出中断.
*/
public class TimeLimitThreadPoolExecutor extends ThreadPoolExecutor {
/**
* 执行时间限制, 单位毫秒.
* 默认30s.
*/
private final AtomicLong executeTimeLimit = new AtomicLong();
/**
* 检查间隔时间.
* 默认100ms.
*/
private final AtomicLong timeoutCheckInterval = new AtomicLong(100);
private final Map<Thread, MonitorInfo> workerThreadMap = new Hashtable<>();
private final Thread timeoutCheckThread = createTimeoutCheckThread();
public TimeLimitThreadPoolExecutor(long executeLimitTime,
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
setInitialTime(executeLimitTime);
timeoutCheckThread.start();
}
public TimeLimitThreadPoolExecutor(long executeLimitTime,
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue,
ThreadFactory threadFactory) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory);
setInitialTime(executeLimitTime);
timeoutCheckThread.start();
}
public TimeLimitThreadPoolExecutor(long executeLimitTime,
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue,
RejectedExecutionHandler handler) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, handler);
setInitialTime(executeLimitTime);
timeoutCheckThread.start();
}
public TimeLimitThreadPoolExecutor(long executeLimitTime,
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue,
ThreadFactory threadFactory,
RejectedExecutionHandler handler) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler);
setInitialTime(executeLimitTime);
timeoutCheckThread.start();
}
private void setInitialTime(long executeLimitTime) {
if(executeLimitTime > 0) {
executeTimeLimit.set(executeLimitTime);
}
}
/**
* 设置执行时间.
* <p>注意: 该修改仅在线程池完全停止后才有效</p>
* @see #isTerminated()
* @param time 新的限制时间(ms)
*/
public void setExecuteTimeLimit(long time) {
if(time <= 0) {
throw new IllegalArgumentException("Time is not allowed to be set to 0 or less");
}
if(this.isTerminated()) {
executeTimeLimit.set(time);
}
}
/**
* 设置超时检查间隔.
* <p>该方法仅会在当前检查后生效.</p>
* @param time 新的检查间隔(ms)
*/
public void setTimeoutCheckInterval(long time) {
if(time <= 0) {
throw new IllegalArgumentException("Time is not allowed to be set to 0 or less");
}
timeoutCheckInterval.set(time);
}
/**
* 获取当前设置的执行时间限制.
* @return 执行时间限制(ms).
*/
public long getExecuteTimeLimit() {
return executeTimeLimit.get();
}
/**
* 获取当前设定的超时检查间隔
* @return 间隔时间(ms).
*/
public long getTimeoutCheckInterval() {
return timeoutCheckInterval.get();
}
private Thread createTimeoutCheckThread() {
Thread checkThread = new Thread(() -> {
if(executeTimeLimit.get() <= 0) {
return;
}
while (true) {
try {
long interval = this.timeoutCheckInterval.get();
//noinspection BusyWait 用于等待超时
Thread.sleep(interval);
// 检查是否存在超时的任务
final long executeTimeLimit = this.executeTimeLimit.get();
workerThreadMap.forEach((thread, info) -> {
long currentTime = info.getTimeRemaining().getAndAdd(interval);
if(currentTime > executeTimeLimit) {
if(!info.getNotifyInterrupted().get() && !thread.isInterrupted()) {
thread.interrupt();
info.getNotifyInterrupted().set(true);
}
}
});
} catch(InterruptedException ignored) {
break;
}
}
});
checkThread.setName("ThreadPool-" + Integer.toHexString(this.hashCode()) +"-TimeoutCheck");
return checkThread;
}
@Override
protected void beforeExecute(Thread t, Runnable r) {
workerThreadMap.put(t, new MonitorInfo());
super.beforeExecute(t, r);
}
@Override
protected void afterExecute(Runnable r, Throwable t) {
workerThreadMap.remove(Thread.currentThread());
super.afterExecute(r, t);
}
@Override
protected void terminated() {
this.timeoutCheckThread.interrupt();
super.terminated();
}
private static class MonitorInfo {
private final AtomicLong timeRemaining = new AtomicLong();
private final AtomicBoolean notifyInterrupted = new AtomicBoolean(false);
public AtomicBoolean getNotifyInterrupted() {
return notifyInterrupted;
}
public AtomicLong getTimeRemaining() {
return timeRemaining;
}
}
}

View File

@ -1,2 +0,0 @@
server.port=8081
server.tomcat.max-threads=1

View File

@ -0,0 +1,12 @@
server:
port: 8081
spring:
cq:
plugin-list:
- net.lamgc.cgj.bot.framework.coolq.CQPluginMain
event:
corePoolSize: 8
maxPoolSize: 16
keepAliveTime: 25000
workQueueSize: 1024

View File

@ -0,0 +1,53 @@
<?xml version="1.0" encoding="UTF-8"?>
<configuration status="WARN">
<properties>
<property name="logStorePath">./logs</property>
<property name="charset">UTF-8</property>
<property name="standard_pattern">[%-d{HH:mm:ss.SSS} %5level][%logger.%method():%-3L][%thread]: %msg%n</property>
<property name="mirai_pattern">[%-d{HH:mm:ss.SSS} %5level][%logger]: %msg%n</property>
<property name="logsDir">${sys:cgj.logsPath:-logs}</property>
</properties>
<Appenders>
<Console name="STANDARD_STDOUT" target="SYSTEM_OUT">
<PatternLayout charset="${charset}">
<MarkerPatternSelector defaultPattern="${standard_pattern}">
<PatternMatch key="mirai" pattern="${mirai_pattern}" />
</MarkerPatternSelector>
</PatternLayout>
<Filters>
<ThresholdFilter level="WARN" onMatch="DENY" onMismatch="ACCEPT"/>
</Filters>
</Console>
<Console name="STANDARD_STDERR" target="SYSTEM_ERR">
<PatternLayout charset="${charset}">
<MarkerPatternSelector defaultPattern="${standard_pattern}">
<PatternMatch key="mirai" pattern="${mirai_pattern}" />
</MarkerPatternSelector>
</PatternLayout>
<Filters>
<ThresholdFilter level="WARN" onMatch="ACCEPT" onMismatch="DENY"/>
</Filters>
</Console>
<RollingFile name="rollingFile" fileName="${logsDir}/latest.log" filePattern="${logsDir}/running.%-d{yyyy-MM-dd_HH-mm-ss}.log.gz">
<PatternLayout charset="${charset}">
<MarkerPatternSelector defaultPattern="${standard_pattern}">
<PatternMatch key="mirai" pattern="${mirai_pattern}" />
</MarkerPatternSelector>
</PatternLayout>
<Policies>
<OnStartupTriggeringPolicy />
</Policies>
</RollingFile>
</Appenders>
<Loggers>
<Logger level="INFO" name="org.apache.http"/>
<Root level="TRACE">
<AppenderRef ref="STANDARD_STDOUT"/>
<AppenderRef ref="STANDARD_STDERR"/>
<AppenderRef ref="rollingFile"/>
</Root>
</Loggers>
</configuration>

View File

@ -1,27 +1,47 @@
<?xml version="1.0" encoding="UTF-8"?>
<configuration status="WARN">
<!--
测试版跟发布版在日志配置文件上的区别仅仅只有'Loggers'的不同, 'properties'和'Appenders'是一致的.
-->
<properties>
<property name="logStorePath">./logs</property>
<property name="charset">UTF-8</property>
<property name="pattern">[%-d{HH:mm:ss.SSS} %5level][%logger.%method():%-3L][%thread]: %msg%n</property>
<property name="standard_pattern">[%-d{HH:mm:ss.SSS} %5level][%logger.%method():%-3L][%thread]: %msg%n</property>
<property name="mirai_pattern">[%-d{HH:mm:ss.SSS} %5level][%logger]: %msg%n</property>
<property name="logsDir">${sys:cgj.logsPath:-logs}</property>
</properties>
<Appenders>
<Console name="CONSOLE_STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="${pattern}" charset="${charset}"/>
<Console name="STANDARD_STDOUT" target="SYSTEM_OUT">
<PatternLayout charset="${charset}">
<MarkerPatternSelector defaultPattern="${standard_pattern}">
<PatternMatch key="mirai" pattern="${mirai_pattern}" />
</MarkerPatternSelector>
</PatternLayout>
<Filters>
<ThresholdFilter level="WARN" onMatch="DENY" onMismatch="ACCEPT"/>
<LevelRangeFilter minLevel="INFO" maxLevel="INFO" onMatch="ACCEPT" onMismatch="DENY"/>
</Filters>
</Console>
<Console name="CONSOLE_STDERR" target="SYSTEM_ERR">
<PatternLayout pattern="${pattern}" charset="${charset}"/>
<Console name="STANDARD_STDERR" target="SYSTEM_ERR">
<PatternLayout charset="${charset}">
<MarkerPatternSelector defaultPattern="${standard_pattern}">
<PatternMatch key="mirai" pattern="${mirai_pattern}" />
</MarkerPatternSelector>
</PatternLayout>
<Filters>
<ThresholdFilter level="WARN" onMatch="ACCEPT" onMismatch="DENY"/>
</Filters>
</Console>
<RollingFile name="rollingFile" fileName="logs/latest.log" filePattern="logs/running.%-d{yyyy-MM-dd_HH-mm-ss}.log.gz">
<PatternLayout pattern="${pattern}" charset="${charset}"/>
<RollingFile name="rollingFile" fileName="${logsDir}/latest.log" filePattern="${logsDir}/running.%-d{yyyy-MM-dd_HH-mm-ss}.log.gz">
<Filters>
<ThresholdFilter level="DEBUG" onMatch="ACCEPT" onMismatch="DENY"/>
</Filters>
<PatternLayout charset="${charset}">
<MarkerPatternSelector defaultPattern="${standard_pattern}">
<PatternMatch key="mirai" pattern="${mirai_pattern}" />
</MarkerPatternSelector>
</PatternLayout>
<Policies>
<OnStartupTriggeringPolicy />
</Policies>
@ -31,8 +51,8 @@
<Loggers>
<Logger level="INFO" name="org.apache.http"/>
<Root level="DEBUG">
<AppenderRef ref="CONSOLE_STDOUT"/>
<AppenderRef ref="CONSOLE_STDERR"/>
<AppenderRef ref="STANDARD_STDOUT"/>
<AppenderRef ref="STANDARD_STDERR"/>
<AppenderRef ref="rollingFile"/>
</Root>
</Loggers>

View File

@ -1,9 +0,0 @@
#用于访问Pixiv的代理服务器
reptile.proxy.type=socks5/socks4/http
reptile.proxy.host=127.0.0.1
reptile.proxy.port=1080
reptile.proxy.username=
reptile.proxy.password=
#登录用代理, 需要让浏览器使用该代理, 访问Pixiv并登录
login.proxy.host=127.0.0.1
login.proxy.port=1080

View File

@ -0,0 +1,30 @@
package net.lamgc.cgj.bot.util;
import org.junit.Assert;
import org.junit.Test;
public class GroupMuteManagerTest {
@Test
public void muteStateTest() {
GroupMuteManager manager = new GroupMuteManager();
Assert.assertNull(manager.isMute(1, true)); // 未设置的群组返回null
Assert.assertFalse(manager.isMute(1, false)); // 未设置就返回false
manager.setMuteState(1, true); // mute == true
Assert.assertNotNull(manager.isMute(1, true)); // 第一次设置后不为null
Assert.assertTrue(manager.isMute(1, false)); // 确保条件正常
manager.setMuteState(2, true); // 不能出现不同群号的冲突
manager.setMuteState(1, false);
Assert.assertTrue(manager.isMute(2, false));
Assert.assertNotNull(manager.isMute(1, true)); // 变更为false后依然不能返回null
Assert.assertFalse(manager.isMute(1, false));
}
@Test
public void invalidGroupIdTest() {
GroupMuteManager manager = new GroupMuteManager();
manager.setMuteState(-1, true); // 设置应该是无效的
Assert.assertFalse(manager.isMute(-1, false)); // 由于设置无效, 返回false即可
}
}

View File

@ -0,0 +1,22 @@
package net.lamgc.cgj.pixiv;
import org.junit.Ignore;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@Ignore
public class ModerateContentDetectorTest {
private final static AdultContentDetector acd = new ModerateContentDetector("d91b6c3fa2bba9ee8f9e68827ba0d937");
private final static Logger log = LoggerFactory.getLogger(ModerateContentDetector.class);
@Test
public void checkTest() throws Exception {
log.info("Detect: {}, isAdult: {}",
acd.detect(80840411, false, 0),
acd.isAdultContent(80840411, false, 0));
}
}

View File

@ -1,8 +1,15 @@
package net.lamgc.cgj.pixiv;
import org.apache.commons.io.IOUtils;
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import org.apache.http.HttpHost;
import org.apache.http.client.CookieStore;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
import org.apache.tomcat.util.http.fileupload.util.Streams;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Ignore;
@ -24,9 +31,9 @@ public class PixivDownloadTest {
private static CookieStore cookieStore;
private final static Logger log = LoggerFactory.getLogger("PixivDownloadTest");
private final static Logger log = LoggerFactory.getLogger(PixivDownloadTest.class);
private static HttpHost proxy = new HttpHost("127.0.0.1", 1001);
private static final HttpHost proxy = new HttpHost("127.0.0.1", 1001);
@BeforeClass
public static void before() throws IOException, ClassNotFoundException {
@ -57,7 +64,7 @@ public class PixivDownloadTest {
ZipEntry entry = new ZipEntry(link.substring(link.lastIndexOf("/") + 1));
log.info("正在写入: " + entry.getName());
zos.putNextEntry(entry);
IOUtils.copy(inputStream, zos);
Streams.copy(inputStream, zos, false);
zos.flush();
} catch (IOException e) {
log.error("写入文件项时发生异常", e);
@ -89,7 +96,7 @@ public class PixivDownloadTest {
ZipEntry entry = new ZipEntry(link.substring(link.lastIndexOf("/") + 1));
log.info("正在写入: " + entry.getName());
zos.putNextEntry(entry);
IOUtils.copy(inputStream, zos);
Streams.copy(inputStream, zos, false);
zos.flush();
log.info("已成功写入 {}", entry.getName());
} catch (IOException e) {
@ -132,7 +139,7 @@ public class PixivDownloadTest {
entry.setComment(rankInfo.toString());
log.info("正在写入: " + entry.getName());
zos.putNextEntry(entry);
IOUtils.copy(inputStream, zos);
Streams.copy(inputStream, zos, false);
zos.flush();
log.info("已成功写入 {}", entry.getName());
} catch (IOException e) {
@ -173,22 +180,8 @@ public class PixivDownloadTest {
log.info("正在调用方法...");
try {
pixivDownload.getRankingAsInputStream(null, null, queryDate, 5, 50, PixivDownload.PageQuality.ORIGINAL, (rank, link, rankInfo, inputStream) -> {
/*try {
ZipEntry entry = new ZipEntry("Rank" + rank + "-" + link.substring(link.lastIndexOf("/") + 1));
entry.setComment(rankInfo.toString());
log.info("正在写入: " + entry.getName());
zos.putNextEntry(entry);
IOUtils.copy(inputStream, zos);
zos.flush();
log.info("已成功写入 {}", entry.getName());
inputStream.close();
} catch (IOException e) {
log.error("写入文件项时发生异常", e);
}*/
log.info("空操作");
});
pixivDownload.getRankingAsInputStream(null, null, queryDate, 5, 50,
PixivDownload.PageQuality.ORIGINAL, (rank, link, rankInfo, inputStream) -> log.info("空操作"));
} finally {
zos.finish();
zos.flush();
@ -202,4 +195,17 @@ public class PixivDownloadTest {
log.info(new PixivDownload(cookieStore, proxy).getIllustPreLoadDataById(64076261).toString());
}
@Test
public void illustInfoTest() throws IOException {
CloseableHttpClient httpClient = HttpClientBuilder.create().setProxy(new HttpHost("127.0.0.1", 1001)).build();
HttpGet request = new HttpGet(PixivURL.getPixivIllustInfoAPI(80880547));
CloseableHttpResponse response = httpClient.execute(request);
String body = EntityUtils.toString(response.getEntity());
JsonObject illustInfoResult = new Gson().fromJson(body, JsonObject.class);
log.info("IllustInfoJsonResult: {}", illustInfoResult);
PixivDownload.PixivIllustType illustType = PixivDownload.PixivIllustType.getIllustTypeByPreLoadData(80880547, illustInfoResult);
log.info("IllustType: {}", illustType);
Assert.assertEquals(PixivDownload.PixivIllustType.UGOIRA, illustType);
}
}

View File

@ -1,26 +0,0 @@
package net.lamgc.cgj.pixiv;
import org.junit.Assert;
import org.junit.Test;
public class PixivSearchBuilderTest {
@Test
public void buildTest() {
PixivSearchBuilder builder = new PixivSearchBuilder("hololive");
//builder.addExcludeKeyword("fubuki").addExcludeKeyword("minato");
builder.addIncludeKeyword("35").addIncludeKeyword("okayu").addIncludeKeyword("百鬼あやめ");
System.out.println(builder.buildURL());
}
@Test
public void equalsTest() {
Assert.assertEquals(new PixivSearchBuilder("风景"), new PixivSearchBuilder("风景"));
}
@Test
public void hashCodeTest() {
Assert.assertEquals(new PixivSearchBuilder("风景").hashCode(), new PixivSearchBuilder("风景").hashCode());
}
}

View File

@ -0,0 +1,25 @@
package net.lamgc.cgj.pixiv;
import org.junit.Assert;
import org.junit.Test;
public class PixivSearchLinkBuilderTest {
@Test
public void buildTest() {
PixivSearchLinkBuilder builder = new PixivSearchLinkBuilder("hololive");
builder.addIncludeKeyword("35").addIncludeKeyword("okayu").addIncludeKeyword("百鬼あやめ");
System.out.println(builder.buildURL());
}
@Test
public void equalsTest() {
Assert.assertEquals(new PixivSearchLinkBuilder("风景"), new PixivSearchLinkBuilder("风景"));
}
@Test
public void hashCodeTest() {
Assert.assertEquals(new PixivSearchLinkBuilder("风景").hashCode(), new PixivSearchLinkBuilder("风景").hashCode());
}
}

Some files were not shown because too many files have changed in this diff Show More