Compare commits

...

94 Commits

Author SHA1 Message Date
5637ef30d4 Merge branch 'master' of github.com:LamGC/ContentGrabbingJi into add-framework-interface 2020-07-15 14:40:38 +08:00
bf8de1ac1e [Fix] 修复潜在的时区错误问题;
[Fix] Dockerfile.sample 将时区配置从`Asia/Shanghai`改成`GMT+8`;
2020-07-15 14:09:47 +08:00
6fc7d8ad78 [Change] 调整Framework接口, 为框架提供特定数据存储目录;
[Change] Framework 适配FrameworkResources的更改, 将`getName`调整为`getIdentify`, 增加`getFrameworkName`方法用于获取框架名(可能会改为注解方式以防止更改);
[Change] FrameworkManager 适配Framework的更改, 增加`checkFramework`方法以对FrameworkName进行检查;
[Change] FrameworkManager, FrameworkResources 将FrameworkResources从FrameworkManager分离成单独的类;
[Change] ConsoleMain, MiraiMain, SpringCQApplication 适配相关更改;
2020-07-15 10:57:30 +08:00
a7c434da61 [Change] 调整Readme内容;
[Fix] README.md 补充管理员命令内容;
2020-07-15 10:35:31 +08:00
575dc0c7fb [Fix] Dockerfile.sample 修复了容器无法正常启动的问题; 2020-07-14 11:01:21 +08:00
56ef463c63 [Change] MessageEvent 调整toString输出中hashCode的格式; 2020-07-13 20:35:39 +08:00
4387da37f5 [Change] 调整群禁言管理职责;
[Delete] BotEventHandler 移除禁言状态管理相关功能;
[Add] GroupMuteManager, GroupMuteManagerTest 增加群禁言状态管理类;
[Change] MiraiMain 将禁言状态对接由使用BotEventHandler改为独立管理;
2020-07-13 09:47:36 +08:00
0fc3e3ab48 [Change] 调整应用内事件线程池参数;
[Change] BotEventHandler 调整线程池参数;
2020-07-13 09:47:31 +08:00
a606ec0423 [Fix] IssueTemplates 修复模板错误的问题; 2020-07-12 11:18:29 +08:00
210aa84ed5 [Update] IssueTemplates 增加功能模板并更新Bug模板; 2020-07-12 11:12:21 +08:00
6fbbe522db [Fix #22] PixivDownload 修复因InputStream意外关闭导致的异常误判; 2020-07-12 01:12:58 +08:00
6d55325fc7 [Fix] PixivUgoiraBuilder 修复ZipInputStream在读取完第一帧图像后被意外关闭的问题; 2020-07-10 09:53:41 +08:00
d1c7f6f973 [Version] 更新版本(2.5.2-20200630.2-SNAPSHOT -> 2.5.2-20200709.1-SNAPSHOT); 2020-07-09 22:45:36 +08:00
0727ef4f93 [Clear] CQPluginMain 整理代码; 2020-07-09 22:19:35 +08:00
73a1caaf46 [Fix #21] 修复了RandomIntervalSendTimer在高版本Java中因非法反射导致应用异常终止的问题; 2020-07-09 22:17:09 +08:00
4784f8773b [Change] Pixiv搜索推荐候选接口.md 更新文档内容; 2020-07-03 10:09:21 +08:00
9a7d16124a [Add] FrameworkManager 添加"frameworkSet"方法; 2020-07-03 10:05:20 +08:00
b754559187 Merge branch 'master' into add-framework-interface 2020-07-03 09:15:27 +08:00
f80b6e72e0 [Fix] CQPluginMain 通过延迟加载来修复ApplicationBoot初始化失败的问题; 2020-07-03 09:11:27 +08:00
05e933838e Merge branch 'update-SpringCQ' 2020-07-03 08:20:18 +08:00
a87735d9e0 [Change] Main, SpringCQApplication 移除仅与SpringCQ相关的Spring相关代码, 转移到SpringCQApplication;
[Change] FrameworkManager 调整"registerFramework"返回值, 调整"shutdownAllFramework"的过程;
2020-07-03 08:19:21 +08:00
6ec99dbf17 Merge branch 'update-SpringCQ' into add-framework-interface
# Conflicts:
#	src/main/java/net/lamgc/cgj/Main.java
#	src/main/java/net/lamgc/cgj/bot/framework/coolq/SpringCQApplication.java
2020-07-03 08:02:50 +08:00
1599a5325a [Change] Framework 调整"getName"方法的默认实现;
[Change] Main 调整pluginMode启动方式;
[Change] SpringCQApplication 实现Framework接口;
2020-07-03 08:01:31 +08:00
3045b571a8 [Change] Main, SpringCQApplication 移除仅与SpringCQ相关的Spring相关代码, 转移到SpringCQApplication;
[Delete] CQPluginMain 移除注释信息;
2020-07-03 07:52:48 +08:00
5c2b6b4ee5 Merge branch 'update-SpringCQ' into add-framework-interface 2020-07-03 07:48:16 +08:00
c2e8a07500 [Change] ConsoleMain, MiraiMain 尝试为平台框架实现Framework接口;
[Change] Framework 补充Javadoc, 增加"getName"方法的默认方法体;
[Change] FrameworkManager 调整向frameworkThreadGroup发起中断的时机;
2020-07-03 00:46:23 +08:00
1b937953c3 [Add] Framework, FrameworkManager 增加框架统一管理接口; 2020-07-03 00:27:54 +08:00
553212e556 [Change] SpringCQApplication, Main 将SpringCQ启动部分迁移到SpringCQApplication内; 2020-07-03 00:23:56 +08:00
5383de7450 Merge remote-tracking branch 'origin/master' 2020-07-02 23:33:36 +08:00
9964205dc2 [Add] Pixiv搜索推荐候选接口.md 添加可用于优化搜索的接口文档; 2020-07-02 23:21:28 +08:00
394c3940e4 Merge remote-tracking branch 'origin/master' 2020-07-02 14:03:59 +08:00
d3d6f151d4 [Delete] application.properties 移除Properties版Application配置文件;
[Delete] CQConfig 移除不兼容的类;
[Add] application.yml 添加yml格式的Application配置文件;
[Change] Main 移除CQConfig类引用;
[Update] net.lz1998:spring-cq 升级依赖项版本(4.14.0.6 -> 4.15.0.1);
[Change] SpringCQMessageSenderFactory, CQPluginMain 使用较为妥协的方法实现SenderFactory;
2020-07-02 14:02:01 +08:00
62e3affef1 [Change] LocalHashCacheStore 调整内部类CacheObject的访问权; 2020-07-02 13:46:46 +08:00
51efc95c1c [Update] net.mamoe:mirai-core, net.mamoe:mirai-core-qqandroid 更新依赖库版本(1.0.2 -> 1.0.4); 2020-07-02 13:41:34 +08:00
d5c3a438b0 [Change] Pixiv预加载数据.md 更改文档标题; 2020-07-02 09:25:01 +08:00
792bfcb1bd [Add] Pixiv预加载数据.md 增加新的Pixiv接口文档; 2020-07-02 09:10:01 +08:00
cbd7db3570 Merge branch 'master' of github.com:LamGC/ContentGrabbingJi 2020-06-30 22:43:22 +08:00
467e7065fa [Update] net.lamgc:java-utils 更新依赖项版本(1.2.0_20200517.1-SNAPSHOT -> 1.3.1); 2020-06-30 22:40:39 +08:00
d2240e56fd [Change] README.md 调整格式; 2020-06-30 10:40:34 +08:00
eef9f285ca [Update] README.md 更新一份更规范的Readme; 2020-06-30 08:58:08 +08:00
f63d52df6d [Version] 更新版本(2.5.2-20200630.1-SNAPSHOT -> 2.5.2-20200630.2-SNAPSHOT); 2020-06-30 08:05:18 +08:00
50a638e97d [Clear] CacheStoreCentral 清理测试代码; 2020-06-30 08:01:08 +08:00
968a616595 [Version] 更新版本(2.5.2-20200618.1-SNAPSHOT -> 2.5.2-20200630.1-SNAPSHOT); 2020-06-30 01:23:04 +08:00
6443ba68ab [Fix] AutoCleanTimer 修复自动清除定时器没有重复工作;
[Fix] LocalHashCacheStore 修复clean方法抛出'ConcurrentModificationException'异常的问题;
[Change] CacheStoreCentral 调整PreLoadData的热点缓存时间;
[Change] HotDataCacheStore 将最近获取的Key重置其过期时间, 调整clean过程;
2020-06-30 01:21:17 +08:00
cbe0a38f59 Merge branch 'master' of github.com:LamGC/ContentGrabbingJi 2020-06-29 23:33:41 +08:00
eadfacb7d0 [Clear] PixivSearchLinkBuilder 清理代码; 2020-06-23 13:53:54 +08:00
572189d906 [Change] Main 增加'buildPassword'命令用于构造加密后的密码, 移除对PixivLoginProxyServer的嵌入支持;
[Delete] pom.xml 移除依赖项'net.lamgc:PixivLoginProxyServer';
2020-06-23 10:02:47 +08:00
811694587d [Version] 更新版本(2.5.2-20200617.1-SNAPSHOT -> 2.5.2-20200618.1-SNAPSHOT); 2020-06-18 19:27:11 +08:00
419f2de055 [Add] BotEventHandler 设计在消息正在发送,被时限线程池中断时, 重新投递事件以重新发送消息; 2020-06-18 10:11:08 +08:00
368c78e171 [Add] CacheStoreCentral, BotCommandProcess Search命令增加对pageIndex参数使用(之前竟然是个摆设!?); 2020-06-18 10:05:34 +08:00
bccf47db6e [Change] MessageEvent toString方法增加hashCode输出; 2020-06-18 09:35:37 +08:00
cf351074cc [Delete] BufferMessageEvent 移除缓冲消息事件;
[Add] BufferedMessageSender 增加缓冲消息发送器, 用于替代BufferMessageEvent;
[Change] BotCommandProcess 将Random命令中所使用的BufferMessageEvent替换成BufferedMessageSender;
2020-06-18 09:35:25 +08:00
c4ce18d37a [Fix] log4j2.xml 修复HttpClient日志输出过多的问题;
[Change] PixivURL 将PixivURL设计为不可继承且不可实例化;
2020-06-18 09:01:16 +08:00
26fd18917d [Delete] PixivAccessProxyServer 移除弃用的类;
[Change] Locker 删除不必要的代码;
[Change] PixivDownload 删除多余的Javadoc内容;
[Change] TimeLimitThreadPoolExecutorTest 调整日志输出方向, 补充测试细节;
2020-06-17 19:49:36 +08:00
acbd990181 [Version] 更新版本(2.5.2-20200611.1-SNAPSHOT -> 2.5.2-20200617.1-SNAPSHOT); 2020-06-17 16:48:47 +08:00
6db9cda08a Merge branch 'optimize-memory-cache' 2020-06-17 16:47:08 +08:00
1c742bfb6f [Fix] TimeLimitThreadPoolExecutor 修复了超时提醒遭到时限线程池中断的问题, 调整线程池仅发起一次中断; 2020-06-17 16:14:32 +08:00
2f30fe1696 [Change] CacheStoreCentral 调整PreLoadData的本地缓存时间, 添加注意事项; 2020-06-17 10:51:06 +08:00
32db952e63 Merge branch 'master' into optimize-memory-cache 2020-06-15 17:42:51 +08:00
bd6b825704 [Fix] pom.xml 排除slf4j-log4j12依赖项以修复package后应用启动报Slf4j实现冲突的问题; 2020-06-15 17:42:31 +08:00
3943963505 [Fix] BotEventHandler 修复线程池最大线程数与最低线程数错误, 导致无法启动应用的问题; 2020-06-15 17:40:23 +08:00
08822f68eb [Change] Locker 增加清理日志;
[Add] LockerMapTest 增加单元测试;
2020-06-15 17:18:28 +08:00
2f647ee9fa [Add] LockerMap, Locker 增加Locker锁对象和LockerMap锁对象存储;
[Change] CacheStoreCentral 将synchronized所使用的的锁对象由String(常量池)转换成Locker<K>以尝试减少内存占用;
2020-06-15 16:22:37 +08:00
44a7f49510 [Change] BotGlobal 补充IllegalStateException错误信息; 2020-06-15 15:48:07 +08:00
85088e1b2c [Change] RandomIntervalSendTimer 调整日志输出内容; 2020-06-13 23:10:01 +08:00
8dfb858b9f [Change] RedisPoolCacheStore 优化内部方法实现; 2020-06-13 17:24:06 +08:00
0e76cebc31 [Fix] PreLoadDataAttributeComparator 修复获取PreLoadData抛出的异常信息未记录到日志的问题; 2020-06-13 16:51:15 +08:00
c64320ad78 [Change] HttpRequestException 统一接口异常, 调整类所在包路径; 2020-06-13 16:44:52 +08:00
91e065f657 [Change] ImageCacheStore 优化中断处理, 自动取消任务, 补充TaskState设置; 2020-06-13 12:54:17 +08:00
91f8b0070f [Change] PreLoadDataAttributeComparator 更改类名; 2020-06-13 00:28:52 +08:00
ad54dbfbf3 [Change] PreLoadDataComparator, PreLoadDataAttribute 将PreLoadData的属性Enum迁移到单独的类;
[Change] BotCommandProcess 适配更改;
2020-06-12 22:28:14 +08:00
ebb3dea99e [Change] ConsoleMessageEvent, ConsoleMessageSender 将消息传递给Sender发送, 设置消息前缀格式;
[Change] ConsoleMain 调整私聊模式下的前缀;
[Change] MessageSource 规范化命名;
2020-06-12 20:13:36 +08:00
abcd26f21b [Fix] BotEventHandler 修复事件处理线程非预期设置的问题;
[Change] BotEventHandler 设置线程超时时间;
[Fix] CacheStoreCentral 整理'InterruptedException'在'getImageById'的传递路径;
2020-06-12 20:00:37 +08:00
d9b08f8ad9 [Change] BotCommandProcess, CacheStoreCentral 调整个别日志的输出级别; 2020-06-12 19:28:59 +08:00
5e030c12b2 [Change] BotCommandProcess 将各命令中'page'参数更名为'p'以简化命令;
[Change] PixivSearchLinkBuilder 更改类名(PixivSearchBuilder -> PixivSearchLinkBuilder);
[Add] PixivURL 增加接口常量;
[Change] Main, CacheStoreCentral, PixivSearchLinkBuilderTest 适配PixivSearchLinkBuilder的更改;
2020-06-12 17:06:04 +08:00
62eabce8f6 [Clear] PixivDownload, RandomIntervalSendTimer 整理代码; 2020-06-12 10:31:16 +08:00
64bca3c8f7 [Change] RandomIntervalSendTimer 更改Timer线程名; 2020-06-12 10:20:56 +08:00
3b3f97e638 [Fix] CacheStoreCentral 修复Search命令中'option'参数区分大小写的问题;
[Fix] BotCommandProcess 修复Search命令中遇到不存在作品时会中断处理的问题;
[Change] BotCommandProcess 调整Ranking命令的'type'参数默认值(ILLUST -> ALL);
2020-06-12 10:12:29 +08:00
951824cbe2 [Version] 更新版本(2.5.2-20200610.4-SNAPSHOT -> 2.5.2-20200611.1-SNAPSHOT); 2020-06-11 16:47:35 +08:00
e104abedeb [Add] MiraiMain 增加原始消息事件日志信息; 2020-06-11 16:46:32 +08:00
c3967d214d [Change] Dockerfile.sample 提升镜像使用的Jdk版本(8-jre -> 14 jdk), 增加Arthas诊断工具, 优化构建指令顺序; 2020-06-11 10:57:40 +08:00
26e377a2c7 [Change] log4j2.xml 调整日志输出到文件的限制;
[Change] 将内容较长的日志设为TRACE级别, 以减少日志占用;
[CLear] 整理代码;
2020-06-11 09:49:19 +08:00
87f2535b48 [Change] BotCommandProcess 优化Tag过滤表达式;
[Change] ImageCacheHandler 调整日志输出级别;
2020-06-11 09:08:58 +08:00
97d06c4fc3 [Change] CacheStoreCentral 调整代码以为后续更改做准备;
[Change] BotCommandProcess, ImageCacheHandler, PreLoadDataComparator, RandomRankingArtworksSender 适配CacheStoreCentral的更改;
2020-06-10 20:21:23 +08:00
bfe25c2012 [Version] 更新版本(2.5.2-20200610.3-SNAPSHOT -> 2.5.2-20200610.4-SNAPSHOT); 2020-06-10 16:39:01 +08:00
036f3eaf4a [Delete] simple.properties 删除弃用的Properties模板文件; 2020-06-10 16:27:13 +08:00
f07c8d0b76 [Add] MiraiMain 增加Bot网络重连相关设置; 2020-06-10 16:26:20 +08:00
d0cb4417b1 [Version] 更新版本(2.5.2-20200610.2-SNAPSHOT -> 2.5.2-20200610.3-SNAPSHOT); 2020-06-10 15:47:30 +08:00
3f256c5a0a [Fix] log4j2.xml 修复日志配置错误的问题;
[Change] BotGlobal, Main 调整日志输出内容和级别;
2020-06-10 15:46:33 +08:00
b9180b9651 [Change] MiraiMain 显性设置DeviceInfo为'randomDeviceInfo'; 2020-06-10 15:26:26 +08:00
7ccb306ca1 Merge pull request #15 from LamGC/change-license-AGPLv3
更改开源许可证(LGPLv3 -> AGPLv3)
2020-06-10 14:57:05 +08:00
5dd13ce088 [Change] pom.xml 增加'licenses'; 2020-06-10 14:49:40 +08:00
e9b33938fa [Add] README.md 增加'LICENSE'部分; 2020-06-10 14:41:12 +08:00
cc040e6ec9 [Change] 更改开源许可证; 2020-06-10 10:44:53 +08:00
64 changed files with 2415 additions and 1086 deletions

View File

@ -1,26 +1,35 @@
--- ---
name: Bug Report name: Bug 反馈
about: Use this template to feedback bugs. about: 使用这个模板反馈应用问题。
title: '' title: ''
labels: bug labels: bug
assignees: '' assignees: ''
--- ---
## Environmental information ## ## 环境信息 ##
OS(e.g: Windows 10 1909): 系统(例如: Windows 10 1909): ``
Java(e.g: Oracle Jdk 8.242): Java版本(例如: Oracle Jdk 8.242): ``
Issue version(versionTag or commitId): <!-- 如果直接使用发布版那么就填写发布版版本号v开头 -->
<!-- 如果你通过编译运行的方式运行开发版请你填写运行所使用的Commit Id -->
发生问题所在的版本: ``
## Problem description ## ## 问题描述 ##
// Describe the problem in as much detail as possible here <!-- 尽可能的清晰描述问题的信息 -->
## Expected behavior ## ## 预期行为 ##
// What will this function do under normal circumstances? <!-- 你觉得正常情况下应该会发生什么? -->
## Actual behavior ## ## 实际行为 ##
// But what does this feature actually look like? <!-- 实际上这个功能做了什么? -->
## Recurrence steps ## ## 复现步骤 ##
// What can we do to recreate this situation? <!-- 将复现步骤详细的写出,如果是偶发问题,可以写已知的,触发几率高的方法 -->
1. 1.
## 相关信息 ##
### 日志 ###
<!-- 如果日志涉及了问题,请务必将日志一同提交,这对排查问题非常有用 -->
```
```

View File

@ -0,0 +1,11 @@
---
name: 功能/想法 提议
about: 使用这个模板将你对应用的想法提出来,或许我们会采纳!
title: ''
labels: function, question
assignees: ''
---
<!-- 如果可以,尽可能的清晰、详细的表达你的想法 -->
<!-- 没关系的,我们会进一步向你交流以尝试了解你的想法! -->

View File

@ -1,10 +1,11 @@
FROM openjdk:8u252-jre FROM openjdk:14
COPY --from=hengyunabc/arthas:latest /opt/arthas /opt/arthas
ENV jarFileName=ContentGrabbingJi-exec.jar ENV jarFileName=ContentGrabbingJi-exec.jar
ENV CGJ_REDIS_URI="127.0.0.1:6379" ENV CGJ_REDIS_URI="127.0.0.1:6379"
ENV CGJ_PROXY="" ENV CGJ_PROXY=""
RUN mkdir /data/ RUN mkdir /data/
ENTRYPOINT ["/usr/java/openjdk-14/bin/java", "-Duser.timezone=GMT+8"]
CMD ["-Dcgj.logsPath=/data/logs", "-jar", "/CGJ.jar", "botMode", "-botDataDir=/data"]
COPY ${jarFileName} /CGJ.jar COPY ${jarFileName} /CGJ.jar
CMD ["java", "-Dcgj.logsPath=/data/logs", "-jar", "/CGJ.jar", "botMode", "-botDataDir=/data"]

746
LICENSE
View File

@ -1,165 +1,661 @@
GNU LESSER GENERAL PUBLIC LICENSE GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 29 June 2007 Version 3, 19 November 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/> Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed. of this license document, but changing it is not allowed.
Preamble
This version of the GNU Lesser General Public License incorporates The GNU Affero General Public License is a free, copyleft license for
the terms and conditions of version 3 of the GNU General Public software and other kinds of works, specifically designed to ensure
License, supplemented by the additional permissions listed below. cooperation with the community in the case of network server software.
0. Additional Definitions. The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
As used herein, "this License" refers to version 3 of the GNU Lesser When we speak of free software, we are referring to freedom, not
General Public License, and the "GNU GPL" refers to version 3 of the GNU price. Our General Public Licenses are designed to make sure that you
General Public License. have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
"The Library" refers to a covered work governed by this License, Developers that use our General Public Licenses protect your rights
other than an Application or a Combined Work as defined below. with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
An "Application" is any work that makes use of an interface provided A secondary benefit of defending all users' freedom is that
by the Library, but which is not otherwise based on the Library. improvements made in alternate versions of the program, if they
Defining a subclass of a class defined by the Library is deemed a mode receive widespread use, become available for other developers to
of using an interface provided by the Library. incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
A "Combined Work" is a work produced by combining or linking an The GNU Affero General Public License is designed specifically to
Application with the Library. The particular version of the Library ensure that, in such cases, the modified source code becomes available
with which the Combined Work was made is also called the "Linked to the community. It requires the operator of a network server to
Version". provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
The "Minimal Corresponding Source" for a Combined Work means the An older license, called the Affero General Public License and
Corresponding Source for the Combined Work, excluding any source code published by Affero, was designed to accomplish similar goals. This is
for portions of the Combined Work that, considered in isolation, are a different license, not a version of the Affero GPL, but Affero has
based on the Application, and not on the Linked Version. released a new version of the Affero GPL which permits relicensing under
this license.
The "Corresponding Application Code" for a Combined Work means the The precise terms and conditions for copying, distribution and
object code and/or source code for the Application, including any data modification follow.
and utility programs needed for reproducing the Combined Work from the
Application, but excluding the System Libraries of the Combined Work.
1. Exception to Section 3 of the GNU GPL. TERMS AND CONDITIONS
You may convey a covered work under sections 3 and 4 of this License 0. Definitions.
without being bound by section 3 of the GNU GPL.
2. Conveying Modified Versions. "This License" refers to version 3 of the GNU Affero General Public License.
If you modify a copy of the Library, and, in your modifications, a "Copyright" also means copyright-like laws that apply to other kinds of
facility refers to a function or data to be supplied by an Application works, such as semiconductor masks.
that uses the facility (other than as an argument passed when the
facility is invoked), then you may convey a copy of the modified
version:
a) under this License, provided that you make a good faith effort to "The Program" refers to any copyrightable work licensed under this
ensure that, in the event an Application does not supply the License. Each licensee is addressed as "you". "Licensees" and
function or data, the facility still operates, and performs "recipients" may be individuals or organizations.
whatever part of its purpose remains meaningful, or
b) under the GNU GPL, with none of the additional permissions of To "modify" a work means to copy from or adapt all or part of the work
this License applicable to that copy. in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
3. Object Code Incorporating Material from Library Header Files. A "covered work" means either the unmodified Program or a work based
on the Program.
The object code form of an Application may incorporate material from To "propagate" a work means to do anything with it that, without
a header file that is part of the Library. You may convey such object permission, would make you directly or secondarily liable for
code under terms of your choice, provided that, if the incorporated infringement under applicable copyright law, except executing it on a
material is not limited to numerical parameters, data structure computer or modifying a private copy. Propagation includes copying,
layouts and accessors, or small macros, inline functions and templates distribution (with or without modification), making available to the
(ten or fewer lines in length), you do both of the following: public, and in some countries other activities as well.
a) Give prominent notice with each copy of the object code that the To "convey" a work means any kind of propagation that enables other
Library is used in it and that the Library and its use are parties to make or receive copies. Mere interaction with a user through
covered by this License. a computer network, with no transfer of a copy, is not conveying.
b) Accompany the object code with a copy of the GNU GPL and this license An interactive user interface displays "Appropriate Legal Notices"
document. to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
4. Combined Works. 1. Source Code.
You may convey a Combined Work under terms of your choice that, The "source code" for a work means the preferred form of the work
taken together, effectively do not restrict modification of the for making modifications to it. "Object code" means any non-source
portions of the Library contained in the Combined Work and reverse form of a work.
engineering for debugging such modifications, if you also do each of
the following:
a) Give prominent notice with each copy of the Combined Work that A "Standard Interface" means an interface that either is an official
the Library is used in it and that the Library and its use are standard defined by a recognized standards body, or, in the case of
covered by this License. interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
b) Accompany the Combined Work with a copy of the GNU GPL and this license The "System Libraries" of an executable work include anything, other
document. than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
c) For a Combined Work that displays copyright notices during The "Corresponding Source" for a work in object code form means all
execution, include the copyright notice for the Library among the source code needed to generate, install, and (for an executable
these notices, as well as a reference directing the user to the work) run the object code and to modify the work, including scripts to
copies of the GNU GPL and this license document. control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
d) Do one of the following: The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
0) Convey the Minimal Corresponding Source under the terms of this The Corresponding Source for a work in source code form is that
License, and the Corresponding Application Code in a form same work.
suitable for, and under terms that permit, the user to
recombine or relink the Application with a modified version of
the Linked Version to produce a modified Combined Work, in the
manner specified by section 6 of the GNU GPL for conveying
Corresponding Source.
1) Use a suitable shared library mechanism for linking with the 2. Basic Permissions.
Library. A suitable mechanism is one that (a) uses at run time
a copy of the Library already present on the user's computer
system, and (b) will operate properly with a modified version
of the Library that is interface-compatible with the Linked
Version.
e) Provide Installation Information, but only if you would otherwise All rights granted under this License are granted for the term of
be required to provide such information under section 6 of the copyright on the Program, and are irrevocable provided the stated
GNU GPL, and only to the extent that such information is conditions are met. This License explicitly affirms your unlimited
necessary to install and execute a modified version of the permission to run the unmodified Program. The output from running a
Combined Work produced by recombining or relinking the covered work is covered by this License only if the output, given its
Application with a modified version of the Linked Version. (If content, constitutes a covered work. This License acknowledges your
you use option 4d0, the Installation Information must accompany rights of fair use or other equivalent, as provided by copyright law.
the Minimal Corresponding Source and Corresponding Application
Code. If you use option 4d1, you must provide the Installation
Information in the manner specified by section 6 of the GNU GPL
for conveying Corresponding Source.)
5. Combined Libraries. You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
You may place library facilities that are a work based on the Conveying under any other circumstances is permitted solely under
Library side by side in a single library together with other library the conditions stated below. Sublicensing is not allowed; section 10
facilities that are not Applications and are not covered by this makes it unnecessary.
License, and convey such a combined library under terms of your
choice, if you do both of the following:
a) Accompany the combined library with a copy of the same work based 3. Protecting Users' Legal Rights From Anti-Circumvention Law.
on the Library, uncombined with any other library facilities,
conveyed under the terms of this License.
b) Give prominent notice with the combined library that part of it No covered work shall be deemed part of an effective technological
is a work based on the Library, and explaining where to find the measure under any applicable law fulfilling obligations under article
accompanying uncombined form of the same work. 11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
6. Revised Versions of the GNU Lesser General Public License. When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
The Free Software Foundation may publish revised and/or new versions 4. Conveying Verbatim Copies.
of the GNU Lesser General Public License from time to time. Such new
versions will be similar in spirit to the present version, but may
differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the You may convey verbatim copies of the Program's source code as you
Library as you received it specifies that a certain numbered version receive it, in any medium, provided that you conspicuously and
of the GNU Lesser General Public License "or any later version" appropriately publish on each copy an appropriate copyright notice;
applies to it, you have the option of following the terms and keep intact all notices stating that this License and any
conditions either of that published version or of any later version non-permissive terms added in accord with section 7 apply to the code;
published by the Free Software Foundation. If the Library as you keep intact all notices of the absence of any warranty; and give all
received it does not specify a version number of the GNU Lesser recipients a copy of this License along with the Program.
General Public License, you may choose any version of the GNU Lesser
General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide You may charge any price or no price for each copy that you convey,
whether future versions of the GNU Lesser General Public License shall and you may offer support or warranty protection for a fee.
apply, that proxy's public statement of acceptance of any version is
permanent authorization for you to choose that version for the 5. Conveying Modified Source Versions.
Library.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published
by the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<https://www.gnu.org/licenses/>.

161
README.md
View File

@ -1,39 +1,138 @@
# ContentGrabbingJi # ContentGrabbingJi - 色图姬 #
Pixiv爬虫一只同时也是一个机器人/插件! 一个以高性能、高效率为目标,多平台/框架支持、持续维护的Pixiv聊天机器人
## 支持的机器人平台 ## 色图姬存在的目的最初是作为**爬虫 + 机器人**的形式开发,在开发过程中逐渐以多聊天平台,高效率为目标进行。
- [Mirai](https://github.com/mamoe/mirai)
- [CoolQ](https://cqp.cc)(基于[`SpringCQ`](https://github.com/lz1998/spring-cq), 不支持多账号使用, 需要使用[`CQHttp`](https://cqhttp.cc/)插件)
## Usage ## ## 安装 ##
> 注意: 运行色图姬前, 你需要准备一个Pixiv账号的会话Cookie存储文件, 否则色图姬将无法运行. ### 通过Jar文件部署 ###
> 详见[PixivLoginProxyServer](https://github.com/LamGC/PixivLoginProxyServer)项目的[Readme](https://github.com/LamGC/PixivLoginProxyServer/blob/master/README.md). 1. 从项目的[版本页面](https://github.com/LamGC/ContentGrabbingJi/releases)下载最新版色图姬主程序jar文件
2. 准备一个目录, 用于存放运行数据(这里以`./runData`为例子).
3. 将通过PixivLoginProxyServer获得的Pixiv登录会话文件放置在目录中(`./runData/cookies.store`).
4. 使用命令`java -jar <CGJ.jar> buildPassword -password "QQ机器人账号的密码"`构造一个登录用的密码.
5. 在数据目录创建一个配置文件`bot.properties`, 并添加如下内容:
```properties
bot.qq=<机器人QQ账号>
bot.password=<通过buildPassword获得的密码>
```
6. 在数据目录创建新文件夹`setting`, 并创建一个全局配置文件`global.properties`, 然后设置如下内容:
```properties
# 该配置为全局配置文件, 当群组没有特定配置时, 将使用全局配置.
# 管理员QQ (必填)
admin.adminId=<管理员QQ号>
# 是否允许r18作品
image.allowR18=false
# 查询排行榜默认的长度(比如15就是发送1~15名的作品), 该选项请适当调整, 设置过长可能导致超出聊天平台的最长消息长度, 导致发送失败!
ranking.itemCountLimit=15
# 排行榜图片数量(比如排行榜长度为15, 可以设置前10名有图片, 后5名没有图片), 调整该配置可有效控制消息发送所需时间.
ranking.imageCountLimit=15
# 搜索结果缓存时间, 默认2小时, 单位毫秒
cache.searchBody.expire=7200000
# 搜索结果长度. 该选项请适当调整, 设置过长可能导致超出聊天平台的最长消息长度, 导致发送失败!
search.itemCountLimit=8
```
7. 配置完成后, 准备一台Redis服务端, 用于缓存数据.
8. Redis服务器准备好后, 使用命令启动色图姬:`java -jar <CGJ.jar> botMode -botDataDir <数据目录> -redisAddress <Redis服务器地址> [-proxy 代理服务器地址]`
9. 完成!好好享受!
### Arguments ### ### 通过Docker部署 ###
> ENV参数名为环境变量名, 用于给Docker容器提供设置方式. 使用Docker将可以更好的管理色图姬所使用的资源和管理色图姬的运行。
(正在完善中...)
- 通用参数 ## 使用 ##
- `-proxy` / `ENV: CGJ_PROXY`: (**可选**) 设置代理 ### 普通用户 ###
- 格式: `协议://地址:端口` 将色图姬部署完成,并且让色图姬在某个平台登录完成后,你就可以通过聊天平台向色图姬发起会话了!
- 示例: `socks5://127.0.0.1:1080` 使用 `.cgj` 向色图姬询问帮助信息!
- 机器人参数
- `-botDataDir` / `ENV: CGJ_BOT_DATA_DIR`: (**可选**) 设置`botMode`运行模式下机器人数据存储目录
- 格式: `路径`
- 示例: `./data`
- 默认: `./`
- `-redisAddress` / `ENV: CGJ_REDIS_URI`: (**必填, 计划支持可选**) Redis服务器地址
- 格式: `地址:端口`
- 示例: `127.0.0.1:6379`
> 例如以BotMode启动应用: `java -jar CGJ.jar botMode -proxy "socks5://127.0.0.1:1080 -redisAddress 127.0.0.1:6379` 另外,由于色图姬在开发过程中直接使用了原本应用在命令行中的参数解析工具,所以你需要了解一下色图姬命令的格式,
色图姬的命令格式为:
```bash
.cgj <命令> -<参数名> <参数值> ...
```
如果色图姬无法识别你的命令,那么它会发送一次帮助信息给你。
### Commands ### ### 管理员用户 ###
- `pluginMode`: CoolQ插件模式(依赖[酷Q机器人](https://cqp.cc/), 支持与CoolQ其他插件共存, 性能耗损大) 你应该注意到了在部署过程中你需要设置一个管理员QQ的配置色图姬支持通过命令来管理色图姬的运行。
- `botMode`: Mirai独立模式(机器人独立运行, 不支持与其他插件共存, 性能耗损小) 目前支持的管理员命令:
- `collectionDownload`: 收藏下载, 以原图画质下载Cookie所属账号的所有收藏作品 ```bash
- `getRecommends`: 将访问主页获得的推荐作品全部以原图画质下载 # 清除缓存(慎用)
- `getRankingIllust`: 以原图画质下载指定排行榜类型的全部作品 # 该操作将会清除Redis服务器内的所有数据, 以及色图姬下载到本地的所有图片缓存.
- `search`: 搜索指定内容并获取相关作品信息(不下载) .cgjadmin cleanCache
> 注意: 除去机器人模式外, 其他功能后续可能会出现改动. # 设置配置项
# 如果不使用group参数, 则设置全局配置
# 注意: 配置项设置后需要使用`.cgjadmin saveProperties`才会保存到文件中,
# 如不保存, 则仅能在本次运行中生效(或使用`.cgjadmin loadProperties`重新加载后失效).
.cgjadmin setProperty <-key 配置项名> <-value 配置项新值> [-group 指定群组]
# 查询配置项
# 如果不使用group参数, 则查询全局配置
.cgjadmin getProperty <-key 配置项名> [-group 指定群组]
# 保存所有配置
.cgjadmin saveProperties
# 读取所有配置
# 使用 reload 参数将会重载所有配置, 而不是覆盖读取
.cgjadmin loadProperties [-reload]
# 运行定时更新任务
# 可指定要更新数据的日期
.cgjadmin runUpdateTask [-date yyyy-MM-dd]
# 增加群组作品推送
# 如果增加了original参数, 则图片为原图发送
# 如果不指定group参数, 则群组为命令发送所在群组
# 最长发送时间 = 最短发送时间 + 随机时间范围
.cgjadmin addPushGroup [-group 指定群组号] [-minTime 最短发送时间] [-floatTime 随机时间范围] [-rankingStart 排行榜起始排名]
[-rankingStop 排行榜结束排名] [-mode 排行榜模式] [-type 排行榜类型] [-original]
# 删除群组推送功能
# 如果不指定group参数, 则群组为命令发送所在群组
.cgjadmin removePushGroup [-group 指定群组号]
# 加载作品推送配置
.cgjadmin loadPushList
# 保存作品推送配置
.cgjadmin savePushList
# 获取被报告的作品列表
# 该命令会返回被其他用户报告其存在问题的作品列表
.cgjadmin getReportList
# 解封被报告的作品
.cgjadmin unBanArtwork <-id 被ban作品的Id>
```
## 贡献 ##
向色图姬贡献不一定需要编程知识,向色图姬项目提出意见,反馈问题同样会为色图姬项目提供很大的帮助!
如果你在使用色图姬的过程中遇到了Bug可以通过色图姬项目的**Issues**使用[Bug反馈模板](https://github.com/LamGC/ContentGrabbingJi/issues/new?assignees=&labels=bug&template=Bug_Report.md&title=)向色图姬提供Bug信息。
如果是为色图姬提供一些新功能想法或者对色图姬有什么意见则可以直接通过Issues发起讨论。
如果你会Java开发又想为色图姬提供一些新功能可以通过Fork仓库的方法实现后发起PR合并到色图姬项目中
向色图姬贡献代码需要遵循一些贡献事项如果你的代码不符合这些事项PR有可能会被关闭
> 注意色图姬的初衷并没有任何恶意的意图如果尝试向色图姬提供恶意功能或代码PR将会被拒绝、关闭。
## LICENSE ##
本项目基于 `GNU Affero General Public License 3.0` 开源许可协议开源,
你可以在本项目目录下的 `LICENSE` 文件查阅协议副本,
或浏览 [https://www.gnu.org/licenses/agpl-3.0.html](https://www.gnu.org/licenses/agpl-3.0.html) 查阅协议副本。
```
ContentGrabbingJi - A pixiv robot used in chat
Copyright (C) 2020 LamGC (lam827@lamgc.net)
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as
published by the Free Software Foundation, either version 3 of the
License, or any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/agpl-3.0.html>.
```
尤其注意:根据协议,如果你基于本项目开发本项目的衍生版本并通过网络为他人提供服务,则必须遵守该协议,向 *服务的使用者* 提供 **为其服务的衍生版本****完整源代码**

View File

@ -0,0 +1,47 @@
## 搜索推荐接口 ##
### 说明 ###
可用于优化搜索内容。
### 接口地址 ###
```
GET https://www.pixiv.net/rpc/cps.php?
```
- 是否需要登录: `否`
- 是否为Pixiv标准接口返回格式: `否`
- 是否需要Referer请求头: `是`
> 补充: Referer请求头只要是Pixiv的就可以了.
### 参数 ###
Url参数:
- `keyword`: 搜索内容
> 注意: 搜索内容需要进行Url编码空格要转换成`%20`而不是`+`
### 请求示例 ###
```
GET https://www.pixiv.net/rpc/cps.php?keyword=幸运星
```
### 返回数据 ###
#### 数据示例 ####
```json
{
"candidates":[
{
"tag_name":"\u3089\u304d\u2606\u3059\u305f",
"access_count":"68286498",
"tag_translation":"\u5e78\u8fd0\u661f",
"type":"tag_translation"
}, // ...
]
}
```
#### 字段说明 ####
- `candidates`: (`Object[]`) 搜索推荐候选列表
- `tag_name`: (`string`) 推荐词原名
- `access_count`: (`string` -> `number`) 推荐词访问量
- `tag_translation`: (`string`) 推荐词对应翻译名
- `type`: (`string`) 推荐词类型
- `tag_translation`: 标签翻译信息
- `prefix`: 前缀

View File

@ -0,0 +1,319 @@
## Pixiv预加载数据 ##
### 说明 ###
作品预加载数据仅在加载作品页面时存在,处理后删除。
### 接口地址 ###
```
GET https://www.pixiv.net/artworks/{IllustId}
```
- 是否需要登录: `是`
- 是否为Pixiv标准接口返回格式: `否`
- 是否需要Referer请求头: `否`
### 参数 ###
Url参数:
- `IllustId`: 作品Id
### 请求示例 ###
```
GET https://www.pixiv.net/artworks/82647306
```
### 返回数据 ###
> 注意: 该接口返回HTML格式数据并不是JSON格式数据。
预加载数据需要对返回的Html数据进行解析路径如下
- CSS Select: meta#meta-preload-data
- html>head>meta#meta-preload-data
获得标签后,获取`content`属性即可获得预加载数据内容
#### 数据示例 ####
```json
{
"timestamp":"2020-07-01T11:32:30+09:00",
"illust":{
"82647306":{
"illustId":"82647306",
"illustTitle":"水着キャルちゃん!",
"illustComment":"水着のキャルちゃんはかわいいぞ!!",
"id":"82647306",
"title":"水着キャルちゃん!",
"description":"水着のキャルちゃんはかわいいぞ!!",
"illustType":0,
"createDate":"2020-06-29T12:28:06+00:00",
"uploadDate":"2020-06-29T12:28:06+00:00",
"restrict":0,
"xRestrict":0,
"sl":2,
"urls":{
"mini":"https://i.pximg.net/c/48x48/img-master/img/2020/06/29/21/28/06/82647306_p0_square1200.jpg",
"thumb":"https://i.pximg.net/c/250x250_80_a2/img-master/img/2020/06/29/21/28/06/82647306_p0_square1200.jpg",
"small":"https://i.pximg.net/c/540x540_70/img-master/img/2020/06/29/21/28/06/82647306_p0_master1200.jpg",
"regular":"https://i.pximg.net/img-master/img/2020/06/29/21/28/06/82647306_p0_master1200.jpg",
"original":"https://i.pximg.net/img-original/img/2020/06/29/21/28/06/82647306_p0.jpg"
},
"tags":{
"authorId":"55859246",
"isLocked":false,
"tags":[
{
"tag":"プリンセスコネクト!Re:Dive",
"locked":true,
"deletable":false,
"userId":"55859246",
"translation":{
"en":"公主连结Re:Dive"
},
"userName":"秋鳩むぎ"
}, // ...
],
"writable":true
},
"alt":"#プリンセスコネクト!Re:Dive 水着キャルちゃん! - 秋鳩むぎ的插画",
"storableTags":[
"_bee-JX46i",
"nAtxkwJ5Sy",
"q303ip6Ui5"
],
"userId":"55859246",
"userName":"秋鳩むぎ",
"userAccount":"pigeonwheat",
"userIllusts":{
"82647306":{
"illustId":"82647306",
"illustTitle":"水着キャルちゃん!",
"id":"82647306",
"title":"水着キャルちゃん!",
"illustType":0,
"xRestrict":0,
"restrict":0,
"sl":2,
"url":"https://i.pximg.net/c/250x250_80_a2/img-master/img/2020/06/29/21/28/06/82647306_p0_square1200.jpg",
"description":"水着のキャルちゃんはかわいいぞ!!",
"tags":[
"プリンセスコネクト!Re:Dive",
"キャル(プリコネ)",
"おへそ"
],
"userId":"55859246",
"userName":"秋鳩むぎ",
"width":2000,
"height":3000,
"pageCount":1,
"isBookmarkable":true,
"bookmarkData":null,
"alt":"#プリンセスコネクト!Re:Dive 水着キャルちゃん! - 秋鳩むぎ的插画",
"isAdContainer":false,
"titleCaptionTranslation":{
"workTitle":null,
"workCaption":null
},
"createDate":"2020-06-29T21:28:06+09:00",
"updateDate":"2020-06-29T21:28:06+09:00",
"seriesId":null,
"seriesTitle":null
}
},
"likeData":false,
"width":2000,
"height":3000,
"pageCount":1,
"bookmarkCount":39,
"likeCount":31,
"commentCount":2,
"responseCount":0,
"viewCount":239,
"isHowto":false,
"isOriginal":false,
"imageResponseOutData":[
],
"imageResponseData":[
],
"imageResponseCount":0,
"pollData":null,
"seriesNavData":null,
"descriptionBoothId":null,
"descriptionYoutubeId":null,
"comicPromotion":null,
"fanboxPromotion":null,
"contestBanners":[
],
"isBookmarkable":true,
"bookmarkData":null,
"contestData":null,
"zoneConfig":{
"responsive":{
"url":"https://pixon.ads-pixiv.net/show?zone_id=illust_responsive&amp;format=js&amp;s=1&amp;up=0&amp;a=22&amp;ng=w&amp;l=zh&amp;uri=%2Fartworks%2F_PARAM_&amp;is_spa=1&amp;K=59bba275c645c&amp;ab_test_digits_first=32&amp;yuid=NDJ3gQk&amp;suid=Pggb9mua3yznnr7lz&amp;num=5efbf5be273&amp;t=_bee-JX46i&amp;t=b8b4-hqot7&amp;t=kY01H5r3Pd"
},
"rectangle":{
"url":"https://pixon.ads-pixiv.net/show?zone_id=illust_rectangle&amp;format=js&amp;s=1&amp;up=0&amp;a=22&amp;ng=w&amp;l=zh&amp;uri=%2Fartworks%2F_PARAM_&amp;is_spa=1&amp;K=59bba275c645c&amp;ab_test_digits_first=32&amp;yuid=NDJ3gQk&amp;suid=Pggb9mua42776dfuu&amp;num=5efbf5be810&amp;t=_bee-JX46i&amp;t=b8b4-hqot7&amp;t=kY01H5r3Pd"
},
"500x500":{
"url":"https://pixon.ads-pixiv.net/show?zone_id=bigbanner&amp;format=js&amp;s=1&amp;up=0&amp;a=22&amp;ng=w&amp;l=zh&amp;uri=%2Fartworks%2F_PARAM_&amp;is_spa=1&amp;K=59bba275c645c&amp;ab_test_digits_first=32&amp;yuid=NDJ3gQk&amp;suid=Pggb9mua442sjsueo&amp;num=5efbf5be568&amp;t=_bee-JX46i&amp;t=b8b4-hqot7&amp;t=kY01H5r3Pd"
},
"header":{
"url":"https://pixon.ads-pixiv.net/show?zone_id=header&amp;format=js&amp;s=1&amp;up=0&amp;a=22&amp;ng=w&amp;l=zh&amp;uri=%2Fartworks%2F_PARAM_&amp;is_spa=1&amp;K=59bba275c645c&amp;ab_test_digits_first=32&amp;yuid=NDJ3gQk&amp;suid=Pggb9mua45spzoimt&amp;num=5efbf5be155&amp;t=_bee-JX46i&amp;t=b8b4-hqot7&amp;t=kY01H5r3Pd"
},
"footer":{
"url":"https://pixon.ads-pixiv.net/show?zone_id=footer&amp;format=js&amp;s=1&amp;up=0&amp;a=22&amp;ng=w&amp;l=zh&amp;uri=%2Fartworks%2F_PARAM_&amp;is_spa=1&amp;K=59bba275c645c&amp;ab_test_digits_first=32&amp;yuid=NDJ3gQk&amp;suid=Pggb9mua47f9zcoim&amp;num=5efbf5be400&amp;t=_bee-JX46i&amp;t=b8b4-hqot7&amp;t=kY01H5r3Pd"
},
"expandedFooter":{
"url":"https://pixon.ads-pixiv.net/show?zone_id=multiple_illust_viewer&amp;format=js&amp;s=1&amp;up=0&amp;a=22&amp;ng=w&amp;l=zh&amp;uri=%2Fartworks%2F_PARAM_&amp;is_spa=1&amp;K=59bba275c645c&amp;ab_test_digits_first=32&amp;yuid=NDJ3gQk&amp;suid=Pggb9mua4928ct0yw&amp;num=5efbf5be471&amp;t=_bee-JX46i&amp;t=b8b4-hqot7&amp;t=kY01H5r3Pd"
},
"logo":{
"url":"https://pixon.ads-pixiv.net/show?zone_id=logo_side&amp;format=js&amp;s=1&amp;up=0&amp;a=22&amp;ng=w&amp;l=zh&amp;uri=%2Fartworks%2F_PARAM_&amp;is_spa=1&amp;K=59bba275c645c&amp;ab_test_digits_first=32&amp;yuid=NDJ3gQk&amp;suid=Pggb9mua4aqu6i4sr&amp;num=5efbf5be844&amp;t=_bee-JX46i&amp;t=b8b4-hqot7&amp;t=kY01H5r3Pd"
}
},
"extraData":{
"meta":{
"title":"#プリンセスコネクト!Re:Dive 水着キャルちゃん! - 秋鳩むぎ的插画 - pixiv",
"description":"この作品 「水着キャルちゃん!」 は 「プリンセスコネクト!Re:Dive」「キャル(プリコネ)」 等のタグがつけられた「秋鳩むぎ」さんのイラストです。 「水着のキャルちゃんはかわいいぞ!!」",
"canonical":"https://www.pixiv.net/artworks/82647306",
"alternateLanguages":{
"ja":"https://www.pixiv.net/artworks/82647306",
"en":"https://www.pixiv.net/en/artworks/82647306"
},
"descriptionHeader":"本作「水着キャルちゃん!」为附有「プリンセスコネクト!Re:Dive」「キャル(プリコネ)」等标签的插画。",
"ogp":{
"description":"水着のキャルちゃんはかわいいぞ!!",
"image":"https://embed.pixiv.net/decorate.php?illust_id=82647306",
"title":"#プリンセスコネクト!Re:Dive 水着キャルちゃん! - 秋鳩むぎ的插画 - pixiv",
"type":"article"
},
"twitter":{
"description":"水着のキャルちゃんはかわいいぞ!!",
"image":"https://embed.pixiv.net/decorate.php?illust_id=82647306",
"title":"水着キャルちゃん!",
"card":"summary_large_image"
}
}
},
"titleCaptionTranslation":{
"workTitle":null,
"workCaption":null
}
}
},
"user":{
"55859246":{
"userId":"55859246",
"name":"秋鳩むぎ",
"image":"https://i.pximg.net/user-profile/img/2020/06/29/21/20/14/18907670_b3f87d819f705ec418f120cd57f9dc41_50.jpg",
"imageBig":"https://i.pximg.net/user-profile/img/2020/06/29/21/20/14/18907670_b3f87d819f705ec418f120cd57f9dc41_170.jpg",
"premium":false,
"isFollowed":false,
"isMypixiv":false,
"isBlocking":false,
"background":null,
"partial":0
}
}
}
```
#### 字段说明 ####
- `timestamp`: (`string`) 请求时间
- `illust`: (`Object`) 作品预加载信息
- `{illustId}`: 作品ID(跟页面请求的IllustId一样)
- `illustId`: (`string` -> `number`) 作品Id
- `illustTitle`: (`string`) 作品标题
- `illustComment`: (`string`) 作品说明
- `id`: (`string` -> `number`) 与`illustId`一致, 猜测是以兼容旧版本为目录而存在
- `title`: (`string`) 与`illustTitle`一致, 猜测是以兼容旧版本为目录而存在
- `description`: (`string`) 作品说明
- `illustType`: (`number`) 作品类型
- `0`: 插画作品
- `1`: 漫画作品
- `2`: 动图作品
- `createDate`: (`string`) 作品创建时间(或者是完成时间?)
- `updateDate`: (`string`) 作品上传时间
- `restrict`: (`number`) 作品限制级(意义不明, 可能是兼容性问题?)?
- `xRestrict`: (`number`) 作品是否为限制级, 基本准确, 少部分不一定(看Pixiv审核怎么理解了)
- `0`: 非限制级内容(即非R18作品)
- `1`: 限制级内容(即R18作品)
- `sl`: (`number`) 不明?
- `urls`: (`string`) 作品图片链接, 需要`Referer`请求头
- `mini`: (`string`) 小尺寸预览图
- `thumb`: (`string`) 小尺寸预览图
- `small`: (`string`) 小尺寸预览图
- `regular`: (`string`) 经压缩,没啥画质损失的原尺寸预览图
- `original`: (`string`) 原图
- `tags`: (`Object`) 作品标签信息
- `authorId`: (`string` -> `number`) 作者用户Id
- `isLocked`: (`boolean`) 标签是否锁定(即不可被访客更改)
- `tags`: (`Object[]`) 标签信息数组
- `tag`: (`string`) 标签原始名
- `locked`: (`boolean`) 标签是否不可更改
- `deletable`: (`boolean`) 标签能否被删除?
- `userId`: (`string` -> `number`) 用户Id
- `translation`: (`Object`) 标签翻译
- `{语种}`: 翻译名
- `userName`: (`string`) 用户名
- `alt`: (`string`) 简略介绍信息(在图片加载失败时可提供给`img`标签使用)
- `storableTags`: (`string[]`) 不明?
- `userId`: (`string` -> `number`) 作者用户Id
- `userName`: (`string`) 作者用户名
- `userAccount`: (`string`) 作者登录名
- `userIllusts`: (`Object`) 作品信息?
- `{IllustId}`: 与请求IllustId一样
- (请转到Pixiv作品信息获取接口.md)
- `likeData`: (`boolean?`) 不明?
- `width`: (`number`) 作品长度
- `height`: (`number`) 作品高度
- `pageCount`: (`number`) 作品页数
- `bookmarkCount`: (`number`) 作品公开的收藏数
- `likeCount`: (`number`) 作品喜欢(点赞)数
- `commentCount`: (`number`) 作品评论数
- `responseCount`: (`number`) 作品响应数?
- `viewCount`: (`number`) 作品阅览数
- `isHowto`: (`boolean`) 不明?
- `isOriginal`: (`boolean`) 不明?
- `imageResponseOutData`: (`Unknown[]`) 不明?
- `imageResponseData`: (`Unknown[]`) 不明?
- `imageResponseCount`: (`number`) 不明?
- `pollData`: (`Unknown`) 不明?
- `seriesNavData`: (`Unknown`) 不明?
- `descriptionBoothId`: (`Unknown`) 不明?
- `descriptionYoutubeId`: (`Unknown`) 不明?
- `comicPromotion`: (`Unknown`) 不明?
- `fanboxPromotion`: (`Unknown`) 不明?
- `contestBanners`: (`Unknown[]`) 不明?
// TODO 待补充
- `isBookmarkable`: (`boolean`) 不明?
- `bookmarkData`: (`Unknown`) 不明?
- `contestData`: (`Unknown`) 不明?
- `zoneConfig`: (`Object`) 猜测是广告信息?
- (基本不用, 忽略...)
- `extraData`: (`Object`) 扩展数据
- `meta`: (`Object`) 元数据
- `title`: (`string`) 网页标题
- `description`: (`string`) Pixiv生成的作品说明
- `canonical`: (`string`) 作品页面链接
- `alternateLanguages`: (`Object`) 不同语言的作品页面链接
- `{语种}`: (`string`) 对应语种的作品链接
- `descriptionHeader`: (`string`) 说明文档(不过似乎是对应了会话所属账号的语种?)
- `ogp`: (`Object`) 猜测是某平台的分享数据?
- `description`: (`string`) 说明内容
- `image`: (`string`) 预览图链接
- `title`: (`string`) 分享标题
- `type`: (`string`) 分享类型?
- `twitter`: (`Object`)
- `description`: (`string`) 说明内容
- `image`: (`string`) 预览图链接
- `title`: (`string`) 分享标题
- `card`: (`string`) 分享类型?
- `titleCaptionTranslation`: (`Object`) 不明?
- `workTitle`: (`Unknown`) 不明?
- `workCaption`: (`Unknown`) 不明?
- `user`: (`Object`) 作者预加载信息
- `{userId}`: 可通过`illust.{illustId}.userId`获得
- `userId`: (`string` -> `number`) 作者用户Id
- `name`: (`string`) 作者用户名
- `image`: (`string`) 作者用户头像(小尺寸)
- `imageBig`: (`string`) 作者用户头像(大尺寸)
- `premium`: (`boolean`) 作者是否为Pixiv会员
- `isFollowed`: (`boolean`) 当前会话用户是否已关注
- `isMypixiv`: (`boolean`) 是否为当前会话本人?
- `isBlocking`: (`boolean`) 是否正在被封禁
- `background`: (`Object`) 背景图片?
- `partial`: (`number`) 不明?

27
pom.xml
View File

@ -6,7 +6,15 @@
<groupId>net.lamgc</groupId> <groupId>net.lamgc</groupId>
<artifactId>ContentGrabbingJi</artifactId> <artifactId>ContentGrabbingJi</artifactId>
<version>2.5.2-20200610.2-SNAPSHOT</version> <version>2.5.2-20200709.1-SNAPSHOT</version>
<licenses>
<license>
<name>GNU Affero General Public License 3.0</name>
<url>https://www.gnu.org/licenses/agpl-3.0.txt</url>
<distribution>repo</distribution>
</license>
</licenses>
<repositories> <repositories>
<repository> <repository>
@ -19,7 +27,7 @@
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<maven.compiler.encoding>UTF-8</maven.compiler.encoding> <maven.compiler.encoding>UTF-8</maven.compiler.encoding>
<mirai.CoreVersion>1.0.2</mirai.CoreVersion> <mirai.CoreVersion>1.0.4</mirai.CoreVersion>
<mirai.JaptVersion>1.1.1</mirai.JaptVersion> <mirai.JaptVersion>1.1.1</mirai.JaptVersion>
<kotlin.version>1.3.71</kotlin.version> <kotlin.version>1.3.71</kotlin.version>
<ktor.version>1.3.2</ktor.version> <ktor.version>1.3.2</ktor.version>
@ -86,7 +94,7 @@
<dependency> <dependency>
<groupId>net.lamgc</groupId> <groupId>net.lamgc</groupId>
<artifactId>java-utils</artifactId> <artifactId>java-utils</artifactId>
<version>1.2.0_20200517.1-SNAPSHOT</version> <version>1.3.1</version>
</dependency> </dependency>
<dependency> <dependency>
@ -113,7 +121,7 @@
<dependency> <dependency>
<groupId>net.lz1998</groupId> <groupId>net.lz1998</groupId>
<artifactId>spring-cq</artifactId> <artifactId>spring-cq</artifactId>
<version>4.14.0.6</version> <version>4.15.0.1</version>
<exclusions> <exclusions>
<exclusion> <exclusion>
<groupId>ch.qos.logback</groupId> <groupId>ch.qos.logback</groupId>
@ -154,11 +162,6 @@
<artifactId>ktor-server-core</artifactId> <artifactId>ktor-server-core</artifactId>
<version>${ktor.version}</version> <version>${ktor.version}</version>
</dependency> </dependency>
<dependency>
<groupId>net.lamgc</groupId>
<artifactId>PixivLoginProxyServer</artifactId>
<version>1.1.1</version>
</dependency>
<dependency> <dependency>
<groupId>com.squareup</groupId> <groupId>com.squareup</groupId>
<artifactId>gifencoder</artifactId> <artifactId>gifencoder</artifactId>
@ -169,6 +172,12 @@
<artifactId>jline</artifactId> <artifactId>jline</artifactId>
<version>3.15.0</version> <version>3.15.0</version>
</dependency> </dependency>
<dependency>
<groupId>com.google.inject</groupId>
<artifactId>guice</artifactId>
<version>4.2.3</version>
</dependency>
</dependencies> </dependencies>
</project> </project>

View File

@ -1,7 +1,5 @@
package net.lamgc.cgj; package net.lamgc.cgj;
import com.github.monkeywie.proxyee.proxy.ProxyConfig;
import com.github.monkeywie.proxyee.proxy.ProxyType;
import com.google.common.base.Strings; import com.google.common.base.Strings;
import com.google.gson.Gson; import com.google.gson.Gson;
import com.google.gson.JsonArray; import com.google.gson.JsonArray;
@ -9,16 +7,17 @@ import com.google.gson.JsonElement;
import com.google.gson.JsonObject; import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.boot.ApplicationBoot; import net.lamgc.cgj.bot.boot.ApplicationBoot;
import net.lamgc.cgj.bot.boot.BotGlobal; import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.framework.FrameworkManager;
import net.lamgc.cgj.bot.framework.cli.ConsoleMain; import net.lamgc.cgj.bot.framework.cli.ConsoleMain;
import net.lamgc.cgj.bot.framework.coolq.CQConfig; import net.lamgc.cgj.bot.framework.coolq.SpringCQApplication;
import net.lamgc.cgj.bot.framework.mirai.MiraiMain; import net.lamgc.cgj.bot.framework.mirai.MiraiMain;
import net.lamgc.cgj.pixiv.PixivDownload; import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivSearchBuilder; import net.lamgc.cgj.pixiv.PixivSearchLinkBuilder;
import net.lamgc.cgj.pixiv.PixivURL; import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.plps.PixivLoginProxyServer;
import net.lamgc.utils.base.runner.Argument; import net.lamgc.utils.base.runner.Argument;
import net.lamgc.utils.base.runner.ArgumentsRunner; import net.lamgc.utils.base.runner.ArgumentsRunner;
import net.lamgc.utils.base.runner.Command; import net.lamgc.utils.base.runner.Command;
import net.lamgc.utils.encrypt.MessageDigestUtils;
import org.apache.http.HttpHost; import org.apache.http.HttpHost;
import org.apache.http.HttpResponse; import org.apache.http.HttpResponse;
import org.apache.http.client.CookieStore; import org.apache.http.client.CookieStore;
@ -27,20 +26,15 @@ import org.apache.http.util.EntityUtils;
import org.apache.tomcat.util.http.fileupload.util.Streams; import org.apache.tomcat.util.http.fileupload.util.Streams;
import org.slf4j.Logger; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import java.io.*; import java.io.*;
import java.nio.charset.StandardCharsets; import java.nio.charset.StandardCharsets;
import java.text.ParseException; import java.text.ParseException;
import java.text.SimpleDateFormat; import java.text.SimpleDateFormat;
import java.util.*; import java.util.*;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.zip.ZipEntry; import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream; import java.util.zip.ZipOutputStream;
@SpringBootApplication
public class Main { public class Main {
private final static Logger log = LoggerFactory.getLogger(Main.class); private final static Logger log = LoggerFactory.getLogger(Main.class);
@ -52,7 +46,15 @@ public class Main {
private static HttpHost proxy; private static HttpHost proxy;
public static void main(String[] args) throws IOException, ClassNotFoundException { public static void main(String[] args) throws IOException, ClassNotFoundException {
log.trace("ContentGrabbingJi 正在启动..."); if(args.length != 0 && args[0].equalsIgnoreCase("buildpassword")) {
ArgumentsRunner.run(Main.class, args);
} else {
standardStart(args);
}
}
private static void standardStart(String[] args) throws IOException, ClassNotFoundException {
log.info("ContentGrabbingJi 正在启动...");
log.debug("Args: {}, LogsPath: {}", Arrays.toString(args), System.getProperty("cgj.logsPath")); log.debug("Args: {}, LogsPath: {}", Arrays.toString(args), System.getProperty("cgj.logsPath"));
log.debug("运行目录: {}", System.getProperty("user.dir")); log.debug("运行目录: {}", System.getProperty("user.dir"));
@ -62,16 +64,9 @@ public class Main {
proxy = BotGlobal.getGlobal().getProxy(); proxy = BotGlobal.getGlobal().getProxy();
File cookieStoreFile = new File(BotGlobal.getGlobal().getDataStoreDir(), "cookies.store"); File cookieStoreFile = new File(BotGlobal.getGlobal().getDataStoreDir(), "cookies.store");
if(!cookieStoreFile.exists()) { if(!cookieStoreFile.exists()) {
log.warn("未找到cookies.store文件, 是否启动PixivLoginProxyServer? (yes/no)"); log.warn("未找到cookies.store文件, 请检查数据存放目录下是否存在'cookies.store'文件!");
try(Scanner scanner = new Scanner(System.in)) { System.exit(1);
if(scanner.nextLine().trim().equalsIgnoreCase("yes")) { return;
startPixivLoginProxyServer();
} else {
System.exit(1);
return;
}
}
} }
ObjectInputStream ois = new ObjectInputStream(new FileInputStream(cookieStoreFile)); ObjectInputStream ois = new ObjectInputStream(new FileInputStream(cookieStoreFile));
cookieStore = (CookieStore) ois.readObject(); cookieStore = (CookieStore) ois.readObject();
@ -86,30 +81,37 @@ public class Main {
} }
@Command @Command
public static void botMode(@Argument(name = "args", force = false) String argsStr) { public static void buildPassword(@Argument(name = "password") String password) {
MiraiMain main = new MiraiMain(); System.out.println("Password: " +
main.init(); Base64.getEncoder().encodeToString(MessageDigestUtils.encrypt(password.getBytes(),
main.close(); MessageDigestUtils.Algorithm.MD5)));
} }
@Command @Command
public static void consoleMode() throws IOException { public static void botMode(@Argument(name = "args", force = false) String argsStr) {
ConsoleMain.start(); try {
FrameworkManager.registerFramework(new MiraiMain()).join();
} catch (InterruptedException ignored) {
}
}
@Command
public static void consoleMode() {
try {
FrameworkManager.registerFramework(new ConsoleMain()).join();
} catch (InterruptedException ignored) {
}
} }
@Command @Command
public static void pluginMode(@Argument(name = "args", force = false) String argsStr) { public static void pluginMode(@Argument(name = "args", force = false) String argsStr) {
log.info("酷Q机器人根目录: {}", BotGlobal.getGlobal().getDataStoreDir().getPath()); try {
CQConfig.init(); FrameworkManager.registerFramework(new SpringCQApplication()).join();
Pattern pattern = Pattern.compile("/\\s*(\".+?\"|[^:\\s])+((\\s*:\\s*(\".+?\"|[^\\s])+)|)|(\".+?\"|[^\"\\s])+"); } catch (InterruptedException ignored) {
Matcher matcher = pattern.matcher(Strings.nullToEmpty(argsStr));
ArrayList<String> argsList = new ArrayList<>();
while (matcher.find()) {
argsList.add(matcher.group());
} }
String[] args = new String[argsList.size()];
argsList.toArray(args);
SpringApplication.run(Main.class, args);
} }
@Command @Command
@ -254,24 +256,24 @@ public class Main {
@Argument(name = "excludeKeywords", force = false) String excludeKeywords, @Argument(name = "excludeKeywords", force = false) String excludeKeywords,
@Argument(name = "contentOption", force = false) String contentOption @Argument(name = "contentOption", force = false) String contentOption
) throws IOException { ) throws IOException {
PixivSearchBuilder searchBuilder = new PixivSearchBuilder(Strings.isNullOrEmpty(content) ? "" : content); PixivSearchLinkBuilder searchBuilder = new PixivSearchLinkBuilder(Strings.isNullOrEmpty(content) ? "" : content);
if (type != null) { if (type != null) {
try { try {
searchBuilder.setSearchType(PixivSearchBuilder.SearchType.valueOf(type.toUpperCase())); searchBuilder.setSearchType(PixivSearchLinkBuilder.SearchType.valueOf(type.toUpperCase()));
} catch (IllegalArgumentException e) { } catch (IllegalArgumentException e) {
log.warn("不支持的SearchType: {}", type); log.warn("不支持的SearchType: {}", type);
} }
} }
if(area != null) { if(area != null) {
try { try {
searchBuilder.setSearchArea(PixivSearchBuilder.SearchArea.valueOf(area)); searchBuilder.setSearchArea(PixivSearchLinkBuilder.SearchArea.valueOf(area));
} catch (IllegalArgumentException e) { } catch (IllegalArgumentException e) {
log.warn("不支持的SearchArea: {}", area); log.warn("不支持的SearchArea: {}", area);
} }
} }
if(contentOption != null) { if(contentOption != null) {
try { try {
searchBuilder.setSearchContentOption(PixivSearchBuilder.SearchContentOption.valueOf(contentOption)); searchBuilder.setSearchContentOption(PixivSearchLinkBuilder.SearchContentOption.valueOf(contentOption));
} catch (IllegalArgumentException e) { } catch (IllegalArgumentException e) {
log.warn("不支持的SearchContentOption: {}", contentOption); log.warn("不支持的SearchContentOption: {}", contentOption);
} }
@ -310,7 +312,7 @@ public class Main {
JsonObject resultBody = jsonObject.getAsJsonObject("body"); JsonObject resultBody = jsonObject.getAsJsonObject("body");
for(PixivSearchBuilder.SearchArea searchArea : PixivSearchBuilder.SearchArea.values()) { for(PixivSearchLinkBuilder.SearchArea searchArea : PixivSearchLinkBuilder.SearchArea.values()) {
if(!resultBody.has(searchArea.jsonKey) || resultBody.getAsJsonObject(searchArea.jsonKey).getAsJsonArray("data").size() == 0) { if(!resultBody.has(searchArea.jsonKey) || resultBody.getAsJsonObject(searchArea.jsonKey).getAsJsonArray("data").size() == 0) {
//log.info("返回数据不包含 {}", searchArea.jsonKey); //log.info("返回数据不包含 {}", searchArea.jsonKey);
continue; continue;
@ -349,57 +351,6 @@ public class Main {
log.info("这里啥都没有哟w"); log.info("这里啥都没有哟w");
} }
private static void saveCookieStoreToFile() throws IOException {
log.info("正在保存CookieStore...");
File outputFile = new File(BotGlobal.getGlobal().getDataStoreDir(), "cookies.store");
if(!outputFile.exists() && !outputFile.createNewFile()){
log.error("保存CookieStore失败.");
return;
}
ObjectOutputStream stream = new ObjectOutputStream(new FileOutputStream(outputFile));
stream.writeObject(cookieStore);
stream.flush();
stream.close();
log.info("CookieStore保存成功.");
}
private static void startPixivLoginProxyServer(){
ProxyConfig proxyConfig = null;
if(proxy != null) {
proxyConfig = new ProxyConfig(ProxyType.HTTP, proxy.getHostName(), proxy.getPort());
}
PixivLoginProxyServer proxyServer = new PixivLoginProxyServer(proxyConfig);
Thread proxyServerStartThread = new Thread(() -> {
log.info("启动代理服务器...");
proxyServer.start(1006);
log.info("代理服务器已关闭.");
});
proxyServerStartThread.setName("LoginProxyServerThread");
proxyServerStartThread.start();
//System.console().readLine();
log.info("登录完成后, 使用\"done\"命令结束登录过程.");
try(Scanner scanner = new Scanner(System.in)) {
while(true) {
if (scanner.nextLine().equalsIgnoreCase("done")) {
log.info("关闭PLPS服务器...");
proxyServer.close();
cookieStore = proxyServer.getCookieStore();
try {
log.info("正在保存CookieStore...");
saveCookieStoreToFile();
log.info("CookieStore保存完成.");
} catch (IOException e) {
log.error("CookieStore保存时发生异常, 本次CookieStore仅可在本次运行使用.", e);
}
break;
} else {
log.warn("要结束登录过程, 请使用\"done\"命令.");
}
}
}
}
private static File getStoreDir() { private static File getStoreDir() {
if(!storeDir.exists() && !storeDir.mkdirs()) { if(!storeDir.exists() && !storeDir.mkdirs()) {
log.error("创建文件夹失败!"); log.error("创建文件夹失败!");

View File

@ -270,7 +270,7 @@ public class BotAdminCommandProcess {
} }
AutoSender sender = new RandomRankingArtworksSender( AutoSender sender = new RandomRankingArtworksSender(
MessageSenderBuilder.getMessageSender(MessageSource.Group, id), MessageSenderBuilder.getMessageSender(MessageSource.GROUP, id),
id, id,
rankingStart, rankingStart,
rankingEnd, rankingEnd,

View File

@ -64,7 +64,7 @@ public class BotCode {
private String platformName; private String platformName;
private String functionName; private String functionName;
private Hashtable<String, String> parameter = new Hashtable<>(); private final Hashtable<String, String> parameter = new Hashtable<>();
/** /**
* 构造一个机器功能码 * 构造一个机器功能码

View File

@ -8,11 +8,12 @@ import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.cache.CacheStore; import net.lamgc.cgj.bot.cache.CacheStore;
import net.lamgc.cgj.bot.cache.CacheStoreCentral; import net.lamgc.cgj.bot.cache.CacheStoreCentral;
import net.lamgc.cgj.bot.cache.JsonRedisCacheStore; import net.lamgc.cgj.bot.cache.JsonRedisCacheStore;
import net.lamgc.cgj.bot.event.BufferMessageEvent; import net.lamgc.cgj.bot.event.BufferedMessageSender;
import net.lamgc.cgj.bot.sort.PreLoadDataComparator; import net.lamgc.cgj.bot.sort.PreLoadDataAttribute;
import net.lamgc.cgj.bot.sort.PreLoadDataAttributeComparator;
import net.lamgc.cgj.pixiv.PixivDownload; import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivDownload.PageQuality; import net.lamgc.cgj.pixiv.PixivDownload.PageQuality;
import net.lamgc.cgj.pixiv.PixivSearchBuilder; import net.lamgc.cgj.pixiv.PixivSearchLinkBuilder;
import net.lamgc.cgj.pixiv.PixivURL; import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.utils.base.runner.Argument; import net.lamgc.utils.base.runner.Argument;
import net.lamgc.utils.base.runner.Command; import net.lamgc.utils.base.runner.Command;
@ -23,6 +24,7 @@ import java.io.File;
import java.io.IOException; import java.io.IOException;
import java.text.SimpleDateFormat; import java.text.SimpleDateFormat;
import java.util.*; import java.util.*;
import java.util.regex.Pattern;
@SuppressWarnings({"SameParameterValue"}) @SuppressWarnings({"SameParameterValue"})
public class BotCommandProcess { public class BotCommandProcess {
@ -90,7 +92,8 @@ public class BotCommandProcess {
*/ */
@Command(commandName = "info") @Command(commandName = "info")
public static String artworkInfo(@Argument(name = "$fromGroup") long fromGroup, public static String artworkInfo(@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "id") int illustId) { @Argument(name = "id") int illustId)
throws InterruptedException {
if(illustId <= 0) { if(illustId <= 0) {
return "这个作品Id是错误的"; return "这个作品Id是错误的";
} }
@ -100,21 +103,21 @@ public class BotCommandProcess {
return "阅览禁止:该作品已被封印!!"; return "阅览禁止:该作品已被封印!!";
} }
JsonObject illustPreLoadData = CacheStoreCentral.getIllustPreLoadData(illustId, false); JsonObject illustPreLoadData = CacheStoreCentral.getCentral().getIllustPreLoadData(illustId, false);
// 在 Java 6 开始, 编译器会将用'+'进行的字符串拼接将自动转换成StringBuilder拼接 // 在 Java 6 开始, 编译器会将用'+'进行的字符串拼接将自动转换成StringBuilder拼接
return "色图姬帮你了解了这个作品的信息!\n" + "---------------- 作品信息 ----------------" + return "色图姬帮你了解了这个作品的信息!\n" + "---------------- 作品信息 ----------------" +
"\n作品Id: " + illustId + "\n作品Id: " + illustId +
"\n作品标题" + illustPreLoadData.get("illustTitle").getAsString() + "\n作品标题" + illustPreLoadData.get("illustTitle").getAsString() +
"\n作者(作者Id)" + illustPreLoadData.get("userName").getAsString() + "\n作者(作者Id)" + illustPreLoadData.get("userName").getAsString() +
"(" + illustPreLoadData.get("userId").getAsInt() + ")" + "(" + illustPreLoadData.get("userId").getAsInt() + ")" +
"\n点赞数" + illustPreLoadData.get(PreLoadDataComparator.Attribute.LIKE.attrName).getAsInt() + "\n点赞数" + illustPreLoadData.get(PreLoadDataAttribute.LIKE.attrName).getAsInt() +
"\n收藏数" + illustPreLoadData.get(PreLoadDataComparator.Attribute.BOOKMARK.attrName).getAsInt() + "\n收藏数" + illustPreLoadData.get(PreLoadDataAttribute.BOOKMARK.attrName).getAsInt() +
"\n围观数" + illustPreLoadData.get(PreLoadDataComparator.Attribute.VIEW.attrName).getAsInt() + "\n围观数" + illustPreLoadData.get(PreLoadDataAttribute.VIEW.attrName).getAsInt() +
"\n评论数" + illustPreLoadData.get(PreLoadDataComparator.Attribute.COMMENT.attrName).getAsInt() + "\n评论数" + illustPreLoadData.get(PreLoadDataAttribute.COMMENT.attrName).getAsInt() +
"\n页数" + illustPreLoadData.get(PreLoadDataComparator.Attribute.PAGE.attrName).getAsInt() + "" + "\n页数" + illustPreLoadData.get(PreLoadDataAttribute.PAGE.attrName).getAsInt() + "" +
"\n作品链接" + artworksLink(fromGroup, illustId) + "\n" + "\n作品链接" + artworksLink(fromGroup, illustId) + "\n" +
"---------------- 作品图片 ----------------\n" + "---------------- 作品图片 ----------------\n" +
CacheStoreCentral.getImageById(fromGroup, illustId, PageQuality.REGULAR, 1) + "\n" + CacheStoreCentral.getCentral().getImageById(fromGroup, illustId, PageQuality.REGULAR, 1) + "\n" +
"使用 \".cgj image -id " + "使用 \".cgj image -id " +
illustId + illustId +
"\" 获取原图。\n如有不当作品可使用\".cgj report -id " + "\" 获取原图。\n如有不当作品可使用\".cgj report -id " +
@ -140,8 +143,8 @@ public class BotCommandProcess {
@Argument(force = false, name = "date") Date queryTime, @Argument(force = false, name = "date") Date queryTime,
@Argument(force = false, name = "force") boolean force, @Argument(force = false, name = "force") boolean force,
@Argument(force = false, name = "mode", defaultValue = "DAILY") String contentMode, @Argument(force = false, name = "mode", defaultValue = "DAILY") String contentMode,
@Argument(force = false, name = "type", defaultValue = "ILLUST") String contentType @Argument(force = false, name = "type", defaultValue = "ALL") String contentType
) { ) throws InterruptedException {
Date queryDate = queryTime; Date queryDate = queryTime;
if (queryDate == null) { if (queryDate == null) {
queryDate = new Date(); queryDate = new Date();
@ -209,7 +212,7 @@ public class BotCommandProcess {
log.warn("配置项 {} 的参数值格式有误!", imageLimitPropertyKey); log.warn("配置项 {} 的参数值格式有误!", imageLimitPropertyKey);
} }
List<JsonObject> rankingInfoList = CacheStoreCentral List<JsonObject> rankingInfoList = CacheStoreCentral.getCentral()
.getRankingInfoByCache(type, mode, queryDate, 1, Math.max(0, itemLimit), false); .getRankingInfoByCache(type, mode, queryDate, 1, Math.max(0, itemLimit), false);
if(rankingInfoList.isEmpty()) { if(rankingInfoList.isEmpty()) {
return "无法查询排行榜,可能排行榜尚未更新。"; return "无法查询排行榜,可能排行榜尚未更新。";
@ -228,7 +231,7 @@ public class BotCommandProcess {
.append(pagesCount).append("p.\n"); .append(pagesCount).append("p.\n");
if (index <= imageLimit) { if (index <= imageLimit) {
resultBuilder resultBuilder
.append(CacheStoreCentral .append(CacheStoreCentral.getCentral()
.getImageById(fromGroup, illustId, PixivDownload.PageQuality.REGULAR, 1)) .getImageById(fromGroup, illustId, PixivDownload.PageQuality.REGULAR, 1))
.append("\n"); .append("\n");
} }
@ -279,12 +282,12 @@ public class BotCommandProcess {
return "参数无效, 请查看帮助信息"; return "参数无效, 请查看帮助信息";
} }
BufferMessageEvent event = new BufferMessageEvent(); BufferedMessageSender bufferedSender = new BufferedMessageSender();
RandomRankingArtworksSender artworksSender = RandomRankingArtworksSender artworksSender =
new RandomRankingArtworksSender(event, fromGroup, 1, 200, mode, type, new RandomRankingArtworksSender(bufferedSender, fromGroup, 1, 200, mode, type,
PageQuality.ORIGINAL); PageQuality.ORIGINAL);
artworksSender.send(); artworksSender.send();
return event.getBufferMessage(); return bufferedSender.getBufferContent();
} }
/** /**
@ -309,11 +312,11 @@ public class BotCommandProcess {
@Argument(name = "in", force = false) String includeKeywords, @Argument(name = "in", force = false) String includeKeywords,
@Argument(name = "ex", force = false) String excludeKeywords, @Argument(name = "ex", force = false) String excludeKeywords,
@Argument(name = "option", force = false) String contentOption, @Argument(name = "option", force = false) String contentOption,
@Argument(name = "page", force = false, defaultValue = "1") int pagesIndex @Argument(name = "p", force = false, defaultValue = "1") int pagesIndex
) throws IOException { ) throws IOException, InterruptedException {
log.info("正在执行搜索..."); log.debug("正在执行搜索...");
JsonObject resultBody = CacheStoreCentral JsonObject resultBody = CacheStoreCentral.getCentral()
.getSearchBody(content, type, area, includeKeywords, excludeKeywords, contentOption); .getSearchBody(content, type, area, includeKeywords, excludeKeywords, contentOption, pagesIndex);
StringBuilder result = new StringBuilder("内容 " + content + " 的搜索结果:\n"); StringBuilder result = new StringBuilder("内容 " + content + " 的搜索结果:\n");
log.debug("正在处理信息..."); log.debug("正在处理信息...");
@ -325,7 +328,7 @@ public class BotCommandProcess {
log.warn("参数转换异常!将使用默认值(" + limit + ")", e); log.warn("参数转换异常!将使用默认值(" + limit + ")", e);
} }
int totalCount = 0; int totalCount = 0;
for (PixivSearchBuilder.SearchArea searchArea : PixivSearchBuilder.SearchArea.values()) { for (PixivSearchLinkBuilder.SearchArea searchArea : PixivSearchLinkBuilder.SearchArea.values()) {
if (!resultBody.has(searchArea.jsonKey) || if (!resultBody.has(searchArea.jsonKey) ||
resultBody.getAsJsonObject(searchArea.jsonKey).getAsJsonArray("data").size() == 0) { resultBody.getAsJsonObject(searchArea.jsonKey).getAsJsonArray("data").size() == 0) {
log.debug("返回数据不包含 {}", searchArea.jsonKey); log.debug("返回数据不包含 {}", searchArea.jsonKey);
@ -335,9 +338,9 @@ public class BotCommandProcess {
.getAsJsonObject(searchArea.jsonKey).getAsJsonArray("data"); .getAsJsonObject(searchArea.jsonKey).getAsJsonArray("data");
ArrayList<JsonElement> illustsList = new ArrayList<>(); ArrayList<JsonElement> illustsList = new ArrayList<>();
illustsArray.forEach(illustsList::add); illustsArray.forEach(illustsList::add);
illustsList.sort(new PreLoadDataComparator(PreLoadDataComparator.Attribute.LIKE)); illustsList.sort(new PreLoadDataAttributeComparator(PreLoadDataAttribute.BOOKMARK));
log.info("已找到与 {} 相关插图信息({})", content, searchArea.name().toLowerCase()); log.debug("已找到与 {} 相关插图信息({})", content, searchArea.name().toLowerCase());
int count = 1; int count = 1;
for (JsonElement jsonElement : illustsList) { for (JsonElement jsonElement : illustsList) {
if (count > limit) { if (count > limit) {
@ -365,8 +368,17 @@ public class BotCommandProcess {
PixivURL.getPixivRefererLink(illustId) PixivURL.getPixivRefererLink(illustId)
); );
String imageMsg = String imageMsg;
CacheStoreCentral.getImageById(fromGroup, illustId, PixivDownload.PageQuality.REGULAR, 1); try {
imageMsg = CacheStoreCentral.getCentral()
.getImageById(fromGroup, illustId, PixivDownload.PageQuality.REGULAR, 1);
} catch (NoSuchElementException e) {
if(e.getMessage().startsWith("No work found: ")) {
log.warn("作品 {} 不存在, 跳过该作品...", illustId);
continue;
}
throw e;
}
if (isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) { if (isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) {
log.warn("作品Id {} 为R-18作品, 跳过.", illustId); log.warn("作品Id {} 为R-18作品, 跳过.", illustId);
continue; continue;
@ -375,20 +387,21 @@ public class BotCommandProcess {
continue; continue;
} }
JsonObject illustPreLoadData = CacheStoreCentral.getIllustPreLoadData(illustId, false); JsonObject illustPreLoadData = CacheStoreCentral.getCentral()
.getIllustPreLoadData(illustId, false);
result.append(searchArea.name()).append(" (").append(count).append(" / ") result.append(searchArea.name()).append(" (").append(count).append(" / ")
.append(limit).append(")\n\t作品id: ").append(illustId) .append(limit).append(")\n\t作品id: ").append(illustId)
.append(", \n\t作者名: ").append(illustObj.get("userName").getAsString()) .append(", \n\t作者名: ").append(illustObj.get("userName").getAsString())
.append("\n\t作品标题: ").append(illustObj.get("illustTitle").getAsString()) .append("\n\t作品标题: ").append(illustObj.get("illustTitle").getAsString())
.append("\n\t作品页数: ").append(illustObj.get("pageCount").getAsInt()).append("") .append("\n\t作品页数: ").append(illustObj.get("pageCount").getAsInt()).append("")
.append("\n\t点赞数") .append("\n\t点赞数")
.append(illustPreLoadData.get(PreLoadDataComparator.Attribute.LIKE.attrName).getAsInt()) .append(illustPreLoadData.get(PreLoadDataAttribute.LIKE.attrName).getAsInt())
.append("\n\t收藏数") .append("\n\t收藏数")
.append(illustPreLoadData.get(PreLoadDataComparator.Attribute.BOOKMARK.attrName).getAsInt()) .append(illustPreLoadData.get(PreLoadDataAttribute.BOOKMARK.attrName).getAsInt())
.append("\n\t围观数") .append("\n\t围观数")
.append(illustPreLoadData.get(PreLoadDataComparator.Attribute.VIEW.attrName).getAsInt()) .append(illustPreLoadData.get(PreLoadDataAttribute.VIEW.attrName).getAsInt())
.append("\n\t评论数") .append("\n\t评论数")
.append(illustPreLoadData.get(PreLoadDataComparator.Attribute.COMMENT.attrName).getAsInt()) .append(illustPreLoadData.get(PreLoadDataAttribute.COMMENT.attrName).getAsInt())
.append("\n").append(imageMsg).append("\n"); .append("\n").append(imageMsg).append("\n");
count++; count++;
totalCount++; totalCount++;
@ -463,7 +476,7 @@ public class BotCommandProcess {
static void clearCache() { static void clearCache() {
log.warn("正在清除所有缓存..."); log.warn("正在清除所有缓存...");
CacheStoreCentral.clearCache(); CacheStoreCentral.getCentral().clearCache();
File imageStoreDir = new File(BotGlobal.getGlobal().getDataStoreDir(), "data/image/cgj/"); File imageStoreDir = new File(BotGlobal.getGlobal().getDataStoreDir(), "data/image/cgj/");
File[] listFiles = imageStoreDir.listFiles(); File[] listFiles = imageStoreDir.listFiles();
if (listFiles == null) { if (listFiles == null) {
@ -482,9 +495,9 @@ public class BotCommandProcess {
@Argument(name = "$fromGroup") long fromGroup, @Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "id") int illustId, @Argument(name = "id") int illustId,
@Argument(name = "quality", force = false) PixivDownload.PageQuality quality, @Argument(name = "quality", force = false) PixivDownload.PageQuality quality,
@Argument(name = "page", force = false, defaultValue = "1") int pageIndex @Argument(name = "p", force = false, defaultValue = "1") int pageIndex
) { ) throws InterruptedException {
return CacheStoreCentral.getImageById(fromGroup, illustId, quality, pageIndex); return CacheStoreCentral.getCentral().getImageById(fromGroup, illustId, quality, pageIndex);
} }
/** /**
@ -521,6 +534,10 @@ public class BotCommandProcess {
return reportStore.exists(String.valueOf(illustId)); return reportStore.exists(String.valueOf(illustId));
} }
/**
* Tag过滤表达式
*/
private final static Pattern tagPattern = Pattern.compile(".*R-*18.*");
/** /**
* 检查指定作品是否为r18 * 检查指定作品是否为r18
* @param illustId 作品Id * @param illustId 作品Id
@ -532,12 +549,12 @@ public class BotCommandProcess {
*/ */
public static boolean isNoSafe(int illustId, Properties settingProp, boolean returnRaw) public static boolean isNoSafe(int illustId, Properties settingProp, boolean returnRaw)
throws IOException, NoSuchElementException { throws IOException, NoSuchElementException {
JsonObject illustInfo = CacheStoreCentral.getIllustInfo(illustId, false); JsonObject illustInfo = CacheStoreCentral.getCentral().getIllustInfo(illustId, false);
JsonArray tags = illustInfo.getAsJsonArray("tags"); JsonArray tags = illustInfo.getAsJsonArray("tags");
boolean rawValue = illustInfo.get("xRestrict").getAsInt() != 0; boolean rawValue = illustInfo.get("xRestrict").getAsInt() != 0;
if(!rawValue) { if(!rawValue) {
for(JsonElement tag : tags) { for(JsonElement tag : tags) {
boolean current = tag.getAsString().matches("R-*18") || tag.getAsString().contains("R18"); boolean current = tagPattern.matcher(tag.getAsString()).matches();
if (current) { if (current) {
rawValue = true; rawValue = true;
break; break;

View File

@ -4,7 +4,6 @@ import com.google.common.base.Throwables;
import org.slf4j.Logger; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
import java.lang.reflect.Field;
import java.util.*; import java.util.*;
import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicBoolean;
@ -13,7 +12,7 @@ import java.util.concurrent.atomic.AtomicBoolean;
*/ */
public class RandomIntervalSendTimer extends TimerTask { public class RandomIntervalSendTimer extends TimerTask {
private final static Timer timer = new Timer("Thread-RIST", true); private final static Timer timer = new Timer("Thread-RandomIntervalSendTimer", true);
private final static Logger log = LoggerFactory.getLogger(RandomIntervalSendTimer.class); private final static Logger log = LoggerFactory.getLogger(RandomIntervalSendTimer.class);
private final static Map<Long, RandomIntervalSendTimer> timerMap = new HashMap<>(); private final static Map<Long, RandomIntervalSendTimer> timerMap = new HashMap<>();
@ -23,7 +22,7 @@ public class RandomIntervalSendTimer extends TimerTask {
private final long time; private final long time;
private final int floatTime; private final int floatTime;
private final AtomicBoolean loop = new AtomicBoolean(); private final AtomicBoolean loop = new AtomicBoolean();
private final AtomicBoolean start = new AtomicBoolean(); private final AtomicBoolean running = new AtomicBoolean();
private final String hashId = Integer.toHexString(this.hashCode()); private final String hashId = Integer.toHexString(this.hashCode());
@ -77,12 +76,17 @@ public class RandomIntervalSendTimer extends TimerTask {
* @param startNow 现在开始 * @param startNow 现在开始
* @param loop 是否循环 * @param loop 是否循环
*/ */
private RandomIntervalSendTimer(long timerId, AutoSender sender, long time, int floatTime, boolean startNow, boolean loop) { private RandomIntervalSendTimer(
long timerId,
AutoSender sender,
long time,
int floatTime,
boolean startNow,
boolean loop) {
this.timerId = timerId; this.timerId = timerId;
this.sender = sender; this.sender = sender;
this.time = time; this.time = time;
this.floatTime = floatTime; this.floatTime = floatTime;
timerMap.put(timerId, this);
if(startNow) { if(startNow) {
start(loop); start(loop);
} }
@ -102,28 +106,29 @@ public class RandomIntervalSendTimer extends TimerTask {
Date nextDate = new Date(); Date nextDate = new Date();
nextDate.setTime(nextDate.getTime() + nextDelay); nextDate.setTime(nextDate.getTime() + nextDelay);
log.info("定时器 {} 下一延迟: {}ms ({})", hashId, nextDelay, nextDate); log.info("定时器 {} 下一延迟: {}ms ({})", hashId, nextDelay, nextDate);
if(start.get()) { if(running.get()) {
try { reset();
Field state = this.getClass().getSuperclass().getDeclaredField("state"); return;
state.setAccessible(true);
state.setInt(this, 0);
state.setAccessible(false);
} catch (NoSuchFieldException | IllegalAccessException e) {
e.printStackTrace();
return;
}
} }
start.set(true); running.set(true);
timer.schedule(this, nextDelay); timer.schedule(this, nextDelay);
} }
public void reset() {
timerMap.put(timerId, (RandomIntervalSendTimer) clone());
}
@Override @Override
public void run() { public void run() {
log.info("定时器 {} 开始执行...(Sender: {}@{})", this.hashId, sender.getClass().getSimpleName(), sender.hashCode());
try { try {
sender.send(); sender.send();
} catch (Exception e) { } catch (Exception e) {
log.error("定时器 {} 执行时发生异常:\n{}", Integer.toHexString(this.hashCode()), Throwables.getStackTraceAsString(e)); log.error("定时器 {} 执行时发生异常:\n{}",
Integer.toHexString(this.hashCode()),
Throwables.getStackTraceAsString(e));
} }
log.info("定时器 {} 执行结束.", this.hashId);
if (this.loop.get()) { if (this.loop.get()) {
start(); start();
} }
@ -135,7 +140,7 @@ public class RandomIntervalSendTimer extends TimerTask {
*/ */
@Override @Override
public boolean cancel() { public boolean cancel() {
start.set(false); running.set(false);
loop.set(false); loop.set(false);
return super.cancel(); return super.cancel();
} }
@ -148,4 +153,18 @@ public class RandomIntervalSendTimer extends TimerTask {
timerMap.remove(this.timerId); timerMap.remove(this.timerId);
} }
/**
* 克隆一个参数完全一样的TimerTask对象.
* @return 返回对象不同, 参数相同的TimerTask对象.
*/
@Override
@SuppressWarnings("MethodDoesntCallSuperMethod")
public Object clone() {
RandomIntervalSendTimer newTimerTask = new RandomIntervalSendTimer(
this.timerId, this.sender,
time, floatTime,
running.get(), loop.get());
this.destroy();
return newTimerTask;
}
} }

View File

@ -92,7 +92,7 @@ public class RandomRankingArtworksSender extends AutoSender {
int selectRanking = rankingStart + new Random().nextInt(rankingStop - rankingStart + 1); int selectRanking = rankingStart + new Random().nextInt(rankingStop - rankingStart + 1);
try { try {
List<JsonObject> rankingList = CacheStoreCentral.getRankingInfoByCache( List<JsonObject> rankingList = CacheStoreCentral.getCentral().getRankingInfoByCache(
contentType, contentType,
mode, mode,
queryDate, queryDate,
@ -119,7 +119,7 @@ public class RandomRankingArtworksSender extends AutoSender {
String message = "#美图推送 - 今日排行榜 第 " + rankingInfo.get("rank").getAsInt() + "\n" + String message = "#美图推送 - 今日排行榜 第 " + rankingInfo.get("rank").getAsInt() + "\n" +
"标题:" + rankingInfo.get("title").getAsString() + "(" + illustId + ")\n" + "标题:" + rankingInfo.get("title").getAsString() + "(" + illustId + ")\n" +
"作者:" + rankingInfo.get("user_name").getAsString() + "\n" + "作者:" + rankingInfo.get("user_name").getAsString() + "\n" +
CacheStoreCentral.getImageById(0, illustId, quality, 1) + CacheStoreCentral.getCentral().getImageById(0, illustId, quality, 1) +
"\n如有不当作品可使用\".cgj report -id " + illustId + "\"向色图姬反馈。"; "\n如有不当作品可使用\".cgj report -id " + illustId + "\"向色图姬反馈。";
getMessageSender().sendMessage(message); getMessageSender().sendMessage(message);
} catch (Exception e) { } catch (Exception e) {

View File

@ -27,7 +27,7 @@ public final class BotGlobal {
public static BotGlobal getGlobal() { public static BotGlobal getGlobal() {
if(instance == null) { if(instance == null) {
throw new IllegalStateException(""); throw new IllegalStateException("BotGlobal has not been initialized");
} }
return instance; return instance;
} }
@ -61,7 +61,7 @@ public final class BotGlobal {
try (Jedis jedis = this.redisServer.getResource()) { try (Jedis jedis = this.redisServer.getResource()) {
log.warn("Redis连接状态(Ping): {}", jedis.ping().equalsIgnoreCase("pong")); log.warn("Redis连接状态(Ping): {}", jedis.ping().equalsIgnoreCase("pong"));
} catch(JedisConnectionException e) { } catch(JedisConnectionException e) {
log.warn("Redis连接失败, 将会影响到后续功能运行.", e); log.warn("Redis连接失败, 将会影响到后续功能运行.({})", e.getCause().getMessage());
} }
String dataStoreDirPath = System.getProperty("cgj.botDataDir"); String dataStoreDirPath = System.getProperty("cgj.botDataDir");

View File

@ -18,7 +18,7 @@ public class AutoCleanTimer extends TimerTask {
private final static Logger log = LoggerFactory.getLogger(AutoCleanTimer.class); private final static Logger log = LoggerFactory.getLogger(AutoCleanTimer.class);
static { static {
cleanTimer.schedule(new AutoCleanTimer(), 100L); cleanTimer.schedule(new AutoCleanTimer(), 100L, 100L);
} }
/** /**

View File

@ -8,9 +8,12 @@ import net.lamgc.cgj.bot.BotCode;
import net.lamgc.cgj.bot.BotCommandProcess; import net.lamgc.cgj.bot.BotCommandProcess;
import net.lamgc.cgj.bot.SettingProperties; import net.lamgc.cgj.bot.SettingProperties;
import net.lamgc.cgj.bot.boot.BotGlobal; import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.exception.HttpRequestException;
import net.lamgc.cgj.pixiv.PixivDownload; import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivSearchBuilder; import net.lamgc.cgj.pixiv.PixivSearchLinkBuilder;
import net.lamgc.cgj.pixiv.PixivURL; import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.cgj.util.Locker;
import net.lamgc.cgj.util.LockerMap;
import net.lamgc.cgj.util.URLs; import net.lamgc.cgj.util.URLs;
import net.lamgc.utils.encrypt.MessageDigestUtils; import net.lamgc.utils.encrypt.MessageDigestUtils;
import net.lz1998.cq.utils.CQCode; import net.lz1998.cq.utils.CQCode;
@ -31,55 +34,80 @@ import java.util.concurrent.atomic.AtomicInteger;
@SuppressWarnings("SynchronizationOnLocalVariableOrMethodParameter") @SuppressWarnings("SynchronizationOnLocalVariableOrMethodParameter")
public final class CacheStoreCentral { public final class CacheStoreCentral {
private CacheStoreCentral() {}
private final static Logger log = LoggerFactory.getLogger(CacheStoreCentral.class); private final static Logger log = LoggerFactory.getLogger(CacheStoreCentral.class);
private final static Hashtable<String, File> imageCache = new Hashtable<>(); private static CacheStoreCentral central = new CacheStoreCentral();
private final static JsonRedisCacheStore imageChecksumCache = public static CacheStoreCentral getCentral() {
if(central == null) {
initialCentral();
}
return central;
}
private synchronized static void initialCentral() {
if(central != null) {
return;
}
central = new CacheStoreCentral();
}
private final LockerMap<String> lockerMap = new LockerMap<>();
private CacheStoreCentral() {}
private final Hashtable<String, File> imageCache = new Hashtable<>();
private final CacheStore<JsonElement> imageChecksumCache =
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(), new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(),
"imageChecksum", BotGlobal.getGlobal().getGson()); "imageChecksum", BotGlobal.getGlobal().getGson());
/*
* 注意:
* 在启用了远端缓存的情况下, 不允许滥用本地缓存
* 只有在处理命令中需要短时间大量存取的缓存项才能进行本地缓存(例如PreLoadData需要在排序中大量获取);
* 如果没有短时间大量存取的需要, 切勿使用本地缓存
*/
/** /**
* 作品信息缓存 - 不过期 * 作品信息缓存 - 不过期
*/ */
private final static CacheStore<JsonElement> illustInfoCache = private final CacheStore<JsonElement> illustInfoCache =
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(), new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(),
"illustInfo", BotGlobal.getGlobal().getGson()); "illustInfo", BotGlobal.getGlobal().getGson());
/** /**
* 作品信息预加载数据 - 有效期 2 小时, 本地缓存有效期1 ± 0.25 * 作品信息预加载数据 - 有效期 2 小时, 本地缓存有效期 0.5 ± 0.25 小时
*/ */
private final static CacheStore<JsonElement> illustPreLoadDataCache = private final CacheStore<JsonElement> illustPreLoadDataCache =
CacheStoreUtils.hashLocalHotDataStore( CacheStoreUtils.hashLocalHotDataStore(
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(), new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(),
"illustPreLoadData", BotGlobal.getGlobal().getGson()), "illustPreLoadData", BotGlobal.getGlobal().getGson()), 600000, 120000);
3600000, 900000);
/** /**
* 搜索内容缓存, 有效期 2 小时 * 搜索内容缓存, 有效期 2 小时
*/ */
private final static CacheStore<JsonElement> searchBodyCache = private final CacheStore<JsonElement> searchBodyCache =
new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(), new JsonRedisCacheStore(BotGlobal.getGlobal().getRedisServer(),
"searchBody", BotGlobal.getGlobal().getGson()); "searchBody", BotGlobal.getGlobal().getGson());
/** /**
* 排行榜缓存, 不过期 * 排行榜缓存, 不过期
*/ */
private final static CacheStore<List<JsonObject>> rankingCache = private final CacheStore<List<JsonObject>> rankingCache =
new JsonObjectRedisListCacheStore(BotGlobal.getGlobal().getRedisServer(), new JsonObjectRedisListCacheStore(BotGlobal.getGlobal().getRedisServer(),
"ranking", BotGlobal.getGlobal().getGson()); "ranking", BotGlobal.getGlobal().getGson());
/** /**
* 作品页面下载链接缓存 - 不过期 * 作品页面下载链接缓存 - 不过期
*/ */
private final static CacheStore<List<String>> pagesCache = private final CacheStore<List<String>> pagesCache =
new StringListRedisCacheStore(BotGlobal.getGlobal().getRedisServer(), "imagePages"); new StringListRedisCacheStore(BotGlobal.getGlobal().getRedisServer(), "imagePages");
/** /**
* 清空所有缓存 * 清空所有缓存
*/ */
public static void clearCache() { public void clearCache() {
imageCache.clear(); imageCache.clear();
illustInfoCache.clear(); illustInfoCache.clear();
illustPreLoadDataCache.clear(); illustPreLoadDataCache.clear();
@ -96,7 +124,7 @@ public final class CacheStoreCentral {
* @param pageIndex 指定页面索引, 从1开始 * @param pageIndex 指定页面索引, 从1开始
* @return 如果成功, 返回BotCode, 否则返回错误信息. * @return 如果成功, 返回BotCode, 否则返回错误信息.
*/ */
public static String getImageById(long fromGroup, int illustId, PixivDownload.PageQuality quality, int pageIndex) { public String getImageById(long fromGroup, int illustId, PixivDownload.PageQuality quality, int pageIndex) throws InterruptedException {
log.debug("IllustId: {}, Quality: {}, PageIndex: {}", illustId, quality.name(), pageIndex); log.debug("IllustId: {}, Quality: {}, PageIndex: {}", illustId, quality.name(), pageIndex);
if(pageIndex <= 0) { if(pageIndex <= 0) {
log.warn("指定的页数不能小于或等于0: {}", pageIndex); log.warn("指定的页数不能小于或等于0: {}", pageIndex);
@ -118,7 +146,7 @@ public final class CacheStoreCentral {
List<String> pagesList; List<String> pagesList;
try { try {
pagesList = CacheStoreCentral.getIllustPages(illustId, quality, false); pagesList = getIllustPages(illustId, quality, false);
} catch (IOException e) { } catch (IOException e) {
log.error("获取下载链接列表时发生异常", e); log.error("获取下载链接列表时发生异常", e);
return "发生网络异常,无法获取图片!"; return "发生网络异常,无法获取图片!";
@ -147,10 +175,10 @@ public final class CacheStoreCentral {
ImageChecksum imageChecksum = getImageChecksum(illustId, pageIndex); ImageChecksum imageChecksum = getImageChecksum(illustId, pageIndex);
if(imageChecksum != null) { if(imageChecksum != null) {
try { try {
log.debug("正在检查作品Id {} 第 {} 页图片文件 {} ...", illustId, pageIndex, imageFile.getName()); log.trace("正在检查作品Id {} 第 {} 页图片文件 {} ...", illustId, pageIndex, imageFile.getName());
if (ImageChecksum.checkFile(imageChecksum, Files.readAllBytes(imageFile.toPath()))) { if (ImageChecksum.checkFile(imageChecksum, Files.readAllBytes(imageFile.toPath()))) {
imageCache.put(URLs.getResourceName(downloadLink), imageFile); imageCache.put(URLs.getResourceName(downloadLink), imageFile);
log.debug("作品Id {} 第 {} 页缓存已补充.", illustId, pageIndex); log.trace("作品Id {} 第 {} 页缓存已补充.", illustId, pageIndex);
return getImageToBotCode(imageFile, false).toString(); return getImageToBotCode(imageFile, false).toString();
} else { } else {
log.warn("图片文件 {} 校验失败, 重新下载图片...", imageFile.getName()); log.warn("图片文件 {} 校验失败, 重新下载图片...", imageFile.getName());
@ -171,13 +199,13 @@ public final class CacheStoreCentral {
} }
} catch (InterruptedException e) { } catch (InterruptedException e) {
log.warn("图片缓存被中断", e); log.warn("图片缓存被中断", e);
return "(错误:图片获取超时)"; throw e;
} catch (Throwable e) { } catch (Throwable e) {
log.error("图片 {} 获取失败:\n{}", illustId + "p" + pageIndex, Throwables.getStackTraceAsString(e)); log.error("图片 {} 获取失败:\n{}", illustId + "p" + pageIndex, Throwables.getStackTraceAsString(e));
return "(错误: 图片获取出错)"; return "(错误: 图片获取出错)";
} }
} else { } else {
log.debug("图片 {} 缓存命中.", fileName); log.trace("图片 {} 缓存命中.", fileName);
} }
return getImageToBotCode(imageCache.get(fileName), false).toString(); return getImageToBotCode(imageCache.get(fileName), false).toString();
@ -190,7 +218,7 @@ public final class CacheStoreCentral {
* @return 返回设定好参数的BotCode * @return 返回设定好参数的BotCode
*/ */
@SuppressWarnings("SameParameterValue") @SuppressWarnings("SameParameterValue")
private static BotCode getImageToBotCode(File targetFile, boolean updateCache) { private BotCode getImageToBotCode(File targetFile, boolean updateCache) {
String fileName = Objects.requireNonNull(targetFile, "targetFile is null").getName(); String fileName = Objects.requireNonNull(targetFile, "targetFile is null").getName();
BotCode code = BotCode.parse( BotCode code = BotCode.parse(
CQCode.image(BotGlobal.getGlobal().getImageStoreDir().getName() + "/" + fileName)); CQCode.image(BotGlobal.getGlobal().getImageStoreDir().getName() + "/" + fileName));
@ -208,22 +236,28 @@ public final class CacheStoreCentral {
* @throws IOException 当Http请求发生异常时抛出 * @throws IOException 当Http请求发生异常时抛出
* @throws NoSuchElementException 当作品未找到时抛出 * @throws NoSuchElementException 当作品未找到时抛出
*/ */
public static JsonObject getIllustInfo(int illustId, boolean flushCache) public JsonObject getIllustInfo(int illustId, boolean flushCache)
throws IOException, NoSuchElementException { throws IOException, NoSuchElementException {
String illustIdStr = buildSyncKey(Integer.toString(illustId)); Locker<String> locker = buildSyncKey(Integer.toString(illustId));
String illustIdStr = locker.getKey();
JsonObject illustInfoObj = null; JsonObject illustInfoObj = null;
if (!illustInfoCache.exists(illustIdStr) || flushCache) { if (!illustInfoCache.exists(illustIdStr) || flushCache) {
synchronized (illustIdStr) { try {
if (!illustInfoCache.exists(illustIdStr) || flushCache) { locker.lock();
illustInfoObj = BotGlobal.getGlobal().getPixivDownload().getIllustInfoByIllustId(illustId); synchronized (locker) {
illustInfoCache.update(illustIdStr, illustInfoObj, null); if (!illustInfoCache.exists(illustIdStr) || flushCache) {
illustInfoObj = BotGlobal.getGlobal().getPixivDownload().getIllustInfoByIllustId(illustId);
illustInfoCache.update(illustIdStr, illustInfoObj, null);
}
} }
} finally {
locker.unlock();
} }
} }
if(Objects.isNull(illustInfoObj)) { if(Objects.isNull(illustInfoObj)) {
illustInfoObj = illustInfoCache.getCache(illustIdStr).getAsJsonObject(); illustInfoObj = illustInfoCache.getCache(illustIdStr).getAsJsonObject();
log.debug("作品Id {} IllustInfo缓存命中.", illustId); log.trace("作品Id {} IllustInfo缓存命中.", illustId);
} }
return illustInfoObj; return illustInfoObj;
} }
@ -236,66 +270,79 @@ public final class CacheStoreCentral {
* @return 成功返回JsonObject对象 * @return 成功返回JsonObject对象
* @throws IOException 当Http请求处理发生异常时抛出 * @throws IOException 当Http请求处理发生异常时抛出
*/ */
public static JsonObject getIllustPreLoadData(int illustId, boolean flushCache) throws IOException { public JsonObject getIllustPreLoadData(int illustId, boolean flushCache) throws IOException {
String illustIdStr = buildSyncKey(Integer.toString(illustId)); Locker<String> locker = buildSyncKey(Integer.toString(illustId));
String illustIdStr = locker.getKey();
JsonObject result = null; JsonObject result = null;
if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) { if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) {
synchronized (illustIdStr) { try {
if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) { locker.lock();
log.debug("IllustId {} 缓存失效, 正在更新...", illustId); synchronized (locker) {
JsonObject preLoadDataObj = BotGlobal.getGlobal().getPixivDownload() if (!illustPreLoadDataCache.exists(illustIdStr) || flushCache) {
.getIllustPreLoadDataById(illustId) log.trace("IllustId {} 缓存失效, 正在更新...", illustId);
.getAsJsonObject("illust") JsonObject preLoadDataObj = BotGlobal.getGlobal().getPixivDownload()
.getAsJsonObject(Integer.toString(illustId)); .getIllustPreLoadDataById(illustId)
.getAsJsonObject("illust")
.getAsJsonObject(Integer.toString(illustId));
long expire = 7200 * 1000; long expire = 7200 * 1000;
String propValue = SettingProperties. String propValue = SettingProperties.
getProperty(SettingProperties.GLOBAL, "cache.illustPreLoadData.expire", "7200000"); getProperty(SettingProperties.GLOBAL, "cache.illustPreLoadData.expire", "7200000");
log.debug("PreLoadData有效时间设定: {}", propValue); log.debug("PreLoadData有效时间设定: {}", propValue);
try { try {
expire = Long.parseLong(propValue); expire = Long.parseLong(propValue);
} catch (Exception e) { } catch (Exception e) {
log.warn("全局配置项 \"{}\" 值非法, 已使用默认值: {}", propValue, expire); log.warn("全局配置项 \"{}\" 值非法, 已使用默认值: {}", propValue, expire);
}
result = preLoadDataObj;
illustPreLoadDataCache.update(illustIdStr, preLoadDataObj, expire);
log.trace("作品Id {} preLoadData缓存已更新(有效时间: {})", illustId, expire);
} }
result = preLoadDataObj;
illustPreLoadDataCache.update(illustIdStr, preLoadDataObj, expire);
log.debug("作品Id {} preLoadData缓存已更新(有效时间: {})", illustId, expire);
} }
} finally {
locker.unlock();
} }
} }
if(Objects.isNull(result)) { if(Objects.isNull(result)) {
result = illustPreLoadDataCache.getCache(illustIdStr).getAsJsonObject(); result = illustPreLoadDataCache.getCache(illustIdStr).getAsJsonObject();
log.debug("作品Id {} PreLoadData缓存命中.", illustId); log.trace("作品Id {} PreLoadData缓存命中.", illustId);
} }
return result; return result;
} }
public static List<String> getIllustPages(int illustId, PixivDownload.PageQuality quality, boolean flushCache) public List<String> getIllustPages(int illustId, PixivDownload.PageQuality quality, boolean flushCache)
throws IOException { throws IOException {
String pagesSign = buildSyncKey(Integer.toString(illustId), ".", quality.name()); Locker<String> locker
= buildSyncKey(Integer.toString(illustId), ".", quality.name());
String pagesSign = locker.getKey();
List<String> result = null; List<String> result = null;
if (!pagesCache.exists(pagesSign) || flushCache) { if (!pagesCache.exists(pagesSign) || flushCache) {
synchronized (pagesSign) { try {
if (!pagesCache.exists(pagesSign) || flushCache) { locker.lock();
List<String> linkList = PixivDownload synchronized (locker) {
.getIllustAllPageDownload(BotGlobal.getGlobal().getPixivDownload().getHttpClient(), if (!pagesCache.exists(pagesSign) || flushCache) {
BotGlobal.getGlobal().getPixivDownload().getCookieStore(), illustId, quality); List<String> linkList = PixivDownload
result = linkList; .getIllustAllPageDownload(BotGlobal.getGlobal().getPixivDownload().getHttpClient(),
pagesCache.update(pagesSign, linkList, null); BotGlobal.getGlobal().getPixivDownload().getCookieStore(), illustId, quality);
result = linkList;
pagesCache.update(pagesSign, linkList, null);
}
} }
} finally {
locker.unlock();
} }
} }
if(Objects.isNull(result)) { if(Objects.isNull(result)) {
result = pagesCache.getCache(pagesSign); result = pagesCache.getCache(pagesSign);
log.debug("作品Id {} Pages缓存命中.", illustId); log.trace("作品Id {} Pages缓存命中.", illustId);
} }
return result; return result;
} }
private final static Random expireTimeFloatRandom = new Random(); private final Random expireTimeFloatRandom = new Random();
/** /**
* 获取排行榜 * 获取排行榜
* @param contentType 排行榜类型 * @param contentType 排行榜类型
@ -307,7 +354,7 @@ public final class CacheStoreCentral {
* @return 成功返回有值List, 失败且无异常返回空 * @return 成功返回有值List, 失败且无异常返回空
* @throws IOException 获取异常时抛出 * @throws IOException 获取异常时抛出
*/ */
public static List<JsonObject> getRankingInfoByCache(PixivURL.RankingContentType contentType, public List<JsonObject> getRankingInfoByCache(PixivURL.RankingContentType contentType,
PixivURL.RankingMode mode, PixivURL.RankingMode mode,
Date queryDate, int start, int range, boolean flushCache) Date queryDate, int start, int range, boolean flushCache)
throws IOException { throws IOException {
@ -324,31 +371,37 @@ public final class CacheStoreCentral {
} }
String date = new SimpleDateFormat("yyyyMMdd").format(queryDate); String date = new SimpleDateFormat("yyyyMMdd").format(queryDate);
String requestSign = buildSyncKey(contentType.name(), ".", mode.name(), ".", date); Locker<String> locker
= buildSyncKey(contentType.name(), ".", mode.name(), ".", date);
String requestSign = locker.getKey();
List<JsonObject> result = null; List<JsonObject> result = null;
if(!rankingCache.exists(requestSign) || flushCache) { if(!rankingCache.exists(requestSign) || flushCache) {
synchronized(requestSign) { try {
if(!rankingCache.exists(requestSign) || flushCache) { locker.lock();
log.debug("Ranking缓存失效, 正在更新...(RequestSign: {})", requestSign); synchronized (locker) {
List<JsonObject> rankingResult = BotGlobal.getGlobal().getPixivDownload() if (!rankingCache.exists(requestSign) || flushCache) {
.getRanking(contentType, mode, queryDate, 1, 500); log.trace("Ranking缓存失效, 正在更新...(RequestSign: {})", requestSign);
long expireTime = 0; List<JsonObject> rankingResult = BotGlobal.getGlobal().getPixivDownload()
if(rankingResult.size() == 0) { .getRanking(contentType, mode, queryDate, 1, 500);
expireTime = 5400000 + expireTimeFloatRandom.nextInt(1800000); long expireTime = 0;
log.warn("数据获取失败, 将设置浮动有效时间以准备下次更新. (ExpireTime: {}ms)", expireTime); if (rankingResult.size() == 0) {
expireTime = 5400000 + expireTimeFloatRandom.nextInt(1800000);
log.warn("数据获取失败, 将设置浮动有效时间以准备下次更新. (ExpireTime: {}ms)", expireTime);
}
result = new ArrayList<>(rankingResult).subList(start - 1, start + range - 1);
rankingCache.update(requestSign, rankingResult, expireTime);
log.trace("Ranking缓存更新完成.(RequestSign: {})", requestSign);
} }
result = new ArrayList<>(rankingResult).subList(start - 1, start + range - 1);
rankingCache.update(requestSign, rankingResult, expireTime);
log.debug("Ranking缓存更新完成.(RequestSign: {})", requestSign);
} }
} finally {
locker.unlock();
} }
} }
if (Objects.isNull(result)) { if (Objects.isNull(result)) {
result = rankingCache.getCache(requestSign, start - 1, range); result = rankingCache.getCache(requestSign, start - 1, range);
log.debug("RequestSign [{}] 缓存命中.", requestSign); log.trace("RequestSign [{}] 缓存命中.", requestSign);
} }
log.debug("Result-Length: {}", result.size());
return PixivDownload.getRanking(result, start - 1, range); return PixivDownload.getRanking(result, start - 1, range);
} }
@ -363,31 +416,34 @@ public final class CacheStoreCentral {
* @return 返回完整搜索结果 * @return 返回完整搜索结果
* @throws IOException 当请求发生异常, 或接口返回异常信息时抛出. * @throws IOException 当请求发生异常, 或接口返回异常信息时抛出.
*/ */
public static JsonObject getSearchBody( public JsonObject getSearchBody(
String content, String content,
String type, String type,
String area, String area,
String includeKeywords, String includeKeywords,
String excludeKeywords, String excludeKeywords,
String contentOption) throws IOException { String contentOption,
PixivSearchBuilder searchBuilder = new PixivSearchBuilder(Strings.isNullOrEmpty(content) ? "" : content); int pageIndex
) throws IOException {
PixivSearchLinkBuilder searchBuilder = new PixivSearchLinkBuilder(Strings.isNullOrEmpty(content) ? "" : content);
if (type != null) { if (type != null) {
try { try {
searchBuilder.setSearchType(PixivSearchBuilder.SearchType.valueOf(type.toUpperCase())); searchBuilder.setSearchType(PixivSearchLinkBuilder.SearchType.valueOf(type.toUpperCase()));
} catch (IllegalArgumentException e) { } catch (IllegalArgumentException e) {
log.warn("不支持的SearchType: {}", type); log.warn("不支持的SearchType: {}", type);
} }
} }
if (area != null) { if (area != null) {
try { try {
searchBuilder.setSearchArea(PixivSearchBuilder.SearchArea.valueOf(area)); searchBuilder.setSearchArea(PixivSearchLinkBuilder.SearchArea.valueOf(area));
} catch (IllegalArgumentException e) { } catch (IllegalArgumentException e) {
log.warn("不支持的SearchArea: {}", area); log.warn("不支持的SearchArea: {}", area);
} }
} }
if (contentOption != null) { if (contentOption != null) {
try { try {
searchBuilder.setSearchContentOption(PixivSearchBuilder.SearchContentOption.valueOf(contentOption)); searchBuilder.setSearchContentOption(
PixivSearchLinkBuilder.SearchContentOption.valueOf(contentOption.trim().toUpperCase()));
} catch (IllegalArgumentException e) { } catch (IllegalArgumentException e) {
log.warn("不支持的SearchContentOption: {}", contentOption); log.warn("不支持的SearchContentOption: {}", contentOption);
} }
@ -397,58 +453,69 @@ public final class CacheStoreCentral {
for (String keyword : includeKeywords.split(";")) { for (String keyword : includeKeywords.split(";")) {
searchBuilder.removeExcludeKeyword(keyword.trim()); searchBuilder.removeExcludeKeyword(keyword.trim());
searchBuilder.addIncludeKeyword(keyword.trim()); searchBuilder.addIncludeKeyword(keyword.trim());
log.debug("已添加关键字: {}", keyword); log.trace("已添加关键字: {}", keyword);
} }
} }
if (!Strings.isNullOrEmpty(excludeKeywords)) { if (!Strings.isNullOrEmpty(excludeKeywords)) {
for (String keyword : excludeKeywords.split(";")) { for (String keyword : excludeKeywords.split(";")) {
searchBuilder.removeIncludeKeyword(keyword.trim()); searchBuilder.removeIncludeKeyword(keyword.trim());
searchBuilder.addExcludeKeyword(keyword.trim()); searchBuilder.addExcludeKeyword(keyword.trim());
log.debug("已添加排除关键字: {}", keyword); log.trace("已添加排除关键字: {}", keyword);
} }
} }
log.info("正在搜索作品, 条件: {}", searchBuilder.getSearchCondition()); if(pageIndex > 0) {
searchBuilder.setPage(pageIndex);
}
String requestUrl = searchBuilder.buildURL().intern(); log.debug("正在搜索作品, 条件: {}", searchBuilder.getSearchCondition());
Locker<String> locker
= buildSyncKey(searchBuilder.buildURL());
String requestUrl = locker.getKey();
log.debug("RequestUrl: {}", requestUrl); log.debug("RequestUrl: {}", requestUrl);
JsonObject resultBody = null; JsonObject resultBody = null;
if(!searchBodyCache.exists(requestUrl)) { if(!searchBodyCache.exists(requestUrl)) {
synchronized (requestUrl) { try {
if (!searchBodyCache.exists(requestUrl)) { locker.lock();
log.debug("searchBody缓存失效, 正在更新..."); synchronized (locker) {
JsonObject jsonObject; if (!searchBodyCache.exists(requestUrl)) {
HttpGet httpGetRequest = BotGlobal.getGlobal().getPixivDownload(). log.trace("searchBody缓存失效, 正在更新...");
createHttpGetRequest(requestUrl); JsonObject jsonObject;
HttpResponse response = BotGlobal.getGlobal().getPixivDownload(). HttpGet httpGetRequest = BotGlobal.getGlobal().getPixivDownload().
getHttpClient().execute(httpGetRequest); createHttpGetRequest(requestUrl);
HttpResponse response = BotGlobal.getGlobal().getPixivDownload().
getHttpClient().execute(httpGetRequest);
String responseBody = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8); String responseBody = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
log.debug("ResponseBody: {}", responseBody); log.trace("ResponseBody: {}", responseBody);
jsonObject = BotGlobal.getGlobal().getGson().fromJson(responseBody, JsonObject.class); jsonObject = BotGlobal.getGlobal().getGson().fromJson(responseBody, JsonObject.class);
if (jsonObject.get("error").getAsBoolean()) { if (jsonObject.get("error").getAsBoolean()) {
log.error("接口请求错误, 错误信息: {}", jsonObject.get("message").getAsString()); log.error("接口请求错误, 错误信息: {}", jsonObject.get("message").getAsString());
throw new IOException("Interface Request Error: " + jsonObject.get("message").getAsString()); throw new HttpRequestException(response.getStatusLine(), responseBody);
}
long expire = 7200 * 1000;
String propValue = SettingProperties
.getProperty(SettingProperties.GLOBAL, "cache.searchBody.expire", "7200000");
try {
expire = Long.parseLong(propValue);
} catch (Exception e) {
log.warn("全局配置项 \"{}\" 值非法, 已使用默认值: {}", propValue, expire);
}
resultBody = jsonObject.getAsJsonObject().getAsJsonObject("body");
searchBodyCache.update(requestUrl, jsonObject, expire);
log.trace("searchBody缓存已更新(有效时间: {})", expire);
} else {
log.trace("搜索缓存命中.");
} }
long expire = 7200 * 1000;
String propValue = SettingProperties
.getProperty(SettingProperties.GLOBAL, "cache.searchBody.expire", "7200000");
try {
expire = Long.parseLong(propValue);
} catch (Exception e) {
log.warn("全局配置项 \"{}\" 值非法, 已使用默认值: {}", propValue, expire);
}
resultBody = jsonObject.getAsJsonObject().getAsJsonObject("body");
searchBodyCache.update(requestUrl, jsonObject, expire);
log.debug("searchBody缓存已更新(有效时间: {})", expire);
} else {
log.debug("搜索缓存命中.");
} }
} finally {
locker.unlock();
} }
} else { } else {
log.debug("搜索缓存命中."); log.trace("搜索缓存命中.");
} }
if(Objects.isNull(resultBody)) { if(Objects.isNull(resultBody)) {
@ -457,7 +524,7 @@ public final class CacheStoreCentral {
return resultBody; return resultBody;
} }
protected static ImageChecksum getImageChecksum(int illustId, int pageIndex) { protected ImageChecksum getImageChecksum(int illustId, int pageIndex) {
String cacheKey = illustId + ":" + pageIndex; String cacheKey = illustId + ":" + pageIndex;
if(!imageChecksumCache.exists(cacheKey)) { if(!imageChecksumCache.exists(cacheKey)) {
return null; return null;
@ -466,7 +533,7 @@ public final class CacheStoreCentral {
} }
} }
protected static void setImageChecksum(ImageChecksum checksum) { protected void setImageChecksum(ImageChecksum checksum) {
String cacheKey = checksum.getIllustId() + ":" + checksum.getPage(); String cacheKey = checksum.getIllustId() + ":" + checksum.getPage();
imageChecksumCache.update(cacheKey, ImageChecksum.toJsonObject(checksum), 0); imageChecksumCache.update(cacheKey, ImageChecksum.toJsonObject(checksum), 0);
} }
@ -476,12 +543,12 @@ public final class CacheStoreCentral {
* @param keys String对象 * @param keys String对象
* @return 合并后, 如果常量池存在合并后的结果, 则返回常量池中的对象, 否则存入常量池后返回. * @return 合并后, 如果常量池存在合并后的结果, 则返回常量池中的对象, 否则存入常量池后返回.
*/ */
private static String buildSyncKey(String... keys) { private Locker<String> buildSyncKey(String... keys) {
StringBuilder sb = new StringBuilder(); StringBuilder sb = new StringBuilder();
for (String string : keys) { for (String string : keys) {
sb.append(string); sb.append(string);
} }
return sb.toString().intern(); return lockerMap.createLocker(sb.toString(), true);
} }
/** /**

View File

@ -75,6 +75,9 @@ public class HotDataCacheStore<T> implements CacheStore<T>, Cleanable {
log.trace("Current缓存库更新完成."); log.trace("Current缓存库更新完成.");
result = parentResult; result = parentResult;
} else { } else {
// 更新该Key的过期时间
current.update(key, result,
expireTime + (expireFloatRange <= 0 ? 0 : random.nextInt(expireFloatRange)));
log.trace("Current缓存库缓存命中."); log.trace("Current缓存库缓存命中.");
} }
return result; return result;
@ -138,10 +141,14 @@ public class HotDataCacheStore<T> implements CacheStore<T>, Cleanable {
* <p>该方法仅清理Current缓存库, 不会对上游缓存库造成影响.</p> * <p>该方法仅清理Current缓存库, 不会对上游缓存库造成影响.</p>
*/ */
@Override @Override
public void clean() { public void clean() throws Exception {
for(String key : this.current.keys()) { if(current instanceof Cleanable) {
if(current.exists(key)) { ((Cleanable) current).clean();
current.remove(key); } else {
for(String key : this.current.keys()) {
if (!current.exists(key)) {
current.remove(key);
}
} }
} }
} }

View File

@ -1,7 +1,7 @@
package net.lamgc.cgj.bot.cache; package net.lamgc.cgj.bot.cache;
import net.lamgc.cgj.bot.boot.BotGlobal; import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.cache.exception.HttpRequestException; import net.lamgc.cgj.exception.HttpRequestException;
import net.lamgc.cgj.pixiv.PixivURL; import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.cgj.util.URLs; import net.lamgc.cgj.util.URLs;
import net.lamgc.utils.event.EventHandler; import net.lamgc.utils.event.EventHandler;
@ -38,7 +38,7 @@ public class ImageCacheHandler implements EventHandler {
} }
try { try {
log.info("图片 {} Event正在进行...({})", event.getStoreFile().getName(), Integer.toHexString(event.hashCode())); log.debug("图片 {} Event正在进行...({})", event.getStoreFile().getName(), Integer.toHexString(event.hashCode()));
File storeFile = event.getStoreFile(); File storeFile = event.getStoreFile();
log.debug("正在缓存图片 {} (Path: {})", storeFile.getName(), storeFile.getAbsolutePath()); log.debug("正在缓存图片 {} (Path: {})", storeFile.getName(), storeFile.getAbsolutePath());
try { try {
@ -66,7 +66,7 @@ public class ImageCacheHandler implements EventHandler {
throw requestException; throw requestException;
} }
log.debug("正在下载...(Content-Length: {}KB)", response.getEntity().getContentLength() / 1024); log.trace("正在下载...(Content-Length: {}KB)", response.getEntity().getContentLength() / 1024);
ByteArrayOutputStream bufferOutputStream = new ByteArrayOutputStream(); ByteArrayOutputStream bufferOutputStream = new ByteArrayOutputStream();
try(FileOutputStream fileOutputStream = new FileOutputStream(storeFile)) { try(FileOutputStream fileOutputStream = new FileOutputStream(storeFile)) {
Streams.copy(response.getEntity().getContent(), bufferOutputStream, false); Streams.copy(response.getEntity().getContent(), bufferOutputStream, false);
@ -80,14 +80,14 @@ public class ImageCacheHandler implements EventHandler {
); );
bufferInputStream.reset(); bufferInputStream.reset();
Streams.copy(bufferInputStream, fileOutputStream, false); Streams.copy(bufferInputStream, fileOutputStream, false);
CacheStoreCentral.setImageChecksum(imageChecksum); CacheStoreCentral.getCentral().setImageChecksum(imageChecksum);
} catch (IOException e) { } catch (IOException e) {
log.error("下载图片时发生异常", e); log.error("下载图片时发生异常", e);
throw e; throw e;
} }
event.getImageCache().put(URLs.getResourceName(event.getDownloadLink()), storeFile); event.getImageCache().put(URLs.getResourceName(event.getDownloadLink()), storeFile);
} finally { } finally {
log.info("图片 {} Event结束({})", event.getStoreFile().getName(), Integer.toHexString(event.hashCode())); log.debug("图片 {} Event结束({})", event.getStoreFile().getName(), Integer.toHexString(event.hashCode()));
cacheQueue.remove(event); cacheQueue.remove(event);
} }
} }

View File

@ -7,7 +7,7 @@ import com.google.gson.JsonObject;
import org.slf4j.Logger; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
import net.lamgc.cgj.bot.cache.exception.HttpRequestException; import net.lamgc.cgj.exception.HttpRequestException;
import java.util.Hashtable; import java.util.Hashtable;
import java.util.Map; import java.util.Map;
@ -66,17 +66,17 @@ public final class ImageCacheStore {
// 置任务状态 // 置任务状态
task.taskState.set(TaskState.RUNNING); task.taskState.set(TaskState.RUNNING);
Throwable throwable = null; Future<Throwable> future = imageCacheExecutor.submit(() -> {
try {
handler.getImageToCache(cacheObject);
} catch (Throwable e) {
return e;
}
return null;
});
Throwable throwable;
try { try {
throwable = imageCacheExecutor.submit(() -> { throwable = future.get();
try {
handler.getImageToCache(cacheObject);
} catch (Throwable e) {
return e;
}
return null;
}).get();
if(throwable == null) { if(throwable == null) {
task.taskState.set(TaskState.COMPLETE); task.taskState.set(TaskState.COMPLETE);
} else { } else {
@ -84,6 +84,12 @@ public final class ImageCacheStore {
} }
} catch (ExecutionException e) { } catch (ExecutionException e) {
log.error("执行图片缓存任务时发生异常", e); log.error("执行图片缓存任务时发生异常", e);
task.taskState.set(TaskState.ERROR);
return e.getCause();
} catch (InterruptedException e) {
future.cancel(true);
task.taskState.set(TaskState.ERROR);
throw e;
} }
return throwable; return throwable;
} finally { } finally {

View File

@ -2,10 +2,7 @@ package net.lamgc.cgj.bot.cache;
import org.jetbrains.annotations.NotNull; import org.jetbrains.annotations.NotNull;
import java.util.Date; import java.util.*;
import java.util.Hashtable;
import java.util.Objects;
import java.util.Set;
import java.util.concurrent.atomic.AtomicReference; import java.util.concurrent.atomic.AtomicReference;
/** /**
@ -149,19 +146,22 @@ public class LocalHashCacheStore<T> implements CacheStore<T>, Cleanable {
} }
@Override @Override
public void clean() throws Exception { public void clean() {
Date currentDate = new Date(); Date currentDate = new Date();
Set<String> expireKeySet = new HashSet<>();
cache.forEach((key, value) -> { cache.forEach((key, value) -> {
if(value.isExpire(currentDate)) { if(value.isExpire(currentDate)) {
cache.remove(key); expireKeySet.add(key);
} }
}); });
expireKeySet.forEach(cache::remove);
} }
public static class CacheObject<T> implements Comparable<CacheObject<T>> { private static class CacheObject<T> implements Comparable<CacheObject<T>> {
private AtomicReference<T> value; private final AtomicReference<T> value;
private AtomicReference<Date> expire; private final AtomicReference<Date> expire;
public CacheObject(T value, Date expire) { public CacheObject(T value, Date expire) {
this.value = new AtomicReference<>(value); this.value = new AtomicReference<>(value);

View File

@ -51,27 +51,27 @@ abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
@Override @Override
public void update(String key, T value, Date expire) { public void update(String key, T value, Date expire) {
try (Jedis jedis = jedisPool.getResource()) { executeJedisCommand(jedis -> {
jedis.set(keyPrefix + key, parse(value)); jedis.set(keyPrefix + key, parse(value));
if(expire != null) { if(expire != null) {
jedis.pexpireAt(keyPrefix + key, expire.getTime()); jedis.pexpireAt(keyPrefix + key, expire.getTime());
log.debug("已设置Key {} 的过期时间(Expire: {})", key, expire.getTime()); log.debug("已设置Key {} 的过期时间(Expire: {})", key, expire.getTime());
} }
} });
} }
@Override @Override
public T getCache(String key) { public T getCache(String key) {
try (Jedis jedis = jedisPool.getResource()) { return executeJedisCommand(jedis -> {
return analysis(jedis.get(keyPrefix + key)); return analysis(jedis.get(keyPrefix + key));
} });
} }
@Override @Override
public boolean exists(String key) { public boolean exists(String key) {
try (Jedis jedis = jedisPool.getResource()) { return executeJedisCommand(jedis -> {
return jedis.exists(keyPrefix + key); return jedis.exists(keyPrefix + key);
} });
} }
@Override @Override
@ -81,23 +81,21 @@ abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
@Override @Override
public boolean clear() { public boolean clear() {
try (Jedis jedis = jedisPool.getResource()) { return executeJedisCommand(jedis -> {
return jedis.flushDB().equalsIgnoreCase("ok"); return jedis.flushDB().equalsIgnoreCase("ok");
} });
} }
@Override @Override
public Set<String> keys() { public Set<String> keys() {
try (Jedis jedis = jedisPool.getResource()) { return executeJedisCommand(jedis -> {
return jedis.keys(keyPrefix + "*"); return jedis.keys(keyPrefix + "*");
} });
} }
@Override @Override
public boolean remove(String key) { public boolean remove(String key) {
try (Jedis jedis = jedisPool.getResource()) { return executeJedisCommand(jedis -> jedis.del(keyPrefix + key) == 1);
return jedis.del(keyPrefix + key) == 1;
}
} }
/** /**

View File

@ -26,7 +26,6 @@ import java.text.SimpleDateFormat;
import java.util.*; import java.util.*;
import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.TimeUnit; import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.regex.Matcher; import java.util.regex.Matcher;
import java.util.regex.Pattern; import java.util.regex.Pattern;
@ -40,17 +39,15 @@ public class BotEventHandler implements EventHandler {
private final static Logger log = LoggerFactory.getLogger(BotEventHandler.class); private final static Logger log = LoggerFactory.getLogger(BotEventHandler.class);
private final static Map<Long, AtomicBoolean> muteStateMap = new Hashtable<>();
/** /**
* 消息事件执行器 * 消息事件执行器
*/ */
private final static EventExecutor executor = new EventExecutor(new TimeLimitThreadPoolExecutor( private final static EventExecutor executor = new EventExecutor(new TimeLimitThreadPoolExecutor(
0, 180000, // 3min limit
Math.max(Runtime.getRuntime().availableProcessors(), 4), Math.max(Runtime.getRuntime().availableProcessors(), 4), // 4 ~ processors
Math.max(Math.max(Runtime.getRuntime().availableProcessors() * 2, 4), 32), Math.min(Math.max(Runtime.getRuntime().availableProcessors() * 2, 8), 32),// (8 ~ processors * 2) ~ 32
30L, 30L,
TimeUnit.SECONDS, TimeUnit.SECONDS,
new LinkedBlockingQueue<>(1536), new LinkedBlockingQueue<>(1536),
new ThreadFactoryBuilder() new ThreadFactoryBuilder()
.setNameFormat("CommandProcess-%d") .setNameFormat("CommandProcess-%d")
@ -69,11 +66,15 @@ public class BotEventHandler implements EventHandler {
return; return;
} }
executor.setEnableEventResend(true);
executor.setEventUncaughtExceptionHandler(new EventUncaughtExceptionHandler() { executor.setEventUncaughtExceptionHandler(new EventUncaughtExceptionHandler() {
private final Logger log = LoggerFactory.getLogger(this.getClass()); private final Logger log = LoggerFactory.getLogger(this.getClass());
@Override @Override
public void exceptionHandler(Thread executeThread, EventHandler handler, Method handlerMethod, EventObject event, Throwable cause) { public void exceptionHandler(Thread executeThread, EventHandler handler, Method handlerMethod, EventObject event, Throwable cause) {
log.error("发生未捕获异常:\nThread:{}, EventHandler: {}, HandlerMethod: {}, EventObject: {}\n{}", log.error("EventExecutor@{} 发生未捕获异常:\n\t" +
"Thread:{}\n\tEventHandler: {}\n\tHandlerMethod: {}\n\tEventObject: {}\n" +
"------------------ Stack Trace ------------------\n{}",
executor.hashCode(),
executeThread.getName(), executeThread.getName(),
handler.toString(), handler.toString(),
handlerMethod.getName(), handlerMethod.getName(),
@ -105,8 +106,6 @@ public class BotEventHandler implements EventHandler {
runnerConfig.addStringParameterParser(new DateParser(new SimpleDateFormat("yyyy-MM-dd"))); runnerConfig.addStringParameterParser(new DateParser(new SimpleDateFormat("yyyy-MM-dd")));
runnerConfig.addStringParameterParser(new PagesQualityParser()); runnerConfig.addStringParameterParser(new PagesQualityParser());
log.debug("DateParser添加情况: {}", runnerConfig.hasStringParameterParser(Date.class));
processRunner = new ArgumentsRunner(BotCommandProcess.class, runnerConfig); processRunner = new ArgumentsRunner(BotCommandProcess.class, runnerConfig);
adminRunner = new ArgumentsRunner(BotAdminCommandProcess.class, runnerConfig); adminRunner = new ArgumentsRunner(BotAdminCommandProcess.class, runnerConfig);
@ -130,6 +129,7 @@ public class BotEventHandler implements EventHandler {
/** /**
* 投递消息事件 * 投递消息事件
* @param event 事件对象 * @param event 事件对象
* @param sync 是否同步执行事件
*/ */
@NotAccepted @NotAccepted
public static void executeMessageEvent(MessageEvent event, boolean sync) throws InterruptedException { public static void executeMessageEvent(MessageEvent event, boolean sync) throws InterruptedException {
@ -164,9 +164,6 @@ public class BotEventHandler implements EventHandler {
log.debug(event.toString()); log.debug(event.toString());
if(mismatch(msg)) { if(mismatch(msg)) {
return; return;
} else if(isMute(event.getFromGroup())) {
log.debug("机器人已被禁言, 忽略请求.");
return;
} }
Pattern pattern = Pattern.compile("/\\s*(\".+?\"|[^:\\s])+((\\s*:\\s*(\".+?\"|[^\\s])+)|)|(\".+?\"|[^\"\\s])+"); Pattern pattern = Pattern.compile("/\\s*(\".+?\"|[^:\\s])+((\\s*:\\s*(\".+?\"|[^\\s])+)|)|(\".+?\"|[^\"\\s])+");
@ -221,8 +218,8 @@ public class BotEventHandler implements EventHandler {
} catch(DeveloperRunnerException e) { } catch(DeveloperRunnerException e) {
Throwable cause = e.getCause(); Throwable cause = e.getCause();
if (cause instanceof InterruptedException) { if (cause instanceof InterruptedException) {
log.error("命令执行超时, 终止执行."); log.error("命令执行超时, 终止执行.", cause);
result = "色图姬发现这个命令的处理时间太久了!所以打断了这个命令。"; result = "色图姬查阅图库太久,被赶出来了!";
} else if(cause instanceof NoSuchElementException && cause.getMessage().startsWith("No work found: ")) { } else if(cause instanceof NoSuchElementException && cause.getMessage().startsWith("No work found: ")) {
String message = cause.getMessage(); String message = cause.getMessage();
log.error("指定作品不存在.(Id: {})", message.substring(message.lastIndexOf(": ") + 2)); log.error("指定作品不存在.(Id: {})", message.substring(message.lastIndexOf(": ") + 2));
@ -233,17 +230,18 @@ public class BotEventHandler implements EventHandler {
} }
} }
long processTime = System.currentTimeMillis() - time; long processTime = System.currentTimeMillis() - time;
if(!Objects.isNull(result) && result instanceof String && !isMute(event.getFromGroup())) { if(!Objects.isNull(result) && result instanceof String) {
try { try {
int sendResult = event.sendMessage((String) result); int sendResult = event.sendMessage((String) result);
if(sendResult < 0) { if (sendResult < 0) {
log.warn("消息发送失败, Sender {} 返回错误代码: {}", event.getClass().getName(), sendResult); log.warn("消息发送失败, Sender {} 返回错误代码: {}", event.getClass().getName(), sendResult);
} }
} catch(InterruptedException e) {
log.info("事件在发送消息时超时, 重新投递该事件.(Event: {})", event);
EventExecutor.resendCurrentEvent();
} catch (Exception e) { } catch (Exception e) {
log.error("发送消息时发生异常", e); log.error("发送消息时发生异常", e);
} }
} else if(isMute(event.getFromGroup())) {
log.warn("命令反馈时机器人已被禁言, 跳过反馈.");
} }
long totalTime = System.currentTimeMillis() - time; long totalTime = System.currentTimeMillis() - time;
log.info("命令反馈完成.(事件耗时: {}ms, P: {}%({}ms), R: {}%({}ms))", totalTime, log.info("命令反馈完成.(事件耗时: {}ms, P: {}%({}ms), R: {}%({}ms))", totalTime,
@ -260,40 +258,4 @@ public class BotEventHandler implements EventHandler {
return !message.startsWith(COMMAND_PREFIX) && !message.startsWith(ADMIN_COMMAND_PREFIX); return !message.startsWith(COMMAND_PREFIX) && !message.startsWith(ADMIN_COMMAND_PREFIX);
} }
private static boolean isMute(long groupId) {
Boolean mute = isMute(groupId, false);
return mute != null && mute;
}
/**
* 查询某群是否被禁言.
* @param groupId 群组Id
* @param rawValue 是否返回原始值(当没有该群状态, 且本参数为true时, 将返回null)
* @return 返回状态值, 如无该群禁言记录且rawValue = true, 则返回null
*/
public static Boolean isMute(long groupId, boolean rawValue) {
if(groupId <= 0) {
return false;
}
AtomicBoolean state = muteStateMap.get(groupId);
if(state == null && rawValue) {
return null;
}
return state != null && state.get();
}
/**
* 设置机器人禁言状态.
* <p>设置该项可防止因机器人在禁言期间反馈请求导致被封号.</p>
* @param mute 如果被禁言, 传入true
*/
public static void setMuteState(long groupId, boolean mute) {
if(!muteStateMap.containsKey(groupId)) {
muteStateMap.put(groupId, new AtomicBoolean(mute));
} else {
muteStateMap.get(groupId).set(mute);
}
log.warn("群组 {} 机器人禁言状态已变更: {}", groupId, mute ? "已禁言" : "已解除");
}
} }

View File

@ -1,70 +0,0 @@
package net.lamgc.cgj.bot.event;
import java.util.Objects;
public class BufferMessageEvent extends MessageEvent {
private final StringBuffer buffer = new StringBuffer();
private final MessageEvent parent;
/**
* 以空消息空Id生成BufferMessageEvent
*/
public BufferMessageEvent() {
super(0, 0, "");
parent = null;
}
/**
* 提供消息内容构造BufferMessageEvent
* @param message 传入的消息内容
*/
public BufferMessageEvent(String message) {
super(0, 0, message);
parent = null;
}
/**
* 提供消息内容构和Id信息造BufferMessageEvent
* @param groupId 群组Id
* @param qqId 发送者Id
* @param message 传入的消息内容
*/
public BufferMessageEvent(int groupId, int qqId, String message) {
super(groupId, qqId, message);
parent = null;
}
/**
* 使用事件构造BufferMessageEvent
* @param parentEvent 父级消息事件对象
*/
public BufferMessageEvent(MessageEvent parentEvent) {
super(parentEvent.getFromGroup(), parentEvent.getFromQQ(), parentEvent.getMessage());
parent = parentEvent;
}
@Override
public int sendMessage(String message) {
buffer.append(message);
return 0;
}
/**
* 当提供了父级消息事件时, 本方法调用父级消息事件对象的{@code getImageUrl(String)}, 如果没有, 返回{@code null}
*/
@Override
public String getImageUrl(String image) {
return Objects.isNull(this.parent) ? null : this.parent.getImageUrl(image);
}
/**
* 获取缓冲区消息内容
* @return 消息内容
*/
public String getBufferMessage() {
return buffer.toString();
}
}

View File

@ -0,0 +1,23 @@
package net.lamgc.cgj.bot.event;
import net.lamgc.cgj.bot.message.MessageSender;
public class BufferedMessageSender implements MessageSender {
private final StringBuffer buffer = new StringBuffer();
@Override
public int sendMessage(String message) {
buffer.append(message);
return 0;
}
/**
* 从缓冲区中取出消息内容.
* @return 返回事件发送的消息内容.
*/
public String getBufferContent() {
return buffer.toString();
}
}

View File

@ -57,7 +57,7 @@ public abstract class MessageEvent implements EventObject, MessageSender {
@Override @Override
public String toString() { public String toString() {
return this.getClass().getSimpleName() + "{" + return this.getClass().getSimpleName() + "@" + Integer.toHexString(this.hashCode()) + "{" +
"fromGroup=" + getFromGroup() + "fromGroup=" + getFromGroup() +
", fromQQ=" + getFromQQ() + ", fromQQ=" + getFromQQ() +
", message='" + getMessage() + '\'' + ", message='" + getMessage() + '\'' +

View File

@ -0,0 +1,41 @@
package net.lamgc.cgj.bot.framework;
public interface Framework {
/**
* 框架初始化方法
* @param resources 框架所分配到的资源.
* @throws Exception 当框架抛出异常时, 将不会继续运行框架.
* @see FrameworkResources
*/
void init(FrameworkResources resources) throws Exception;
/**
* 框架运行方法
* @throws Exception 当框架抛出异常时, 将会终止框架的所有活动.
*/
void run() throws Exception;
/**
* 关闭框架
* @throws Exception 即使该方法抛出异常, {@link FrameworkManager}依然会尝试向框架所属的线程发起中断, 以试图清除框架资源.
*/
void close() throws Exception;
/**
* 获取框架标识名.
* <p>可根据需要自行调整框架标识名.</p>
* @return 返回标识名.
*/
default String getIdentify() {
return this.getClass().getSimpleName() + "@" + Integer.toHexString(this.hashCode());
}
/**
* 获取框架名称.
* <p>框架名称不可更改.</p>
* @return 返回框架名称.
*/
String getFrameworkName();
}

View File

@ -0,0 +1,75 @@
package net.lamgc.cgj.bot.framework;
import org.slf4j.Logger;
import java.util.*;
import java.util.regex.Pattern;
public final class FrameworkManager {
private FrameworkManager() {}
private final static Map<Framework, FrameworkResources> resourcesMap = new HashMap<>();
private final static ThreadGroup frameworkRootGroup = new ThreadGroup("FrameworkRootGroup");
static {
Runtime.getRuntime()
.addShutdownHook(new Thread(FrameworkManager::shutdownAllFramework, "FrameworkManager-Shutdown"));
}
public static Thread registerFramework(Framework framework) {
checkFramework(framework);
FrameworkResources resources = new FrameworkResources(framework);
resourcesMap.put(framework, resources);
Thread frameworkThread = new Thread(resources.getFrameworkThreadGroup(),
() -> FrameworkManager.runFramework(framework), "FrameworkThread-" + framework.getIdentify());
frameworkThread.start();
return frameworkThread;
}
private static final Pattern FRAMEWORK_NAME_CHECK_PATTERN = Pattern.compile("^[A-Za-z0-9_\\-$]+$");
private static void checkFramework(Framework framework) {
if(!FRAMEWORK_NAME_CHECK_PATTERN.matcher(framework.getFrameworkName()).matches()) {
throw new IllegalStateException("Invalid Framework Name: " + framework.getFrameworkName());
}
}
public static Set<Framework> frameworkSet() {
return new HashSet<>(resourcesMap.keySet());
}
public static void shutdownAllFramework() {
for (Framework framework : resourcesMap.keySet()) {
FrameworkResources frameworkResources = resourcesMap.get(framework);
Logger frameworkLogger = frameworkResources.getLogger();
try {
frameworkLogger.info("正在关闭框架...");
framework.close();
frameworkLogger.info("框架已关闭.");
frameworkResources.getFrameworkThreadGroup().interrupt();
resourcesMap.remove(framework);
} catch(Throwable e) {
frameworkLogger.error("退出框架时发生异常", e);
}
}
}
static ThreadGroup getFrameworkRootGroup() {
return frameworkRootGroup;
}
private static void runFramework(Framework framework) {
FrameworkResources frameworkResources = resourcesMap.get(framework);
try {
framework.init(frameworkResources);
framework.run();
} catch(Throwable e) {
frameworkResources.getLogger().error("框架未捕获异常, 导致异常退出.", e);
} finally {
frameworkResources.getFrameworkThreadGroup().interrupt();
}
}
}

View File

@ -0,0 +1,35 @@
package net.lamgc.cgj.bot.framework;
import net.lamgc.cgj.bot.boot.BotGlobal;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
public class FrameworkResources {
private final static File frameworkDataStoreRootDir = new File(BotGlobal.getGlobal().getDataStoreDir(),
"frameworks/");
private final ThreadGroup frameworkThreadGroup;
private final Logger logger;
private final File frameworkDataStoreDir;
public FrameworkResources(Framework framework) {
frameworkThreadGroup = new ThreadGroup(FrameworkManager.getFrameworkRootGroup(),
"Framework-" + framework.getIdentify());
frameworkDataStoreDir = new File(frameworkDataStoreRootDir, framework.getClass().getSimpleName());
logger = LoggerFactory.getLogger("Framework-" + framework.getIdentify());
}
ThreadGroup getFrameworkThreadGroup() {
return frameworkThreadGroup;
}
public Logger getLogger() {
return logger;
}
}

View File

@ -2,6 +2,9 @@ package net.lamgc.cgj.bot.framework.cli;
import net.lamgc.cgj.bot.boot.ApplicationBoot; import net.lamgc.cgj.bot.boot.ApplicationBoot;
import net.lamgc.cgj.bot.event.BotEventHandler; import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.framework.Framework;
import net.lamgc.cgj.bot.framework.FrameworkManager;
import net.lamgc.cgj.bot.framework.FrameworkResources;
import net.lamgc.cgj.bot.framework.cli.message.ConsoleMessageEvent; import net.lamgc.cgj.bot.framework.cli.message.ConsoleMessageEvent;
import net.lamgc.cgj.bot.framework.cli.message.ConsoleMessageSenderFactory; import net.lamgc.cgj.bot.framework.cli.message.ConsoleMessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSenderBuilder; import net.lamgc.cgj.bot.message.MessageSenderBuilder;
@ -12,13 +15,18 @@ import org.jline.terminal.TerminalBuilder;
import org.slf4j.Logger; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
import java.io.IOException; import java.util.concurrent.atomic.AtomicBoolean;
public class ConsoleMain { public class ConsoleMain implements Framework {
private final static Logger log = LoggerFactory.getLogger(ConsoleMain.class); private final static Logger log = LoggerFactory.getLogger(ConsoleMain.class);
private final AtomicBoolean quitState = new AtomicBoolean();
public static void start() throws IOException { @Override
public void init(FrameworkResources resources) { }
@Override
public void run() throws Exception {
MessageSenderBuilder.setCurrentMessageSenderFactory(new ConsoleMessageSenderFactory()); MessageSenderBuilder.setCurrentMessageSenderFactory(new ConsoleMessageSenderFactory());
ApplicationBoot.initialBot(); ApplicationBoot.initialBot();
LineReader lineReader = LineReaderBuilder.builder() LineReader lineReader = LineReaderBuilder.builder()
@ -31,7 +39,7 @@ public class ConsoleMain {
long groupId = Long.parseLong(lineReader.readLine("会话群组号:")); long groupId = Long.parseLong(lineReader.readLine("会话群组号:"));
boolean isGroup = false; boolean isGroup = false;
do { do {
String input = lineReader.readLine("App " + qqId + (isGroup ? "@" + groupId : "$private") + " >"); String input = lineReader.readLine("App " + qqId + (isGroup ? "@" + groupId : "#private") + " >");
if(input.equalsIgnoreCase("#exit")) { if(input.equalsIgnoreCase("#exit")) {
System.out.println("退出应用..."); System.out.println("退出应用...");
break; break;
@ -45,7 +53,22 @@ public class ConsoleMain {
} catch (InterruptedException e) { } catch (InterruptedException e) {
log.error("执行时发生中断", e); log.error("执行时发生中断", e);
} }
} while(true); } while(!quitState.get());
} }
@Override
public void close() {
quitState.set(true);
Thread.currentThread().getThreadGroup().interrupt();
}
@Override
public String getIdentify() {
return this.toString();
}
@Override
public String getFrameworkName() {
return "console";
}
} }

View File

@ -1,8 +1,8 @@
package net.lamgc.cgj.bot.framework.cli.message; package net.lamgc.cgj.bot.framework.cli.message;
import net.lamgc.cgj.bot.event.MessageEvent; import net.lamgc.cgj.bot.event.MessageEvent;
import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import java.util.Date; import net.lamgc.cgj.bot.message.MessageSource;
public class ConsoleMessageEvent extends MessageEvent { public class ConsoleMessageEvent extends MessageEvent {
@ -11,9 +11,15 @@ public class ConsoleMessageEvent extends MessageEvent {
} }
@Override @Override
public int sendMessage(String message) { public int sendMessage(String message) throws Exception {
System.out.println(new Date() + " Bot: " + message); if(getFromGroup() <= 0) {
return 0; return MessageSenderBuilder
.getMessageSender(MessageSource.PRIVATE, getFromQQ()).sendMessage(message);
} else {
return MessageSenderBuilder
.getMessageSender(MessageSource.GROUP, getFromQQ()).sendMessage(message);
}
} }
@Override @Override

View File

@ -1,13 +1,26 @@
package net.lamgc.cgj.bot.framework.cli.message; package net.lamgc.cgj.bot.framework.cli.message;
import net.lamgc.cgj.bot.message.MessageSender; import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import java.text.SimpleDateFormat;
import java.util.Date; import java.util.Date;
public class ConsoleMessageSender implements MessageSender { public class ConsoleMessageSender implements MessageSender {
private final static SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS");
private final MessageSource source;
private final long id;
ConsoleMessageSender(MessageSource source, long id) {
this.source = source;
this.id = id;
}
@Override @Override
public synchronized int sendMessage(String message) { public synchronized int sendMessage(String message) {
System.out.println(new Date() + " Bot: " + message); System.out.println(dateFormat.format(new Date()) + " Bot -> " +
(source == MessageSource.PRIVATE ? "#" : "@") + id + ": " + message);
return 0; return 0;
} }
} }

View File

@ -6,10 +6,8 @@ import net.lamgc.cgj.bot.message.MessageSource;
public class ConsoleMessageSenderFactory implements MessageSenderFactory { public class ConsoleMessageSenderFactory implements MessageSenderFactory {
private final static ConsoleMessageSender sender = new ConsoleMessageSender();
@Override @Override
public MessageSender createMessageSender(MessageSource source, long id) { public MessageSender createMessageSender(MessageSource source, long id) {
return sender; return new ConsoleMessageSender(source, id);
} }
} }

View File

@ -1,27 +0,0 @@
package net.lamgc.cgj.bot.framework.coolq;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import net.lz1998.cq.CQGlobal;
import net.lz1998.cq.EnableCQ;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
@EnableCQ
public class CQConfig {
public static void init() {
CQGlobal.pluginList.add(CQPluginMain.class);
CQGlobal.executor = new ThreadPoolExecutor(
(int) Math.ceil(Runtime.getRuntime().availableProcessors() / 2F),
Runtime.getRuntime().availableProcessors(),
25, TimeUnit.SECONDS,
new LinkedBlockingQueue<>(512),
new ThreadFactoryBuilder()
.setNameFormat("Plugin-ProcessThread-%d")
.build()
);
}
}

View File

@ -3,6 +3,7 @@ package net.lamgc.cgj.bot.framework.coolq;
import net.lamgc.cgj.bot.boot.ApplicationBoot; import net.lamgc.cgj.bot.boot.ApplicationBoot;
import net.lamgc.cgj.bot.event.BotEventHandler; import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.framework.coolq.message.SpringCQMessageEvent; import net.lamgc.cgj.bot.framework.coolq.message.SpringCQMessageEvent;
import net.lamgc.cgj.bot.framework.coolq.message.SpringCQMessageSenderFactory;
import net.lamgc.utils.event.EventHandler; import net.lamgc.utils.event.EventHandler;
import net.lz1998.cq.event.message.CQDiscussMessageEvent; import net.lz1998.cq.event.message.CQDiscussMessageEvent;
import net.lz1998.cq.event.message.CQGroupMessageEvent; import net.lz1998.cq.event.message.CQGroupMessageEvent;
@ -13,13 +14,15 @@ import net.lz1998.cq.robot.CoolQ;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component; import org.springframework.stereotype.Component;
import java.util.concurrent.atomic.AtomicBoolean;
@Component @Component
@SuppressWarnings("unused")
public class CQPluginMain extends CQPlugin implements EventHandler { public class CQPluginMain extends CQPlugin implements EventHandler {
private final static AtomicBoolean initialState = new AtomicBoolean();
public CQPluginMain() { public CQPluginMain() {
// TODO(LamGC, 2020.04.21): SpringCQ无法适配MessageSenderBuilder
// MessageSenderBuilder.setCurrentMessageSenderFactory(new SpringCQMessageSenderFactory());
ApplicationBoot.initialBot();
LoggerFactory.getLogger(CQPluginMain.class) LoggerFactory.getLogger(CQPluginMain.class)
.info("BotEventHandler.COMMAND_PREFIX = {}", BotEventHandler.COMMAND_PREFIX); .info("BotEventHandler.COMMAND_PREFIX = {}", BotEventHandler.COMMAND_PREFIX);
} }
@ -46,6 +49,13 @@ public class CQPluginMain extends CQPlugin implements EventHandler {
* @return 是否拦截消息 * @return 是否拦截消息
*/ */
private static int processMessage(CoolQ cq, CQMessageEvent event) { private static int processMessage(CoolQ cq, CQMessageEvent event) {
SpringCQMessageSenderFactory.setCoolQ(cq);
synchronized (initialState) {
if(!initialState.get()) {
ApplicationBoot.initialBot();
initialState.set(true);
}
}
if(BotEventHandler.mismatch(event.getMessage())) { if(BotEventHandler.mismatch(event.getMessage())) {
return MESSAGE_IGNORE; return MESSAGE_IGNORE;
} }

View File

@ -0,0 +1,64 @@
package net.lamgc.cgj.bot.framework.coolq;
import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.framework.Framework;
import net.lamgc.cgj.bot.framework.FrameworkResources;
import org.slf4j.Logger;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.context.event.ApplicationFailedEvent;
import org.springframework.context.ApplicationListener;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.context.event.ContextClosedEvent;
import org.springframework.context.event.ContextStoppedEvent;
@SpringBootApplication
public class SpringCQApplication implements Framework {
private Logger log;
private final Object quitLock = new Object();
@Override
public void init(FrameworkResources resources) {
this.log = resources.getLogger();
}
public void run() {
log.info("酷Q机器人根目录: {}", BotGlobal.getGlobal().getDataStoreDir().getPath());
ConfigurableApplicationContext context = SpringApplication.run(SpringCQApplication.class);
registerShutdownHook(context);
try {
synchronized (quitLock) {
quitLock.wait();
}
} catch (InterruptedException e) {
log.warn("发生中断, 退出SpringCQ...", e);
}
context.stop();
context.close();
}
private void registerShutdownHook(ConfigurableApplicationContext context) {
context.addApplicationListener((ApplicationListener<ApplicationFailedEvent>)
event -> close());
context.addApplicationListener((ApplicationListener<ContextClosedEvent>)
event -> close());
context.addApplicationListener((ApplicationListener<ContextStoppedEvent>)
event -> close());
Runtime.getRuntime().addShutdownHook(new Thread(this::close));
}
public void close() {
synchronized (quitLock) {
quitLock.notify();
}
}
@Override
public String getFrameworkName() {
return "SpringCoolQ";
}
}

View File

@ -26,13 +26,13 @@ public class SpringCQMessageEvent extends MessageEvent {
this.cq = Objects.requireNonNull(cq); this.cq = Objects.requireNonNull(cq);
MessageSource source; MessageSource source;
if(messageEvent instanceof CQGroupMessageEvent) { if(messageEvent instanceof CQGroupMessageEvent) {
source = MessageSource.Group; source = MessageSource.GROUP;
} else if (messageEvent instanceof CQDiscussMessageEvent) { } else if (messageEvent instanceof CQDiscussMessageEvent) {
source = MessageSource.Discuss; source = MessageSource.DISCUSS;
} else { } else {
source = MessageSource.Private; source = MessageSource.PRIVATE;
} }
messageSender = new SpringCQMessageSender(cq, source, source == MessageSource.Private ? getFromQQ() : getFromGroup()); messageSender = new SpringCQMessageSender(cq, source, source == MessageSource.PRIVATE ? getFromQQ() : getFromGroup());
} }
@Override @Override

View File

@ -6,9 +6,9 @@ import net.lz1998.cq.robot.CoolQ;
public class SpringCQMessageSender implements MessageSender { public class SpringCQMessageSender implements MessageSender {
private CoolQ coolQ; private final CoolQ coolQ;
private MessageSource source; private final MessageSource source;
private long target; private final long target;
public SpringCQMessageSender(CoolQ coolQ, MessageSource source, long target) { public SpringCQMessageSender(CoolQ coolQ, MessageSource source, long target) {
this.coolQ = coolQ; this.coolQ = coolQ;
@ -19,11 +19,11 @@ public class SpringCQMessageSender implements MessageSender {
@Override @Override
public int sendMessage(String message) { public int sendMessage(String message) {
switch (source) { switch (source) {
case Private: case PRIVATE:
return coolQ.sendPrivateMsg(target, message, false).getData().getMessageId(); return coolQ.sendPrivateMsg(target, message, false).getData().getMessageId();
case Group: case GROUP:
return coolQ.sendGroupMsg(target, message, false).getData().getMessageId(); return coolQ.sendGroupMsg(target, message, false).getData().getMessageId();
case Discuss: case DISCUSS:
return coolQ.sendDiscussMsg(target, message, false).getData().getMessageId(); return coolQ.sendDiscussMsg(target, message, false).getData().getMessageId();
default: default:
return -1; return -1;

View File

@ -6,13 +6,26 @@ import net.lamgc.cgj.bot.message.MessageSource;
import net.lz1998.cq.robot.CoolQ; import net.lz1998.cq.robot.CoolQ;
import java.util.Objects; import java.util.Objects;
import java.util.concurrent.atomic.AtomicReference;
public class SpringCQMessageSenderFactory implements MessageSenderFactory { public class SpringCQMessageSenderFactory implements MessageSenderFactory {
private final static ThreadLocal<CoolQ> threadCoolQ = new ThreadLocal<>(); private final static AtomicReference<CoolQ> coolQ = new AtomicReference<>();
/**
* 设置CoolQ对象.
* <p>该方法仅接受第一次设置的CoolQ对象, 其他对象将会忽略.</p>
* @param coolQObj CoolQ对象
*/
public static void setCoolQ(CoolQ coolQObj) {
if(coolQ.get() == null) {
coolQ.set(coolQObj);
}
}
@Override @Override
public MessageSender createMessageSender(MessageSource source, long id) { public MessageSender createMessageSender(MessageSource source, long id) {
return new SpringCQMessageSender( return new SpringCQMessageSender(
Objects.requireNonNull(threadCoolQ.get(), "CoolQ object is not included in ThreadLocal"), source, id); Objects.requireNonNull(coolQ.get(), "CoolQ object not ready"), source, id);
} }
} }

View File

@ -3,9 +3,12 @@ package net.lamgc.cgj.bot.framework.mirai;
import net.lamgc.cgj.bot.boot.ApplicationBoot; import net.lamgc.cgj.bot.boot.ApplicationBoot;
import net.lamgc.cgj.bot.boot.BotGlobal; import net.lamgc.cgj.bot.boot.BotGlobal;
import net.lamgc.cgj.bot.event.BotEventHandler; import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.framework.Framework;
import net.lamgc.cgj.bot.framework.FrameworkResources;
import net.lamgc.cgj.bot.framework.mirai.message.MiraiMessageEvent; import net.lamgc.cgj.bot.framework.mirai.message.MiraiMessageEvent;
import net.lamgc.cgj.bot.framework.mirai.message.MiraiMessageSenderFactory; import net.lamgc.cgj.bot.framework.mirai.message.MiraiMessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSenderBuilder; import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import net.lamgc.cgj.bot.util.GroupMuteManager;
import net.mamoe.mirai.Bot; import net.mamoe.mirai.Bot;
import net.mamoe.mirai.BotFactoryJvm; import net.mamoe.mirai.BotFactoryJvm;
import net.mamoe.mirai.event.events.BotMuteEvent; import net.mamoe.mirai.event.events.BotMuteEvent;
@ -24,15 +27,18 @@ import java.io.*;
import java.util.Base64; import java.util.Base64;
import java.util.Properties; import java.util.Properties;
public class MiraiMain implements Closeable { public class MiraiMain implements Framework {
private final Logger log = LoggerFactory.getLogger(MiraiMain.class); private final Logger log = LoggerFactory.getLogger(MiraiMain.class);
private Bot bot; private Bot bot;
private final static Properties botProperties = new Properties(); private final Properties botProperties = new Properties();
public void init() { private final GroupMuteManager muteManager = new GroupMuteManager();
@Override
public void init(FrameworkResources resources) {
Runtime.getRuntime().addShutdownHook(new Thread(this::close)); Runtime.getRuntime().addShutdownHook(new Thread(this::close));
try { try {
Class.forName(BotEventHandler.class.getName()); Class.forName(BotEventHandler.class.getName());
@ -51,32 +57,58 @@ public class MiraiMain implements Closeable {
Utils.setDefaultLogger(MiraiToSlf4jLoggerAdapter::new); Utils.setDefaultLogger(MiraiToSlf4jLoggerAdapter::new);
BotConfiguration configuration = new BotConfiguration(); BotConfiguration configuration = new BotConfiguration();
configuration.randomDeviceInfo();
configuration.setProtocol(BotConfiguration.MiraiProtocol.ANDROID_PAD); configuration.setProtocol(BotConfiguration.MiraiProtocol.ANDROID_PAD);
// 心跳包周期间隔 (ms)
configuration.setHeartbeatPeriodMillis(
Long.parseLong(botProperties.getProperty("network.heartbeatPeriodMillis", "60000")));
// 心跳包超时时间 (ms)
configuration.setHeartbeatTimeoutMillis(
Long.parseLong(botProperties.getProperty("network.heartbeatTimeoutMillis", "5000")));
// 重连间隔时间
configuration.setReconnectPeriodMillis(
Integer.parseInt(botProperties.getProperty("network.reconnectPeriodMillis", "5")));
// 重连最大次数
configuration.setReconnectionRetryTimes(
Integer.parseInt(botProperties.getProperty("network.reconnectionRetryTimes", "10")));
bot = BotFactoryJvm.newBot(Long.parseLong(botProperties.getProperty("bot.qq", "0")), bot = BotFactoryJvm.newBot(Long.parseLong(botProperties.getProperty("bot.qq", "0")),
Base64.getDecoder().decode(botProperties.getProperty("bot.password", "")), configuration); Base64.getDecoder().decode(botProperties.getProperty("bot.password", "")), configuration);
// TODO: 看看能不能单独订阅某个Bot?
Events.subscribeAlways(GroupMessageEvent.class, this::executeMessageEvent); Events.subscribeAlways(GroupMessageEvent.class, this::executeMessageEvent);
Events.subscribeAlways(FriendMessageEvent.class, this::executeMessageEvent); Events.subscribeAlways(FriendMessageEvent.class, this::executeMessageEvent);
Events.subscribeAlways(TempMessageEvent.class, this::executeMessageEvent); Events.subscribeAlways(TempMessageEvent.class, this::executeMessageEvent);
Events.subscribeAlways(BotMuteEvent.class, Events.subscribeAlways(BotMuteEvent.class,
event -> BotEventHandler.setMuteState(event.getGroup().getId(), true)); event -> muteManager.setMuteState(event.getGroup().getId(), true));
Events.subscribeAlways(BotUnmuteEvent.class, Events.subscribeAlways(BotUnmuteEvent.class,
event -> BotEventHandler.setMuteState(event.getGroup().getId(), false)); event -> muteManager.setMuteState(event.getGroup().getId(), false));
bot.login(); bot.login();
MessageSenderBuilder.setCurrentMessageSenderFactory(new MiraiMessageSenderFactory(bot)); MessageSenderBuilder.setCurrentMessageSenderFactory(new MiraiMessageSenderFactory(bot));
ApplicationBoot.initialBot(); ApplicationBoot.initialBot();
bot.join(); bot.join();
} }
@Override
public void run() {
bot.login();
bot.join();
}
/** /**
* 处理消息事件 * 处理消息事件
* @param message 消息事件对象 * @param message 消息事件对象
*/ */
private void executeMessageEvent(MessageEvent message) { private void executeMessageEvent(MessageEvent message) {
log.debug("Mirai Message: {}", message);
if(message instanceof GroupMessageEvent) { if(message instanceof GroupMessageEvent) {
GroupMessageEvent GroupMessageEvent = (GroupMessageEvent) message; GroupMessageEvent GroupMessageEvent = (GroupMessageEvent) message;
if(BotEventHandler.isMute(GroupMessageEvent.getGroup().getId(), true) == null) { Boolean muteState = muteManager.isMute(GroupMessageEvent.getGroup().getId(), true);
BotEventHandler.setMuteState(GroupMessageEvent.getGroup().getId(), if(muteState == null) {
muteManager.setMuteState(GroupMessageEvent.getGroup().getId(),
((GroupMessageEvent) message).getGroup().getBotMuteRemaining() != 0); ((GroupMessageEvent) message).getGroup().getBotMuteRemaining() != 0);
} else if(muteState) {
return;
} }
} }
BotEventHandler.executeMessageEvent(MiraiMessageEvent.covertEventObject(message)); BotEventHandler.executeMessageEvent(MiraiMessageEvent.covertEventObject(message));
@ -85,6 +117,7 @@ public class MiraiMain implements Closeable {
/** /**
* 关闭机器人 * 关闭机器人
*/ */
@Override
public synchronized void close() { public synchronized void close() {
if(bot == null) { if(bot == null) {
return; return;
@ -95,4 +128,9 @@ public class MiraiMain implements Closeable {
log.warn("机器人已关闭."); log.warn("机器人已关闭.");
} }
@Override
public String getFrameworkName() {
return "MiraiQQ";
}
} }

View File

@ -28,9 +28,9 @@ public class MiraiMessageEvent extends net.lamgc.cgj.bot.event.MessageEvent {
message.getSender().getId(), getMessageBodyWithoutSource(message.getMessage().toString())); message.getSender().getId(), getMessageBodyWithoutSource(message.getMessage().toString()));
this.messageObject = Objects.requireNonNull(message); this.messageObject = Objects.requireNonNull(message);
if(message instanceof GroupMessageEvent) { if(message instanceof GroupMessageEvent) {
messageSender = new MiraiMessageSender(((GroupMessageEvent) message).getGroup(), MessageSource.Group); messageSender = new MiraiMessageSender(((GroupMessageEvent) message).getGroup(), MessageSource.GROUP);
} else { } else {
messageSender = new MiraiMessageSender(message.getSender(), MessageSource.Private); messageSender = new MiraiMessageSender(message.getSender(), MessageSource.PRIVATE);
} }
} }
@ -45,9 +45,9 @@ public class MiraiMessageEvent extends net.lamgc.cgj.bot.event.MessageEvent {
super(groupId, qqId, getMessageBodyWithoutSource(message.toString())); super(groupId, qqId, getMessageBodyWithoutSource(message.toString()));
this.messageObject = Objects.requireNonNull(messageObject, "messageObject is null"); this.messageObject = Objects.requireNonNull(messageObject, "messageObject is null");
if(groupId != 0) { if(groupId != 0) {
this.messageSender = new MiraiMessageSender(((GroupMessageEvent) messageObject).getGroup(), MessageSource.Group); this.messageSender = new MiraiMessageSender(((GroupMessageEvent) messageObject).getGroup(), MessageSource.GROUP);
} else { } else {
this.messageSender = new MiraiMessageSender(messageObject.getSender(), MessageSource.Group); this.messageSender = new MiraiMessageSender(messageObject.getSender(), MessageSource.GROUP);
} }
} }

View File

@ -40,7 +40,7 @@ public class MiraiMessageSender implements MessageSender {
* @throws NoSuchElementException 当在机器人好友列表或群列表里没有这个好友或群的时候抛出 * @throws NoSuchElementException 当在机器人好友列表或群列表里没有这个好友或群的时候抛出
*/ */
public MiraiMessageSender(Bot bot, MessageSource source, long id) { public MiraiMessageSender(Bot bot, MessageSource source, long id) {
this(source == MessageSource.Private ? bot.getFriend(id) : bot.getGroup(id), source); this(source == MessageSource.PRIVATE ? bot.getFriend(id) : bot.getGroup(id), source);
} }
/** /**
@ -151,7 +151,7 @@ public class MiraiMessageSender implements MessageSender {
synchronized (imageName) { synchronized (imageName) {
if(!imageIdCache.exists(imageName) || if(!imageIdCache.exists(imageName) ||
Strings.nullToEmpty(code.getParameter("updateCache")) .equalsIgnoreCase("true")) { Strings.nullToEmpty(code.getParameter("updateCache")) .equalsIgnoreCase("true")) {
log.debug("imageName [{}] 缓存失效或强制更新, 正在更新缓存...", imageName); log.trace("imageName [{}] 缓存失效或强制更新, 正在更新缓存...", imageName);
image = uploadImage0(new File(absolutePath)); image = uploadImage0(new File(absolutePath));
String cacheExpireAt; String cacheExpireAt;
long expireTime = 864000000; // 10d long expireTime = 864000000; // 10d
@ -163,13 +163,13 @@ public class MiraiMessageSender implements MessageSender {
} }
} }
imageIdCache.update(imageName, image.getImageId(), expireTime); imageIdCache.update(imageName, image.getImageId(), expireTime);
log.debug("imageName [{}] 缓存更新完成.(有效时间: {})", imageName, expireTime); log.trace("imageName [{}] 缓存更新完成.(有效时间: {})", imageName, expireTime);
} else { } else {
log.debug("ImageName: [{}] 缓存命中.", imageName); log.trace("ImageName: [{}] 缓存命中.", imageName);
} }
} }
} else { } else {
log.debug("ImageName: [{}] 缓存命中.", imageName); log.trace("ImageName: [{}] 缓存命中.", imageName);
} }
if(image == null) { if(image == null) {

View File

@ -7,17 +7,17 @@ public enum MessageSource {
/** /**
* 私聊消息 * 私聊消息
*/ */
Private, PRIVATE,
/** /**
* 群组消息 * 群组消息
*/ */
Group, GROUP,
/** /**
* 讨论组消息 * 讨论组消息
*/ */
Discuss, DISCUSS,
/** /**
* 未知来源 * 未知来源
*/ */
Unknown UNKNOWN
} }

View File

@ -0,0 +1,41 @@
package net.lamgc.cgj.bot.sort;
@SuppressWarnings("unused")
public enum PreLoadDataAttribute {
/**
* 按点赞数排序
*/
LIKE("likeCount"),
/**
* 按页面数排序
*/
PAGE("pageCount"),
/**
* 按收藏数排序
*/
BOOKMARK("bookmarkCount"),
/**
* 按评论数排序
*/
COMMENT("commentCount"),
/**
* 不明
*/
RESPONSE("responseCount"),
/**
* 按查看次数排序
*/
VIEW("viewCount"),
;
public final String attrName;
PreLoadDataAttribute(String attrName) {
this.attrName = attrName;
}
}

View File

@ -3,6 +3,8 @@ package net.lamgc.cgj.bot.sort;
import com.google.gson.JsonElement; import com.google.gson.JsonElement;
import com.google.gson.JsonObject; import com.google.gson.JsonObject;
import net.lamgc.cgj.bot.cache.CacheStoreCentral; import net.lamgc.cgj.bot.cache.CacheStoreCentral;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException; import java.io.IOException;
import java.util.Comparator; import java.util.Comparator;
@ -10,11 +12,13 @@ import java.util.Comparator;
/** /**
* 收藏数比较器 * 收藏数比较器
*/ */
public class PreLoadDataComparator implements Comparator<JsonElement> { public class PreLoadDataAttributeComparator implements Comparator<JsonElement> {
private final Attribute attribute; private final static Logger log = LoggerFactory.getLogger(PreLoadDataAttributeComparator.class);
public PreLoadDataComparator(Attribute attribute) { private final PreLoadDataAttribute attribute;
public PreLoadDataAttributeComparator(PreLoadDataAttribute attribute) {
this.attribute = attribute; this.attribute = attribute;
} }
@ -39,57 +43,17 @@ public class PreLoadDataComparator implements Comparator<JsonElement> {
} }
} }
try { try {
JsonObject illustPreLoadData1 = JsonObject illustPreLoadData1 = CacheStoreCentral.getCentral()
CacheStoreCentral.getIllustPreLoadData(o1.getAsJsonObject().get("illustId").getAsInt(), false); .getIllustPreLoadData(o1.getAsJsonObject().get("illustId").getAsInt(), false);
JsonObject illustPreLoadData2 = JsonObject illustPreLoadData2 = CacheStoreCentral.getCentral()
CacheStoreCentral.getIllustPreLoadData(o2.getAsJsonObject().get("illustId").getAsInt(), false); .getIllustPreLoadData(o2.getAsJsonObject().get("illustId").getAsInt(), false);
return Integer.compare( return Integer.compare(
illustPreLoadData2.get(attribute.attrName).getAsInt(), illustPreLoadData2.get(attribute.attrName).getAsInt(),
illustPreLoadData1.get(attribute.attrName).getAsInt()); illustPreLoadData1.get(attribute.attrName).getAsInt());
} catch (IOException e) { } catch (IOException e) {
e.printStackTrace(); log.error("获取预加载数据失败", e);
return 0; return 0;
} }
} }
public enum Attribute {
/**
* 按点赞数排序
*/
LIKE("likeCount"),
/**
* 按页面数排序
*/
PAGE("pageCount"),
/**
* 按收藏数排序
*/
BOOKMARK("bookmarkCount"),
/**
* 按评论数排序
*/
COMMENT("commentCount"),
/**
* 不明
*/
RESPONSE("responseCount"),
/**
* 按查看次数排序
*/
VIEW("viewCount"),
;
public final String attrName;
Attribute(String attrName) {
this.attrName = attrName;
}
}
} }

View File

@ -0,0 +1,48 @@
package net.lamgc.cgj.bot.util;
import java.util.Hashtable;
import java.util.Map;
import java.util.concurrent.atomic.AtomicBoolean;
/**
* 群禁言管理器.
* <p>该管理器用于存取群组禁言状态.</p>
*/
public class GroupMuteManager {
private final Map<Long, AtomicBoolean> muteStateMap = new Hashtable<>();
/**
* 查询某群是否被禁言.
* @param groupId 群组Id
* @param rawValue 是否返回原始值(当没有该群状态, 且本参数为true时, 将返回null)
* @return 返回状态值, 如无该群禁言记录且rawValue = true, 则返回null
*/
public Boolean isMute(long groupId, boolean rawValue) {
if(groupId <= 0) {
return false;
}
AtomicBoolean state = muteStateMap.get(groupId);
if(state == null && rawValue) {
return null;
}
return state != null && state.get();
}
/**
* 设置机器人禁言状态.
* <p>设置该项可防止因机器人在禁言期间反馈请求导致被封号.</p>
* @param mute 如果被禁言, 传入true
*/
public void setMuteState(long groupId, boolean mute) {
if(groupId <= 0) {
return;
}
if(!muteStateMap.containsKey(groupId)) {
muteStateMap.put(groupId, new AtomicBoolean(mute));
} else {
muteStateMap.get(groupId).set(mute);
}
}
}

View File

@ -1,4 +1,4 @@
package net.lamgc.cgj.bot.cache.exception; package net.lamgc.cgj.exception;
import java.io.IOException; import java.io.IOException;
import java.util.Objects; import java.util.Objects;

View File

@ -6,6 +6,7 @@ import com.google.gson.JsonElement;
import com.google.gson.JsonObject; import com.google.gson.JsonObject;
import com.google.gson.reflect.TypeToken; import com.google.gson.reflect.TypeToken;
import io.netty.handler.codec.http.HttpHeaderNames; import io.netty.handler.codec.http.HttpHeaderNames;
import net.lamgc.cgj.exception.HttpRequestException;
import org.apache.http.Header; import org.apache.http.Header;
import org.apache.http.HttpHost; import org.apache.http.HttpHost;
import org.apache.http.HttpRequest; import org.apache.http.HttpRequest;
@ -31,6 +32,7 @@ import java.util.*;
import java.util.concurrent.atomic.AtomicInteger; import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.BiConsumer; import java.util.function.BiConsumer;
@SuppressWarnings("ALL")
public class PixivDownload { public class PixivDownload {
private final static Logger log = LoggerFactory.getLogger(PixivDownload.class); private final static Logger log = LoggerFactory.getLogger(PixivDownload.class);
@ -115,10 +117,10 @@ public class PixivDownload {
} }
} }
} while(!document.select(".pager-container>.next").isEmpty()); } while(!document.select(".pager-container>.next").isEmpty());
log.debug("获取完成."); log.trace("获取完成.");
AtomicInteger count = new AtomicInteger(1); AtomicInteger count = new AtomicInteger(1);
linkList.forEach(link -> { linkList.forEach(link -> {
log.debug("Next Link [{}]: {}", count.getAndIncrement(), link); log.trace("Next Link [{}]: {}", count.getAndIncrement(), link);
InputStream imageInputStream = null; InputStream imageInputStream = null;
int tryCount = 0; int tryCount = 0;
do { do {
@ -133,9 +135,9 @@ public class PixivDownload {
} while(imageInputStream == null); } while(imageInputStream == null);
try(InputStream imageInput = new BufferedInputStream(imageInputStream, 256 * 1024)) { try(InputStream imageInput = new BufferedInputStream(imageInputStream, 256 * 1024)) {
log.debug("调用回调方法..."); log.trace("调用回调方法...");
fn.accept(link, imageInput); fn.accept(link, imageInput);
log.debug("回调方法调用完成."); log.trace("回调方法调用完成.");
} catch (IOException e) { } catch (IOException e) {
log.error("图片获取失败", e); log.error("图片获取失败", e);
} }
@ -219,8 +221,8 @@ public class PixivDownload {
int authorId = rankInfo.get("user_id").getAsInt(); int authorId = rankInfo.get("user_id").getAsInt();
String authorName = rankInfo.get("user_name").getAsString(); String authorName = rankInfo.get("user_name").getAsString();
String title = rankInfo.get("title").getAsString(); String title = rankInfo.get("title").getAsString();
log.debug("当前到第 {}/{} 名(总共 {} 名), IllustID: {}, Author: ({}) {}, Title: {}", rank, rankStart + range - 1, range, illustId, authorId, authorName, title); log.trace("当前到第 {}/{} 名(总共 {} 名), IllustID: {}, Author: ({}) {}, Title: {}", rank, rankStart + range - 1, range, illustId, authorId, authorName, title);
log.debug("正在获取PagesLink..."); log.trace("正在获取PagesLink...");
List<String> linkList; List<String> linkList;
try { try {
linkList = getIllustAllPageDownload(httpClient, this.cookieStore, illustId, quality); linkList = getIllustAllPageDownload(httpClient, this.cookieStore, illustId, quality);
@ -235,14 +237,14 @@ public class PixivDownload {
log.debug("PagesLink 获取完成, 总数: {}", linkList.size()); log.debug("PagesLink 获取完成, 总数: {}", linkList.size());
for (int pageIndex = 0; pageIndex < linkList.size(); pageIndex++) { for (int pageIndex = 0; pageIndex < linkList.size(); pageIndex++) {
String downloadLink = linkList.get(pageIndex); String downloadLink = linkList.get(pageIndex);
log.debug("当前Page: {}/{}", pageIndex + 1, linkList.size()); log.trace("当前Page: {}/{}", pageIndex + 1, linkList.size());
try(InputStream imageInputStream = new BufferedInputStream(getImageAsInputStream(HttpClientBuilder.create().build(), downloadLink), 256 * 1024)) { try(InputStream imageInputStream = new BufferedInputStream(getImageAsInputStream(HttpClientBuilder.create().build(), downloadLink), 256 * 1024)) {
fn.download(rank, downloadLink, rankInfo.deepCopy(), imageInputStream); fn.download(rank, downloadLink, rankInfo.deepCopy(), imageInputStream);
} catch(IOException e) { } catch(IOException e) {
log.error("下载插画时发生异常", e); log.error("下载插画时发生异常", e);
return; return;
} }
log.debug("完成."); log.trace("完成.");
} }
}); });
} }
@ -265,7 +267,7 @@ public class PixivDownload {
int authorId = rankInfo.get("user_id").getAsInt(); int authorId = rankInfo.get("user_id").getAsInt();
String authorName = rankInfo.get("user_name").getAsString(); String authorName = rankInfo.get("user_name").getAsString();
String title = rankInfo.get("title").getAsString(); String title = rankInfo.get("title").getAsString();
log.debug("Array-当前到第 {}/{} 名(总共 {} 名), IllustID: {}, Author: ({}) {}, Title: {}", rank, rankStart + range, range, illustId, authorId, authorName, title); log.trace("Array-当前到第 {}/{} 名(总共 {} 名), IllustID: {}, Author: ({}) {}, Title: {}", rank, rankStart + range, range, illustId, authorId, authorName, title);
results.add(rankInfo); results.add(rankInfo);
} }
log.debug("JsonArray读取完成."); log.debug("JsonArray读取完成.");
@ -320,12 +322,12 @@ public class PixivDownload {
boolean canNext = true; boolean canNext = true;
for (int pageIndex = startPages; canNext && pageIndex <= endPages && count < range; pageIndex++) { for (int pageIndex = startPages; canNext && pageIndex <= endPages && count < range; pageIndex++) {
HttpGet request = createHttpGetRequest(PixivURL.getRankingLink(contentType, mode, time, pageIndex, true)); HttpGet request = createHttpGetRequest(PixivURL.getRankingLink(contentType, mode, time, pageIndex, true));
log.debug("RequestUri: {}", request.getURI()); log.trace("RequestUri: {}", request.getURI());
HttpResponse response = httpClient.execute(request); HttpResponse response = httpClient.execute(request);
String responseBody = EntityUtils.toString(response.getEntity()); String responseBody = EntityUtils.toString(response.getEntity());
log.debug("ResponseBody: {}", responseBody); log.trace("ResponseBody: {}", responseBody);
if(response.getStatusLine().getStatusCode() != 200) { if(response.getStatusLine().getStatusCode() != 200) {
throw new IOException("Http Response Error: '" + response.getStatusLine() + "', ResponseBody: '" + responseBody + '\''); throw new HttpRequestException(response.getStatusLine(), responseBody);
} }
JsonObject resultObject = gson.fromJson(responseBody, JsonObject.class); JsonObject resultObject = gson.fromJson(responseBody, JsonObject.class);
@ -352,7 +354,7 @@ public class PixivDownload {
HttpResponse response = httpClient.execute(request); HttpResponse response = httpClient.execute(request);
if(response.getStatusLine().getStatusCode() != 200) { if(response.getStatusLine().getStatusCode() != 200) {
throw new IOException("Http响应码非200: " + response.getStatusLine()); throw new HttpRequestException(response);
} }
Document document = Jsoup.parse(EntityUtils.toString(response.getEntity())); Document document = Jsoup.parse(EntityUtils.toString(response.getEntity()));
@ -399,8 +401,8 @@ public class PixivDownload {
if(resultObject.get("error").getAsBoolean()) { if(resultObject.get("error").getAsBoolean()) {
String message = resultObject.get("message").getAsString(); String message = resultObject.get("message").getAsString();
log.debug("请求错误, 错误信息: {}", message); log.warn("作品页面接口请求错误, 错误信息: {}", message);
throw new IOException(message); throw new HttpRequestException(response.getStatusLine(), resultObject.toString());
} }
JsonArray linkArray = resultObject.getAsJsonArray("body"); JsonArray linkArray = resultObject.getAsJsonArray("body");
@ -473,9 +475,11 @@ public class PixivDownload {
request.addHeader(HttpHeaderNames.REFERER.toString(), referer); request.addHeader(HttpHeaderNames.REFERER.toString(), referer);
HttpResponse response = httpClient.execute(request); HttpResponse response = httpClient.execute(request);
log.debug("response: {}", response); log.trace("response: {}", response);
log.debug("Content Length: {}KB", Float.parseFloat(response.getFirstHeader(HttpHeaderNames.CONTENT_LENGTH.toString()).getValue()) / 1024F); log.trace("Content Length: {}KB",
log.debug("{}", response.getFirstHeader(HttpHeaderNames.CONTENT_TYPE.toString())); Float.parseFloat(response.getFirstHeader(HttpHeaderNames.CONTENT_LENGTH.toString()).getValue()) / 1024F
);
log.trace(response.getFirstHeader(HttpHeaderNames.CONTENT_TYPE.toString()).toString());
return response.getEntity().getContent(); return response.getEntity().getContent();
} }
@ -496,51 +500,12 @@ public class PixivDownload {
} else { } else {
return false; return false;
} }
} }
/** /**
* 获取作品信息 * 获取作品信息
* @param illustId 作品ID * @param illustId 作品ID
* @return 成功获取返回JsonObject, 失败返回null, <br/> * @return 成功获取返回JsonObject, 失败返回null.
* Json示例: <br/>
* <pre>
* {
* "illustId": "79584670",
* "illustTitle": "このヤンキーはウブすぎる",
* "id": "79584670",
* "title": "このヤンキーはウブすぎる",
* "illustType": 1,
* "xRestrict": 0,
* "restrict": 0,
* "sl": 2,
* "url": "https://i.pximg.net/c/360x360_70/img-master/img/2020/02/19/00/38/23/79584670_p0_square1200.jpg",
* "description": "",
* "tags": [
* "漫画",
* "オリジナル",
* "創作",
* "創作男女",
* "コロさん、ポリさん此方です!",
* "恋の予感",
* "あまずっぺー",
* "交換日記",
* "続編希望!!",
* "オリジナル10000users入り"
* ],
* "userId": "4778293",
* "userName": "隈浪さえ",
* "width": 3288,
* "height": 4564,
* "pageCount": 4,
* "isBookmarkable": true,
* "bookmarkData": null,
* "alt": "#オリジナル このヤンキーはウブすぎる - 隈浪さえ的漫画",
* "isAdContainer": false,
* "profileImageUrl": "https://i.pximg.net/user-profile/img/2019/12/04/18/56/19/16639046_fea29ce38ea89b0cb2313b40b3a72f9a_50.jpg",
* "type": "illust"
* }
* </pre>
* @throws IOException 当请求发生异常, 或接口返回错误信息时抛出. * @throws IOException 当请求发生异常, 或接口返回错误信息时抛出.
* @throws NoSuchElementException 当该作品不存在时抛出异常 * @throws NoSuchElementException 当该作品不存在时抛出异常
*/ */
@ -548,11 +513,11 @@ public class PixivDownload {
HttpGet request = createHttpGetRequest(PixivURL.getPixivIllustInfoAPI(illustId)); HttpGet request = createHttpGetRequest(PixivURL.getPixivIllustInfoAPI(illustId));
HttpResponse response = httpClient.execute(request); HttpResponse response = httpClient.execute(request);
String responseStr = EntityUtils.toString(response.getEntity()); String responseStr = EntityUtils.toString(response.getEntity());
log.debug("Response Content: {}", responseStr); log.trace("Response Content: {}", responseStr);
JsonObject responseObj = new Gson().fromJson(responseStr, JsonObject.class); JsonObject responseObj = new Gson().fromJson(responseStr, JsonObject.class);
if(responseObj.get("error").getAsBoolean()) { if(responseObj.get("error").getAsBoolean()) {
throw new IOException(responseObj.get("message").getAsString()); throw new HttpRequestException(response.getStatusLine(), responseStr);
} }
JsonArray illustsArray = responseObj.getAsJsonObject("body").getAsJsonArray("illusts"); JsonArray illustsArray = responseObj.getAsJsonObject("body").getAsJsonArray("illusts");

View File

@ -15,8 +15,8 @@ import java.util.Objects;
* @author LamGC * @author LamGC
* @see PixivURL#PIXIV_SEARCH_CONTENT_URL * @see PixivURL#PIXIV_SEARCH_CONTENT_URL
*/ */
@SuppressWarnings("ALL") @SuppressWarnings("unused")
public class PixivSearchBuilder { public class PixivSearchLinkBuilder {
private final String content; private final String content;
@ -26,8 +26,8 @@ public class PixivSearchBuilder {
private SearchOrder searchOrder = SearchOrder.DATE_D; private SearchOrder searchOrder = SearchOrder.DATE_D;
private SearchContentOption searchContentOption = SearchContentOption.ALL; private SearchContentOption searchContentOption = SearchContentOption.ALL;
private HashSet<String> includeKeywords = new HashSet<>(0); private final HashSet<String> includeKeywords = new HashSet<>(0);
private HashSet<String> excludeKeywords = new HashSet<>(0); private final HashSet<String> excludeKeywords = new HashSet<>(0);
private int page = 1; private int page = 1;
@ -42,7 +42,7 @@ public class PixivSearchBuilder {
private Date startDate = null; private Date startDate = null;
private Date endDate = null; private Date endDate = null;
public PixivSearchBuilder(String searchContent) { public PixivSearchLinkBuilder(String searchContent) {
this.content = Objects.requireNonNull(searchContent); this.content = Objects.requireNonNull(searchContent);
} }
@ -99,7 +99,7 @@ public class PixivSearchBuilder {
public boolean equals(Object o) { public boolean equals(Object o) {
if (this == o) return true; if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false; if (o == null || getClass() != o.getClass()) return false;
PixivSearchBuilder that = (PixivSearchBuilder) o; PixivSearchLinkBuilder that = (PixivSearchLinkBuilder) o;
return page == that.page && return page == that.page &&
wgt == that.wgt && wgt == that.wgt &&
hgt == that.hgt && hgt == that.hgt &&
@ -141,7 +141,7 @@ public class PixivSearchBuilder {
@Override @Override
public String toString() { public String toString() {
return "PixivSearchBuilder{" + return "PixivSearchLinkBuilder{" +
"content='" + content + '\'' + "content='" + content + '\'' +
", searchArea=" + searchArea + ", searchArea=" + searchArea +
", searchMode=" + searchMode + ", searchMode=" + searchMode +
@ -161,19 +161,11 @@ public class PixivSearchBuilder {
'}'; '}';
} }
public PixivSearchBuilder setSearchArea(SearchArea searchArea) { public PixivSearchLinkBuilder setSearchArea(SearchArea searchArea) {
this.searchArea = Objects.requireNonNull(searchArea); this.searchArea = Objects.requireNonNull(searchArea);
return this; return this;
} }
/**
* 获取搜索区域
* @return 返回搜索区域对象
*/
public SearchArea getSearchArea() {
return searchArea;
}
/** /**
* 获取搜索条件. * 获取搜索条件.
* @return 搜索条件内容 * @return 搜索条件内容
@ -202,50 +194,50 @@ public class PixivSearchBuilder {
return searchContent.toString(); return searchContent.toString();
} }
public PixivSearchBuilder setSearchMode(SearchMode searchMode) { public PixivSearchLinkBuilder setSearchMode(SearchMode searchMode) {
this.searchMode = Objects.requireNonNull(searchMode); this.searchMode = Objects.requireNonNull(searchMode);
return this; return this;
} }
public PixivSearchBuilder setSearchType(SearchType searchType) { public PixivSearchLinkBuilder setSearchType(SearchType searchType) {
this.searchType = Objects.requireNonNull(searchType); this.searchType = Objects.requireNonNull(searchType);
return this; return this;
} }
public PixivSearchBuilder setSearchOrder(SearchOrder searchOrder) { public PixivSearchLinkBuilder setSearchOrder(SearchOrder searchOrder) {
this.searchOrder = Objects.requireNonNull(searchOrder); this.searchOrder = Objects.requireNonNull(searchOrder);
return this; return this;
} }
public PixivSearchBuilder setSearchContentOption(SearchContentOption searchContentOption) { public PixivSearchLinkBuilder setSearchContentOption(SearchContentOption searchContentOption) {
this.searchContentOption = Objects.requireNonNull(searchContentOption); this.searchContentOption = Objects.requireNonNull(searchContentOption);
return this; return this;
} }
public PixivSearchBuilder setRatioOption(RatioOption ratioOption) { public PixivSearchLinkBuilder setRatioOption(RatioOption ratioOption) {
this.ratioOption = Objects.requireNonNull(ratioOption); this.ratioOption = Objects.requireNonNull(ratioOption);
return this; return this;
} }
public PixivSearchBuilder setDateRange(Date startDate, Date endDate) { public PixivSearchLinkBuilder setDateRange(Date startDate, Date endDate) {
this.startDate = startDate; this.startDate = startDate;
this.endDate = endDate; this.endDate = endDate;
return this; return this;
} }
public PixivSearchBuilder setMaxSize(int width, int height) { public PixivSearchLinkBuilder setMaxSize(int width, int height) {
this.wgt = width; this.wgt = width;
this.hgt = height; this.hgt = height;
return this; return this;
} }
public PixivSearchBuilder setMinSize(int width, int height) { public PixivSearchLinkBuilder setMinSize(int width, int height) {
this.wlt = width; this.wlt = width;
this.hlt = height; this.hlt = height;
return this; return this;
} }
public PixivSearchBuilder setPage(int pageIndex) { public PixivSearchLinkBuilder setPage(int pageIndex) {
if (pageIndex <= 0) { if (pageIndex <= 0) {
throw new IllegalArgumentException("Invalid pageIndex: " + pageIndex); throw new IllegalArgumentException("Invalid pageIndex: " + pageIndex);
} }
@ -253,22 +245,22 @@ public class PixivSearchBuilder {
return this; return this;
} }
public PixivSearchBuilder addExcludeKeyword(String keyword) { public PixivSearchLinkBuilder addExcludeKeyword(String keyword) {
excludeKeywords.add(keyword); excludeKeywords.add(keyword);
return this; return this;
} }
public PixivSearchBuilder removeExcludeKeyword(String keyword) { public PixivSearchLinkBuilder removeExcludeKeyword(String keyword) {
excludeKeywords.remove(keyword); excludeKeywords.remove(keyword);
return this; return this;
} }
public PixivSearchBuilder addIncludeKeyword(String keyword) { public PixivSearchLinkBuilder addIncludeKeyword(String keyword) {
includeKeywords.add(keyword); includeKeywords.add(keyword);
return this; return this;
} }
public PixivSearchBuilder removeIncludeKeyword(String keyword) { public PixivSearchLinkBuilder removeIncludeKeyword(String keyword) {
includeKeywords.remove(keyword); includeKeywords.remove(keyword);
return this; return this;
} }

View File

@ -10,55 +10,56 @@ import java.util.GregorianCalendar;
* 目前已整理的一些Pixiv接口列表 * 目前已整理的一些Pixiv接口列表
*/ */
@SuppressWarnings("unused") @SuppressWarnings("unused")
public class PixivURL { public final class PixivURL {
private PixivURL() {}
public static final String PIXIV_INDEX_URL = "https://www.pixiv.net"; public final static String PIXIV_INDEX_URL = "https://www.pixiv.net";
/** /**
* P站预登陆url * P站预登陆url
*/ */
public static final String PIXIV_LOGIN_PAGE_URL = "https://accounts.pixiv.net/login?lang=zh&source=pc&view_type=page&ref=wwwtop_accounts_index"; public final static String PIXIV_LOGIN_PAGE_URL = "https://accounts.pixiv.net/login?lang=zh&source=pc&view_type=page&ref=wwwtop_accounts_index";
/** /**
* P站登录请求url * P站登录请求url
*/ */
public static final String PIXIV_LOGIN_URL = "https://accounts.pixiv.net/api/login?lang=zh"; public final static String PIXIV_LOGIN_URL = "https://accounts.pixiv.net/api/login?lang=zh";
/** /**
* P站搜索请求url * P站搜索请求url
* @deprecated 该接口已被替换, 请使用{@link PixivSearchBuilder}构造搜索Url * @deprecated 该接口已被替换, 请使用{@link PixivSearchLinkBuilder}构造搜索Url
* @see PixivSearchBuilder * @see PixivSearchLinkBuilder
*/ */
@Deprecated @Deprecated
private static final String PIXIV_SEARCH_URL = "https://www.pixiv.net/search.php"; private final static String PIXIV_SEARCH_URL = "https://www.pixiv.net/search.php";
/** /**
* P站搜索用户url * P站搜索用户url
* 需要替换的参数: * 需要替换的参数:
* {nick} - 用户昵称、部分名称 * {nick} - 用户昵称、部分名称
* @deprecated 该接口已被替换, 请使用{@link PixivSearchBuilder}构造搜索Url * @deprecated 该接口已被替换, 请使用{@link PixivSearchLinkBuilder}构造搜索Url
* @see PixivSearchBuilder * @see PixivSearchLinkBuilder
*/ */
@Deprecated @Deprecated
public static final String PIXIV_SEARCH_USER_URL = PIXIV_SEARCH_URL + "?s_mode=s_usr&nick={nick}"; public final static String PIXIV_SEARCH_USER_URL = PIXIV_SEARCH_URL + "?s_mode=s_usr&nick={nick}";
/** /**
* P站搜索插画url * P站搜索插画url
* 需要替换的参数: * 需要替换的参数:
* {word} - 插画相关文本 * {word} - 插画相关文本
* @deprecated 该接口已被替换, 请使用{@link PixivSearchBuilder}构造搜索Url * @deprecated 该接口已被替换, 请使用{@link PixivSearchLinkBuilder}构造搜索Url
* @see PixivSearchBuilder * @see PixivSearchLinkBuilder
*/ */
@Deprecated @Deprecated
public static final String PIXIV_SEARCH_TAG_URL = PIXIV_SEARCH_URL + "?s_mode=s_tag&word={word}"; public final static String PIXIV_SEARCH_TAG_URL = PIXIV_SEARCH_URL + "?s_mode=s_tag&word={word}";
/** /**
* P站插图下载链接获取url * P站插图下载链接获取url
* 需要替换的文本: * 需要替换的文本:
* {illustId} - 插画ID * {illustId} - 插画ID
*/ */
public static final String PIXIV_ILLUST_API_URL = "https://www.pixiv.net/ajax/illust/{illustId}/pages"; public final static String PIXIV_ILLUST_API_URL = "https://www.pixiv.net/ajax/illust/{illustId}/pages";
/** /**
* P站用户插图列表获取API * P站用户插图列表获取API
@ -67,42 +68,42 @@ public class PixivURL {
* {userId} - 用户ID * {userId} - 用户ID
*/ */
//{"error":false,"message":"","body":{"illusts":{"74369837":null,"70990542":null,"70608653":null,"69755191":null,"69729450":null,"69729416":null,"69503608":null,"69288766":null,"69083882":null,"69051458":null,"68484200":null,"68216927":null,"68216866":null,"68192333":null,"67915106":null,"67914932":null,"67854803":null,"67854745":null,"67854670":null,"67787211":null,"67772199":null,"67770637":null,"67754861":null,"67754804":null,"67754726":null,"67740486":null,"67740480":null,"67740450":null,"67740434":null,"67726337":null,"67499196":null,"67499163":null,"67499145":null,"67499111":null,"67499085":null,"67499038":null,"67498987":null,"67473178":null,"66271465":null,"63682753":null,"63682697":null,"59385148":null,"59383265":null,"59383240":null,"59383227":null,"59383173":null},"manga":[],"novels":[],"mangaSeries":[],"novelSeries":[],"pickup":[],"bookmarkCount":{"public":{"illust":1,"novel":0},"private":{"illust":0,"novel":0}}}} //{"error":false,"message":"","body":{"illusts":{"74369837":null,"70990542":null,"70608653":null,"69755191":null,"69729450":null,"69729416":null,"69503608":null,"69288766":null,"69083882":null,"69051458":null,"68484200":null,"68216927":null,"68216866":null,"68192333":null,"67915106":null,"67914932":null,"67854803":null,"67854745":null,"67854670":null,"67787211":null,"67772199":null,"67770637":null,"67754861":null,"67754804":null,"67754726":null,"67740486":null,"67740480":null,"67740450":null,"67740434":null,"67726337":null,"67499196":null,"67499163":null,"67499145":null,"67499111":null,"67499085":null,"67499038":null,"67498987":null,"67473178":null,"66271465":null,"63682753":null,"63682697":null,"59385148":null,"59383265":null,"59383240":null,"59383227":null,"59383173":null},"manga":[],"novels":[],"mangaSeries":[],"novelSeries":[],"pickup":[],"bookmarkCount":{"public":{"illust":1,"novel":0},"private":{"illust":0,"novel":0}}}}
public static final String PIXIV_USER_ILLUST_LIST_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/all"; public final static String PIXIV_USER_ILLUST_LIST_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/all";
/** /**
* 能够同时获取插图信息的用户插图列表获取API * 能够同时获取插图信息的用户插图列表获取API
* 需要替换的文本: * 需要替换的文本:
* {userId} - 用户ID * {userId} - 用户ID
*/ */
public static final String PIXIV_USER_TOP_ILLUST_LIST_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/top"; public final static String PIXIV_USER_TOP_ILLUST_LIST_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/top";
/** /**
* P站单图详情页url * P站单图详情页url
* 需要替换的文本: * 需要替换的文本:
* {illustId} - 插画ID * {illustId} - 插画ID
*/ */
public static final String PIXIV_ILLUST_MEDIUM_URL = "https://www.pixiv.net/member_illust.php?mode=medium&illust_id={illustId}"; public final static String PIXIV_ILLUST_MEDIUM_URL = "https://www.pixiv.net/member_illust.php?mode=medium&illust_id={illustId}";
/** /**
* P站多图详情页url * P站多图详情页url
* 需要替换的文本: * 需要替换的文本:
* {illustId} - 插画ID * {illustId} - 插画ID
*/ */
public static final String PIXIV_ILLUST_MANGA_URL = "https://www.pixiv.net/member_illust.php?mode=manga&illust_id={illustId}"; public final static String PIXIV_ILLUST_MANGA_URL = "https://www.pixiv.net/member_illust.php?mode=manga&illust_id={illustId}";
/** /**
* P站用户页面url * P站用户页面url
* 需要替换的文本: * 需要替换的文本:
* {userId} - 用户ID * {userId} - 用户ID
*/ */
public static final String PIXIV_USER_URL = "https://www.pixiv.net/member.php?id={userId}"; public final static String PIXIV_USER_URL = "https://www.pixiv.net/member.php?id={userId}";
/** /**
* P站插图信息获取API * P站插图信息获取API
* 这个API能获取插图基本信息但不能获取大小 * 这个API能获取插图基本信息但不能获取大小
* 请使用{@link #getPixivIllustInfoAPI(int[])}获取URL * 请使用{@link #getPixivIllustInfoAPI(int[])}获取URL
*/ */
private static final String PIXIV_GET_ILLUST_INFO_URL = "https://www.pixiv.net/ajax/illust/recommend/illusts?"; private final static String PIXIV_GET_ILLUST_INFO_URL = "https://www.pixiv.net/ajax/illust/recommend/illusts?";
/** /**
* P站获取用户所有插图ID的Api * P站获取用户所有插图ID的Api
@ -110,7 +111,15 @@ public class PixivURL {
* 需要替换的文本: * 需要替换的文本:
* {userId} - 用户ID * {userId} - 用户ID
*/ */
public static final String PIXIV_GET_USER_ALL_ILLUST_ID_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/all"; public final static String PIXIV_GET_USER_ALL_ILLUST_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/all";
/**
* P站获取用户推荐插画及用户基本数据
* 这个API能获得作者的部分(推荐)作品, 每个作品有详细数据, 还能获取作者主页信息(比如主页说明, 看板图)
* 需要替换的文本:
* {userId} - 用户ID
*/
public final static String PIXIV_GET_USER_TOP_ILLUST_URL = "https://www.pixiv.net/ajax/user/{userId}/profile/top";
/** /**
* P站标签搜索URL * P站标签搜索URL
@ -118,7 +127,7 @@ public class PixivURL {
* 需要替换的文本: * 需要替换的文本:
* {content} - 大致tag内容 * {content} - 大致tag内容
*/ */
public static final String PIXIV_TAG_SEARCH_URL = "https://www.pixiv.net/ajax/search/tags/{content}"; public final static String PIXIV_TAG_SEARCH_URL = "https://www.pixiv.net/ajax/search/tags/{content}";
/** /**
* 获取动图下载链接和拼接数据. * 获取动图下载链接和拼接数据.
@ -128,11 +137,17 @@ public class PixivURL {
*/ */
public final static String PIXIV_GET_UGOIRA_META_URL = "https://www.pixiv.net/ajax/illust/{illustId}/ugoira_meta"; public final static String PIXIV_GET_UGOIRA_META_URL = "https://www.pixiv.net/ajax/illust/{illustId}/ugoira_meta";
/**
* 获取自己帐号的部分数据(目前仅能获取: 关注数, 粉丝数和看板图)
* 需要登录.
*/
public final static String PIXIV_GET_USER_EXTRA_URL = "https://www.pixiv.net/ajax/user/extra";
/** /**
* 请求时带上需要退出的Cookies * 请求时带上需要退出的Cookies
* 无论成功与否都会返回302重定向到{@linkplain #PIXIV_LOGIN_PAGE_URL 登录页面} * 无论成功与否都会返回302重定向到{@linkplain #PIXIV_LOGIN_PAGE_URL 登录页面}
*/ */
public static final String PIXIV_LOGOUT_URL = "https://www.pixiv.net/logout.php"; public final static String PIXIV_LOGOUT_URL = "https://www.pixiv.net/logout.php";
/** /**
* 构造P站获取插图信息的Api Url * 构造P站获取插图信息的Api Url
@ -279,7 +294,7 @@ public class PixivURL {
/** /**
* Pixiv搜索接口.<br/> * Pixiv搜索接口.<br/>
* 要使用该链接请使用{@link PixivSearchBuilder}构造链接.<br/> * 要使用该链接请使用{@link PixivSearchLinkBuilder}构造链接.<br/>
* 需要替换的参数: <br/> * 需要替换的参数: <br/>
* content - 搜索内容 * content - 搜索内容
*/ */

View File

@ -1,5 +1,6 @@
package net.lamgc.cgj.pixiv; package net.lamgc.cgj.pixiv;
import com.google.common.io.ByteStreams;
import com.google.gson.Gson; import com.google.gson.Gson;
import com.google.gson.JsonArray; import com.google.gson.JsonArray;
import com.google.gson.JsonObject; import com.google.gson.JsonObject;
@ -7,11 +8,11 @@ import com.squareup.gifencoder.GifEncoder;
import com.squareup.gifencoder.Image; import com.squareup.gifencoder.Image;
import com.squareup.gifencoder.ImageOptions; import com.squareup.gifencoder.ImageOptions;
import io.netty.handler.codec.http.HttpHeaderNames; import io.netty.handler.codec.http.HttpHeaderNames;
import net.lamgc.cgj.exception.HttpRequestException;
import org.apache.http.HttpResponse; import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient; import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet; import org.apache.http.client.methods.HttpGet;
import org.apache.http.util.EntityUtils; import org.apache.http.util.EntityUtils;
import org.apache.tomcat.util.http.fileupload.util.Streams;
import org.slf4j.Logger; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
@ -46,12 +47,12 @@ public final class PixivUgoiraBuilder {
log.debug("Request Url: {}", request.getURI()); log.debug("Request Url: {}", request.getURI());
HttpResponse response = httpClient.execute(request); HttpResponse response = httpClient.execute(request);
String bodyStr = EntityUtils.toString(response.getEntity()); String bodyStr = EntityUtils.toString(response.getEntity());
log.debug("JsonBodyStr: {}", bodyStr); log.trace("JsonBodyStr: {}", bodyStr);
JsonObject resultObject = new Gson().fromJson(bodyStr, JsonObject.class); JsonObject resultObject = new Gson().fromJson(bodyStr, JsonObject.class);
if(resultObject.get("error").getAsBoolean()) { if(resultObject.get("error").getAsBoolean()) {
String message = resultObject.get("message").getAsString(); String message = resultObject.get("message").getAsString();
log.error("获取动图元数据失败!(接口报错: {})", message); log.error("获取动图元数据失败!(接口报错: {})", message);
throw new IOException(message); throw new HttpRequestException(response.getStatusLine(), bodyStr);
} else if(!resultObject.has("body")) { } else if(!resultObject.has("body")) {
String message = "接口返回数据不存在body属性, 可能接口发生改变!"; String message = "接口返回数据不存在body属性, 可能接口发生改变!";
log.error(message); log.error(message);
@ -126,14 +127,13 @@ public final class PixivUgoiraBuilder {
HashMap<String, InputStream> frameMap = new HashMap<>(frames.size()); HashMap<String, InputStream> frameMap = new HashMap<>(frames.size());
while((entry = zipInputStream.getNextEntry()) != null) { while((entry = zipInputStream.getNextEntry()) != null) {
log.trace("ZipEntry {} 正在接收...", entry); log.trace("ZipEntry {} 正在接收...", entry);
Streams.copy(zipInputStream, cacheOutputStream, false); ByteStreams.copy(zipInputStream, cacheOutputStream);
frameMap.put(entry.getName(), new ByteArrayInputStream(cacheOutputStream.toByteArray())); frameMap.put(entry.getName(), new ByteArrayInputStream(cacheOutputStream.toByteArray()));
log.trace("ZipEntry {} 已接收完成.", entry); log.trace("ZipEntry {} 已接收完成.", entry);
cacheOutputStream.reset(); cacheOutputStream.reset();
} }
InputStream firstFrameInput = frameMap.get(frames.get(0).getAsJsonObject().get("file").getAsString());
InputStream firstFrameInput = frameMap.get("000000.jpg");
BufferedImage firstFrame = ImageIO.read(firstFrameInput); BufferedImage firstFrame = ImageIO.read(firstFrameInput);
firstFrameInput.reset(); firstFrameInput.reset();
if(width != firstFrame.getWidth() || height != firstFrame.getHeight()) { if(width != firstFrame.getWidth() || height != firstFrame.getHeight()) {
@ -173,15 +173,15 @@ public final class PixivUgoiraBuilder {
private void getUgoiraImageSize() throws IOException { private void getUgoiraImageSize() throws IOException {
log.debug("正在从Pixiv获取动图尺寸..."); log.debug("正在从Pixiv获取动图尺寸...");
HttpGet request = new HttpGet(PixivURL.getPixivIllustInfoAPI(illustId)); HttpGet request = new HttpGet(PixivURL.getPixivIllustInfoAPI(illustId));
log.debug("Request Url: {}", request.getURI()); log.trace("Request Url: {}", request.getURI());
HttpResponse response = httpClient.execute(request); HttpResponse response = httpClient.execute(request);
String responseBody = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8); String responseBody = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
log.debug("ResponseBody: {}", responseBody); log.trace("ResponseBody: {}", responseBody);
JsonObject resultObject = new Gson().fromJson(responseBody, JsonObject.class); JsonObject resultObject = new Gson().fromJson(responseBody, JsonObject.class);
if(resultObject.get("error").getAsBoolean()) { if(resultObject.get("error").getAsBoolean()) {
String message = resultObject.get("message").getAsString(); String message = resultObject.get("message").getAsString();
log.error("接口返回错误: {}", message); log.error("接口返回错误: {}", message);
throw new IOException(message); throw new HttpRequestException(response.getStatusLine(), responseBody);
} }
JsonArray illustsArray = resultObject.getAsJsonObject("body").getAsJsonArray("illusts"); JsonArray illustsArray = resultObject.getAsJsonObject("body").getAsJsonArray("illusts");

View File

@ -1,149 +0,0 @@
package net.lamgc.cgj.proxy;
import com.github.monkeywie.proxyee.intercept.HttpProxyIntercept;
import com.github.monkeywie.proxyee.intercept.HttpProxyInterceptInitializer;
import com.github.monkeywie.proxyee.intercept.HttpProxyInterceptPipeline;
import com.github.monkeywie.proxyee.intercept.common.CertDownIntercept;
import com.github.monkeywie.proxyee.proxy.ProxyConfig;
import com.github.monkeywie.proxyee.server.HttpProxyServer;
import com.github.monkeywie.proxyee.server.HttpProxyServerConfig;
import io.netty.channel.Channel;
import io.netty.handler.codec.http.HttpHeaderNames;
import io.netty.handler.codec.http.HttpHeaders;
import io.netty.handler.codec.http.HttpRequest;
import io.netty.handler.codec.http.HttpResponse;
import org.apache.http.client.CookieStore;
import org.apache.http.impl.cookie.BasicClientCookie;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.net.HttpCookie;
import java.util.Date;
import java.util.List;
/**
* 登录成功后提供CookieStore, 然后由程序自动登录Pixiv
* @author LamGC
*/
public class PixivAccessProxyServer {
private final Logger log = LoggerFactory.getLogger(PixivAccessProxyServer.class);
private final HttpProxyServer proxyServer;
private final CookieStore cookieStore;
public PixivAccessProxyServer(CookieStore cookieStore){
this(cookieStore, null);
}
public PixivAccessProxyServer(CookieStore cookieStore, ProxyConfig proxyConfig){
HttpProxyServerConfig config = new HttpProxyServerConfig();
this.cookieStore = cookieStore;
config.setHandleSsl(true);
this.proxyServer = new HttpProxyServer();
this.proxyServer
.serverConfig(config)
.proxyConfig(proxyConfig)
.proxyInterceptInitializer(new HttpProxyInterceptInitializer(){
@Override
public void init(HttpProxyInterceptPipeline pipeline) {
pipeline.addLast(new CertDownIntercept());
pipeline.addLast(new HttpProxyIntercept(){
private boolean match(HttpRequest request){
String host = request.headers().get(HttpHeaderNames.HOST);
return host.equalsIgnoreCase("pixiv.net") || host.contains(".pixiv.net");
}
@Override
public void beforeRequest(Channel clientChannel, HttpRequest httpRequest, HttpProxyInterceptPipeline pipeline) throws Exception {
log.info("URL: " + httpRequest.headers().get(HttpHeaderNames.HOST) + httpRequest.uri());
if(!match(httpRequest)){
super.beforeRequest(clientChannel, httpRequest, pipeline);
return;
}
log.info("正在注入Cookies...");
HttpHeaders requestHeaders = httpRequest.headers();
if(requestHeaders.contains(HttpHeaderNames.COOKIE)){
log.info("原请求存在自带Cookies, 正在清除Cookies...");
log.debug("原Cookies: {}", requestHeaders.getAsString(HttpHeaderNames.COOKIE));
requestHeaders.remove(HttpHeaderNames.COOKIE);
}
StringBuilder cookieBuilder = new StringBuilder();
cookieStore.getCookies().forEach(cookie -> {
if(cookie.isExpired(new Date())){
return;
}
cookieBuilder.append(cookie.getName()).append("=").append(cookie.getValue()).append("; ");
});
log.info("Cookies构造完成, 结果: " + cookieBuilder.toString());
requestHeaders.add(HttpHeaderNames.COOKIE, cookieBuilder.toString());
log.info("Cookies注入完成.");
super.beforeRequest(clientChannel, httpRequest, pipeline);
}
@Override
public void afterResponse(Channel clientChannel, Channel proxyChannel, HttpResponse httpResponse, HttpProxyInterceptPipeline pipeline) throws Exception {
if(!match(pipeline.getHttpRequest())){
super.afterResponse(clientChannel, proxyChannel, httpResponse, pipeline);
return;
}
log.info("正在更新Response Cookie...(Header Name: " + HttpHeaderNames.SET_COOKIE + ")");
List<String> responseCookies = httpResponse.headers().getAll(HttpHeaderNames.SET_COOKIE);
responseCookies.forEach(value -> {
/*if(check(value)){
log.info("黑名单Cookie, 已忽略: " + value);
return;
}*/
log.info("Response Cookie: " + value);
BasicClientCookie cookie = parseRawCookie(value);
cookieStore.addCookie(cookie);
});
httpResponse.headers().remove(HttpHeaderNames.SET_COOKIE);
super.afterResponse(clientChannel, proxyChannel, httpResponse, pipeline);
}
protected BasicClientCookie parseRawCookie(String rawCookie) {
List<HttpCookie> cookies = HttpCookie.parse(rawCookie);
if (cookies.size() < 1)
return null;
HttpCookie httpCookie = cookies.get(0);
BasicClientCookie cookie = new BasicClientCookie(httpCookie.getName(), httpCookie.getValue());
if (httpCookie.getMaxAge() >= 0) {
Date expiryDate = new Date(System.currentTimeMillis() + httpCookie.getMaxAge() * 1000);
cookie.setExpiryDate(expiryDate);
}
if (httpCookie.getDomain() != null)
cookie.setDomain(httpCookie.getDomain());
if (httpCookie.getPath() != null)
cookie.setPath(httpCookie.getPath());
if (httpCookie.getComment() != null)
cookie.setComment(httpCookie.getComment());
cookie.setSecure(httpCookie.getSecure());
return cookie;
}
});
}
});
}
public void start(int port){
this.proxyServer.start(port);
}
public void close(){
this.proxyServer.close();
}
/**
* 导出CookieStore.
* 注意!该方法导出的CookieStore不适用于ApacheHttpClient, 如需使用则需要进行转换.
* @return CookieStore对象
*/
public CookieStore getCookieStore(){
return this.cookieStore;
}
}

View File

@ -0,0 +1,65 @@
package net.lamgc.cgj.util;
import java.util.concurrent.atomic.AtomicInteger;
public final class Locker<K> {
private final LockerMap<K> fromMap;
private final K key;
private final boolean autoDestroy;
private final AtomicInteger lockCount = new AtomicInteger(0);
/**
* 构造一个锁对象
* @param map 所属LockerMap
* @param key 所属Key
*/
Locker(LockerMap<K> map, K key, boolean autoDestroy) {
this.fromMap = map;
this.key = key;
this.autoDestroy = autoDestroy;
}
/**
* 上锁
*/
public void lock() {
lockCount.incrementAndGet();
}
/**
* 解锁
*/
public void unlock() {
int newValue = lockCount.decrementAndGet();
if(newValue <= 0 && autoDestroy) {
destroy();
}
}
/**
* 获取锁对象所属Key
*/
public K getKey() {
return key;
}
/**
* 销毁锁对象
*/
public void destroy() {
fromMap.destroyLocker(this);
}
@Override
public String toString() {
return "Locker@" + this.hashCode() + "{" +
"fromMap=" + fromMap +
", key=" + key +
'}';
}
}

View File

@ -0,0 +1,31 @@
package net.lamgc.cgj.util;
import java.util.HashMap;
public class LockerMap<K> {
private final HashMap<K, Locker<K>> lockerHashMap = new HashMap<>();
/**
* 创建锁
* @param key Key
* @return 如果Key所属锁存在, 则返回对应锁, 否则返回新锁
*/
public Locker<K> createLocker(K key, boolean autoDestroy) {
if(lockerHashMap.containsKey(key)) {
return lockerHashMap.get(key);
}
Locker<K> newLocker = new Locker<>(this, key, autoDestroy);
lockerHashMap.put(key, newLocker);
return newLocker;
}
/**
* 销毁锁
* @param locker 锁对象
*/
public void destroyLocker(Locker<K> locker) {
lockerHashMap.remove(locker.getKey());
}
}

View File

@ -2,6 +2,7 @@ package net.lamgc.cgj.util;
import java.util.*; import java.util.*;
import java.util.concurrent.*; import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicLong; import java.util.concurrent.atomic.AtomicLong;
/** /**
@ -22,29 +23,53 @@ public class TimeLimitThreadPoolExecutor extends ThreadPoolExecutor {
*/ */
private final AtomicLong timeoutCheckInterval = new AtomicLong(100); private final AtomicLong timeoutCheckInterval = new AtomicLong(100);
private final Map<Thread, AtomicLong> workerThreadMap = new Hashtable<>(); private final Map<Thread, MonitorInfo> workerThreadMap = new Hashtable<>();
private final Thread timeoutCheckThread = createTimeoutCheckThread(); private final Thread timeoutCheckThread = createTimeoutCheckThread();
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue) { public TimeLimitThreadPoolExecutor(long executeLimitTime,
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue); super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
setInitialTime(executeLimitTime); setInitialTime(executeLimitTime);
timeoutCheckThread.start(); timeoutCheckThread.start();
} }
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory) { public TimeLimitThreadPoolExecutor(long executeLimitTime,
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue,
ThreadFactory threadFactory) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory); super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory);
setInitialTime(executeLimitTime); setInitialTime(executeLimitTime);
timeoutCheckThread.start(); timeoutCheckThread.start();
} }
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, RejectedExecutionHandler handler) { public TimeLimitThreadPoolExecutor(long executeLimitTime,
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue,
RejectedExecutionHandler handler) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, handler); super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, handler);
setInitialTime(executeLimitTime); setInitialTime(executeLimitTime);
timeoutCheckThread.start(); timeoutCheckThread.start();
} }
public TimeLimitThreadPoolExecutor(long executeLimitTime, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory, RejectedExecutionHandler handler) { public TimeLimitThreadPoolExecutor(long executeLimitTime,
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue,
ThreadFactory threadFactory,
RejectedExecutionHandler handler) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler); super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler);
setInitialTime(executeLimitTime); setInitialTime(executeLimitTime);
timeoutCheckThread.start(); timeoutCheckThread.start();
@ -107,14 +132,17 @@ public class TimeLimitThreadPoolExecutor extends ThreadPoolExecutor {
while (true) { while (true) {
try { try {
long interval = this.timeoutCheckInterval.get(); long interval = this.timeoutCheckInterval.get();
//noinspection BusyWait 用于等待超时
Thread.sleep(interval); Thread.sleep(interval);
// 检查是否存在超时的任务 // 检查是否存在超时的任务
workerThreadMap.forEach((thread, time) -> { final long executeTimeLimit = this.executeTimeLimit.get();
long currentTime = time.getAndAdd(interval); workerThreadMap.forEach((thread, info) -> {
if(currentTime > executeTimeLimit.get()) { long currentTime = info.getTimeRemaining().getAndAdd(interval);
if(!thread.isInterrupted()) { if(currentTime > executeTimeLimit) {
if(!info.getNotifyInterrupted().get() && !thread.isInterrupted()) {
thread.interrupt(); thread.interrupt();
info.getNotifyInterrupted().set(true);
} }
} }
}); });
@ -130,7 +158,7 @@ public class TimeLimitThreadPoolExecutor extends ThreadPoolExecutor {
@Override @Override
protected void beforeExecute(Thread t, Runnable r) { protected void beforeExecute(Thread t, Runnable r) {
workerThreadMap.put(t, new AtomicLong()); workerThreadMap.put(t, new MonitorInfo());
super.beforeExecute(t, r); super.beforeExecute(t, r);
} }
@ -145,4 +173,20 @@ public class TimeLimitThreadPoolExecutor extends ThreadPoolExecutor {
this.timeoutCheckThread.interrupt(); this.timeoutCheckThread.interrupt();
super.terminated(); super.terminated();
} }
private static class MonitorInfo {
private final AtomicLong timeRemaining = new AtomicLong();
private final AtomicBoolean notifyInterrupted = new AtomicBoolean(false);
public AtomicBoolean getNotifyInterrupted() {
return notifyInterrupted;
}
public AtomicLong getTimeRemaining() {
return timeRemaining;
}
}
} }

View File

@ -1,2 +0,0 @@
server.port=8081
server.tomcat.max-threads=1

View File

@ -0,0 +1,12 @@
server:
port: 8081
spring:
cq:
plugin-list:
- net.lamgc.cgj.bot.framework.coolq.CQPluginMain
event:
corePoolSize: 8
maxPoolSize: 16
keepAliveTime: 25000
workQueueSize: 1024

View File

@ -19,7 +19,7 @@
</MarkerPatternSelector> </MarkerPatternSelector>
</PatternLayout> </PatternLayout>
<Filters> <Filters>
<ThresholdFilter level="WARN" onMatch="DENY" onMismatch="ACCEPT"/> <LevelRangeFilter minLevel="INFO" maxLevel="INFO" onMatch="ACCEPT" onMismatch="DENY"/>
</Filters> </Filters>
</Console> </Console>
<Console name="STANDARD_STDERR" target="SYSTEM_ERR"> <Console name="STANDARD_STDERR" target="SYSTEM_ERR">
@ -34,6 +34,9 @@
</Console> </Console>
<RollingFile name="rollingFile" fileName="${logsDir}/latest.log" filePattern="${logsDir}/running.%-d{yyyy-MM-dd_HH-mm-ss}.log.gz"> <RollingFile name="rollingFile" fileName="${logsDir}/latest.log" filePattern="${logsDir}/running.%-d{yyyy-MM-dd_HH-mm-ss}.log.gz">
<Filters>
<ThresholdFilter level="DEBUG" onMatch="ACCEPT" onMismatch="DENY"/>
</Filters>
<PatternLayout charset="${charset}"> <PatternLayout charset="${charset}">
<MarkerPatternSelector defaultPattern="${standard_pattern}"> <MarkerPatternSelector defaultPattern="${standard_pattern}">
<PatternMatch key="mirai" pattern="${mirai_pattern}" /> <PatternMatch key="mirai" pattern="${mirai_pattern}" />
@ -46,19 +49,10 @@
</Appenders> </Appenders>
<Loggers> <Loggers>
<Logger level="INFO" name="org.apache.http"> <Logger level="INFO" name="org.apache.http"/>
<Root level="DEBUG">
<AppenderRef ref="STANDARD_STDOUT"/> <AppenderRef ref="STANDARD_STDOUT"/>
<AppenderRef ref="STANDARD_STDERR"/> <AppenderRef ref="STANDARD_STDERR"/>
</Logger>
<Logger level="INFO" name="mirai">
<AppenderRef ref="STANDARD_STDOUT"/>
<AppenderRef ref="STANDARD_STDERR"/>
</Logger>
<Logger level="INFO" name="net.lamgc.cgj">
<AppenderRef ref="STANDARD_STDOUT"/>
<AppenderRef ref="STANDARD_STDERR"/>
</Logger>
<Root level="TRACE">
<AppenderRef ref="rollingFile"/> <AppenderRef ref="rollingFile"/>
</Root> </Root>
</Loggers> </Loggers>

View File

@ -1,9 +0,0 @@
#用于访问Pixiv的代理服务器
reptile.proxy.type=socks5/socks4/http
reptile.proxy.host=127.0.0.1
reptile.proxy.port=1080
reptile.proxy.username=
reptile.proxy.password=
#登录用代理, 需要让浏览器使用该代理, 访问Pixiv并登录
login.proxy.host=127.0.0.1
login.proxy.port=1080

View File

@ -0,0 +1,30 @@
package net.lamgc.cgj.bot.util;
import org.junit.Assert;
import org.junit.Test;
public class GroupMuteManagerTest {
@Test
public void muteStateTest() {
GroupMuteManager manager = new GroupMuteManager();
Assert.assertNull(manager.isMute(1, true)); // 未设置的群组返回null
Assert.assertFalse(manager.isMute(1, false)); // 未设置就返回false
manager.setMuteState(1, true); // mute == true
Assert.assertNotNull(manager.isMute(1, true)); // 第一次设置后不为null
Assert.assertTrue(manager.isMute(1, false)); // 确保条件正常
manager.setMuteState(2, true); // 不能出现不同群号的冲突
manager.setMuteState(1, false);
Assert.assertTrue(manager.isMute(2, false));
Assert.assertNotNull(manager.isMute(1, true)); // 变更为false后依然不能返回null
Assert.assertFalse(manager.isMute(1, false));
}
@Test
public void invalidGroupIdTest() {
GroupMuteManager manager = new GroupMuteManager();
manager.setMuteState(-1, true); // 设置应该是无效的
Assert.assertFalse(manager.isMute(-1, false)); // 由于设置无效, 返回false即可
}
}

View File

@ -1,26 +0,0 @@
package net.lamgc.cgj.pixiv;
import org.junit.Assert;
import org.junit.Test;
public class PixivSearchBuilderTest {
@Test
public void buildTest() {
PixivSearchBuilder builder = new PixivSearchBuilder("hololive");
//builder.addExcludeKeyword("fubuki").addExcludeKeyword("minato");
builder.addIncludeKeyword("35").addIncludeKeyword("okayu").addIncludeKeyword("百鬼あやめ");
System.out.println(builder.buildURL());
}
@Test
public void equalsTest() {
Assert.assertEquals(new PixivSearchBuilder("风景"), new PixivSearchBuilder("风景"));
}
@Test
public void hashCodeTest() {
Assert.assertEquals(new PixivSearchBuilder("风景").hashCode(), new PixivSearchBuilder("风景").hashCode());
}
}

View File

@ -0,0 +1,25 @@
package net.lamgc.cgj.pixiv;
import org.junit.Assert;
import org.junit.Test;
public class PixivSearchLinkBuilderTest {
@Test
public void buildTest() {
PixivSearchLinkBuilder builder = new PixivSearchLinkBuilder("hololive");
builder.addIncludeKeyword("35").addIncludeKeyword("okayu").addIncludeKeyword("百鬼あやめ");
System.out.println(builder.buildURL());
}
@Test
public void equalsTest() {
Assert.assertEquals(new PixivSearchLinkBuilder("风景"), new PixivSearchLinkBuilder("风景"));
}
@Test
public void hashCodeTest() {
Assert.assertEquals(new PixivSearchLinkBuilder("风景").hashCode(), new PixivSearchLinkBuilder("风景").hashCode());
}
}

View File

@ -0,0 +1,18 @@
package net.lamgc.cgj.util;
import org.junit.Assert;
import org.junit.Test;
public class LockerMapTest {
@Test
public void createAndFinalizeTest() {
LockerMap<String> map = new LockerMap<>();
Locker<String> locker = map.createLocker("Test", true);
Assert.assertEquals(locker, map.createLocker("Test", true));
locker.lock();
locker.unlock();
System.gc();
}
}

View File

@ -2,26 +2,37 @@ package net.lamgc.cgj.util;
import org.junit.Assert; import org.junit.Assert;
import org.junit.Test; import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.concurrent.ArrayBlockingQueue; import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.TimeUnit; import java.util.concurrent.TimeUnit;
public class TimeLimitThreadPoolExecutorTest { public class TimeLimitThreadPoolExecutorTest {
private final static Logger log = LoggerFactory.getLogger(TimeLimitThreadPoolExecutorTest.class);
@Test @Test
public void timeoutTest() throws InterruptedException { public void timeoutTest() throws InterruptedException {
TimeLimitThreadPoolExecutor executor = new TimeLimitThreadPoolExecutor(1000, 1, 1, 30, TimeUnit.SECONDS, new ArrayBlockingQueue<>(50)); TimeLimitThreadPoolExecutor executor = new TimeLimitThreadPoolExecutor(1000, 1, 1, 30, TimeUnit.SECONDS, new ArrayBlockingQueue<>(50));
System.out.println(executor.isTerminated()); log.info("ThreadPoolExecutor.isTerminated: {}", executor.isTerminated());
System.out.println(executor.isShutdown()); log.info("ThreadPoolExecutor.isShutdown: {}", executor.isShutdown());
executor.setTimeoutCheckInterval(150); executor.setTimeoutCheckInterval(150);
System.out.println("当前设定: ETL: " + executor.getExecuteTimeLimit() + "ms, TCI: " + executor.getTimeoutCheckInterval() + "ms"); log.info("当前设定: ExecuteTimeLimit: {}ms, CheckInterval: {}ms", executor.getExecuteTimeLimit(),
executor.getTimeoutCheckInterval());
executor.execute(() -> { executor.execute(() -> {
try { try {
Thread.sleep(5 * 1000); Thread.sleep(5000);
} catch (InterruptedException e) { } catch (InterruptedException e) {
System.out.println("线程 " + Thread.currentThread().getName() + " 被中断"); System.out.println("线程 " + Thread.currentThread().getName() + " 被中断");
} }
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
Assert.fail("Multiple interrupts occurred");
}
}); });
executor.shutdown(); executor.shutdown();
Assert.assertTrue(executor.awaitTermination(5 * 1000, TimeUnit.MILLISECONDS)); Assert.assertTrue(executor.awaitTermination(5 * 1000, TimeUnit.MILLISECONDS));