Compare commits

...

30 Commits

Author SHA1 Message Date
60fa641962 [Version] 更新版本(2.5.2-20200517.1-SNAPSHOT -> 2.5.2-20200520.1-SNAPSHOT); 2020-05-20 10:36:30 +08:00
3ab373cc70 [Update] .gitignore 补充VSCode忽略项; 2020-05-20 10:34:55 +08:00
461cd246d8 [Add] BotCommandProcess 新增'色图'功能以随机从排行榜中获取一张图片;
[Add] BufferMessageEvent 增加用于获取Sender发送内容的Event实现;
[Change] BotEventHandler Handler现在允许命令处理方法不返回消息(返回'null');
2020-05-20 09:55:50 +08:00
f040f130d7 [Update] Main, AutoArtworksSender, ImageCacheStore, PixivAccessProxyServer, PagesQualityParser 整理代码问题, 删除无用类; 2020-05-20 09:05:50 +08:00
1f402fbbac [Change] LocalHashCacheStore 实现'Cleanable'接口, 增加'<init>(int, int, boolean)'构造函数以允许自动清理
[Update] LocalHashCacheStore 补充Javadoc内容;
[Update] .gitignore 补充并整理忽略项;
2020-05-20 08:56:27 +08:00
f7f3c3beaf [Fix] BotCommandProcess 修复help命令中帮助信息的错误; 2020-05-17 19:38:17 +08:00
04c1753c22 [Version] 更新版本(2.5.1 -> 2.5.2-20200517.1-SNAPSHOT); 2020-05-17 17:45:06 +08:00
d993e9d719 [Add] MiraiMessageSender 增加对默认表情的支持; 2020-05-12 15:25:29 +08:00
60e91987d1 Merge branch 'mirai' 2020-05-12 15:14:58 +08:00
65392fc2fe [Fix] net.mamoe:mirai-core, net.mamoe:mirai-core-qqandroid 更新Mirai-Core版本号以修复VerifyError问题(1.0-RC2 -> 1.0-RC2-1);
[Fix] MiraiMessageSender 调整日志输出, 以修复日志打印处理后消息顺序混乱的问题;
2020-05-12 15:14:28 +08:00
d38934a0f4 [Change] RankingUpdateTimer 调整时间补齐的阀值(12:00 -> 11:30);
[Add] documents/interfaces/* 整理部分Pixiv接口的文档;
2020-05-12 15:03:08 +08:00
7943357d96 [Fix] MiraiMain 修复未登录时异常退出, 抛出NPE的问题; 2020-05-11 12:16:35 +08:00
a170ff040b [Update] net.mamoe:mirai-core, net.mamoe:mirai-core-qqandroid 更新依赖项版本(0.39.4 -> 1.0-RC2);
[Update] MiraiMain, MiraiMessageEvent 适配mirai新版本;
[Change] MiraiMessageSenderFactory 增加参数检查并优化过程;
[Change] MiraiMain 显性指定Mirai所用协议为'BotConfiguration.MiraiProtocol.ANDROID_PAD';
2020-05-11 11:36:37 +08:00
cb2ebfdb73 [Change] MessageEvent 对消息内容删除首尾空格以提高用户体验性; 2020-05-09 23:13:53 +08:00
66b22c543a [Fix] SettingProperties 修复了配置文件无法创建, 导致无法保存配置的问题;
[Fix] Main 修复了潜在的文件创建失败的问题;
2020-05-09 22:54:32 +08:00
6bace4b048 [Version] 更新版本(2.5.0 -> 2.5.1); 2020-05-09 22:35:46 +08:00
597aac4e95 [Fix] BotCommandProcess 修复部分指令无法使用群组配置的问题; 2020-05-09 22:35:21 +08:00
181d60285b [Version] 更新版本(2.5.0-20200509.1-SNAPSHOT -> 2.5.0); 2020-05-09 22:20:07 +08:00
c06b8717c3 [Add] CacheStoreUtils 增加用于缓存库的工具类;
[Update] BotCommandProcess 补充注释, 优化"info"命令输出内容的格式;
2020-05-09 22:19:29 +08:00
b4a28b6735 [Version] 更新版本(2.5.0-20200506.1-SNAPSHOT -> 2.5.0-20200509.1-SNAPSHOT); 2020-05-09 18:16:54 +08:00
40057c3683 [Fix] Issue #1 修复了ImageCache对重复缓存请求的策略(忽略并返回)导致的后续处理发生NPE的问题;
[Update] ImageCache 更新了ImageCache, 将该部分单独分离到 ImageCacheStore;
[Update] BotCommandProcess 适配新的ImageCache, 调整内容检查的时机以减少不必要的API访问;
2020-05-09 18:16:16 +08:00
4beb4d78fb [Fix] BotCommandProcess 修复Search命令格式不正确的问题; 2020-05-09 12:52:15 +08:00
d9edaa681f [Add] BotAdminCommandProcess 为 addPushGroup 增加参数检查; 2020-05-09 10:10:48 +08:00
7b52abde60 [Change] BotCommandProcess 增加Search命令的反馈内容(可能必须要QQ支持长文本才能使用); 2020-05-08 23:31:24 +08:00
2cb4fe1dbc [Fix] SettingProperties 修复无法设置全局配置项的问题; 2020-05-08 12:23:43 +08:00
5830327dad [Add] BotEventHandler 增加禁言记录, 防止应用因过多的在禁言状态反馈导致被封号;
[Update] MiraiMain 适配禁言记录功能;
2020-05-07 19:46:54 +08:00
cbd10ff281 [Fix] Issue #4 修复因Mirai API变动导致应用无法从消息获取ImageId的问题; 2020-05-07 18:24:09 +08:00
84b8006f89 [Change] RankingUpdateTimer 将定时更新时间调整为"+8 11:30", 并增加日期指定; 2020-05-07 15:21:05 +08:00
d32d891ad7 [Update] BotCommandProcess 补充Javadoc内容;
[Add] BotCommandProcess 在Ranking命令增加force参数以允许强制查询预计不在排行榜范围的日期;
2020-05-07 11:29:31 +08:00
aabd35ce71 [Add] AutoCleanTimer, Cleanable 增加缓存库自动清理;
[Update] HotDataCacheStore 增加对自动清理的支持;
[Change] RedisPoolCacheStore 将Redis缓存库抽象类设为default(原public);
[Change] MiraiMessageSender, BotCommandProcess 适配HotDataCacheStore的修改;
[Change] BotCommandProcess 调整ranking命令的输出格式;
2020-05-07 10:26:11 +08:00
28 changed files with 849 additions and 220 deletions

14
.gitignore vendored
View File

@ -1,6 +1,16 @@
# Ignore test date folder
/pluginData/
/logs/
/.idea/
/CGJ_2.iml
/cookies.store
/target/
# Ignore Idea files
/.idea/
/CGJ_2.iml
# Ignore Visual Studio Code files
.classpath
.factorypath
.project
/.settings/
/.vscode/

View File

@ -0,0 +1,18 @@
## Pixiv接口标准返回格式 ##
Pixiv大部分接口在返回数据时都会遵循以下格式
```json
{
"error": false,
"message": "",
"body": {
}
}
```
大部分是如此(部分接口比较特别, 不是这个格式)
属性|类型|说明
---|---|---
error|Boolean|如果接口返回错误信息, 该属性为`true`
message|String|如果`error`属性为`true`, 则该属性不为空字符串
body|Object/Array|如果`error`不为`true`, 该属性有数据, 否则属性可能不存在

View File

@ -0,0 +1,16 @@
## Pixiv内容搜索接口 ##
接口地址:
```
GET https://www.pixiv.net/ajax/search/{Type}/{Content}?{参数...}
```
变量名:
- `Type`: 内容类型
- illustrations(插画)
- top(推荐)
- manga(漫画)
- novels(小说)
- `Content`: 搜索内容
参数列表:
- `word`

View File

@ -0,0 +1,127 @@
## Pixiv首页数据接口 ##
说明:
该接口涉及用户账户隐私, 不要尝试对该接口返回数据做不安全云端存储, 或未经用户允许的发送出去.
接口地址:
```
GET https://www.pixiv.net/ajax/top/{type}?mode={mode}&lang={lang}
```
参数:
- `type`: 首页类型
- `illust`: 插画
- `manga`: 漫画
- `novel`: 小说
- `mode`: 内容类型
- `all`: 不限类型
- `r18`: 成人内容
- `lang`: 语言(只写几个)
- `zh`: 中文
> 是否需要登录: `是`
> 是否为Pixiv接口标准返回格式: `是`
> 是否需要Referer请求头: `未知`
请求Url示例:
```
GET https://www.pixiv.net/ajax/top/illust?mode=all&lang=zh
```
响应示例:
```
(内容过长, 略)
```
返回内容(Json):
- `page`: 网页相关内容
- `tags`: (`Object[]`) 热门标签
- `tag`: (`String`) 标签名
- `count`: (`Number`) 作品数量?
- `lev`: (`Number`) 不明?
- `ids`: (`Number[]`) 作品数组
- `follow`: (`Number[]`) 已关注作者的作品推荐
- `mypixiv`: (`?[]`) 不明
- `recommend`: (`String[]` -> `Number[]`) 推荐作品的Id?
- `recommendByTag`: (`Object[]`) 标签的推荐作品
- `tag`: (`String`) 标签名
- `ids`: ((`String[]` -> `Number[]`) 作品Id数组
- `ranking`: (`Object`) 排行榜前100名数据
- `date`: (`String`) 排行榜日期(`yyyyMMdd`)
- `items`: (`Object[]`) 排行榜简略数据
- `rank`: (`String` -> `Number`) 排行榜名次
- `id`: (`String` -> `Number`) 作品Id
- `pixivision`: (`Object[]`) Pixiv的推荐文章
- `id`: (`String` -> `Number`) 文章Id
- `title`: (`String`) 文章标题
- `thumbnailUrl`: (`String`) 文章封面图的Url地址
- `abtestId`: (`String` -> `Number`) pixivision文章地址的参数`utm_content`的值
- `recommendUser`: (`Object[]`) 推荐用户及其作品
- `id`: (`Number`) 用户Id
- `illustIds`: (`String[]` -> `Number[]`) 插画作品Id
- `novelIds`: (`String[]` -> `Number[]`) 小说作品Id
- `contestOngoing`: (`Object[]`) 进行中的比赛活动(没找到更多信息了, 先这么说着)
- (待完善)
- `contestResult`: (`Object[]`) 比赛结果信息
- `slug`: (`String`) 比赛代号?
- `type`: (`String`) 比赛作品类型?
- `name`: (`String`) 比赛名
- `url`: (`String`) 结果公布链接
- `iconUrl`: (`String`) 图标链接(不明意义的图标)
- `editorRecommend`: (`Object[]`) 编辑推荐(小说), 后续可能会删除, 这个应该是配合活动出的数据
- (待完善)
- `boothFollowItemIds`: (`String[]` -> `Number[]`) 已关注用户的最新商品
- `sketchLiveFollowIds`: (`?[]`) 不明?
- `sketchLivePopularIds`: (`String[]` -> `Number[]`) 不明?
- `myFavoriteTags`: (`?[]`) 关注的标签
- `newPost`: (`String[]` -> `Number[]`) 不明?
- `thumbnails`: (`Object`) 已关注用户的作品
- `illust`: (`Object[]`) 插画作品
- `illustId`: (`String` -> `Number`) 作品Id(或者准确来讲是 插画Id?)
- `illustTitle`: (`String`) 插画标题
- `id`: (`String` -> `Number`) 作品Id
- `title`: (`String`) 作品标题
- `illustType`: (`Number`) 作品类型
- `xRestrict`: (`Number`) 不明?
- `restrict`: (`Number`) 不明?
- `sl`: (`Number`) 不明?
- `url`: (`String`) 作品在主页的封面图Url
- `description`: (`String`) 作品说明?
- `tags`: (`String[]`) 标签原始名数组(不带翻译的原始名称)
- `userId`: (`String` -> `Number`) 用户Id
- `userName`: (`String`) 用户名
- `width`: (`Number`) 作品宽度
- `height`: (`Number`) 作品高度
- `pageCount`: (`Number`) 作品页数
- `isBookmarkable`: (`Boolean`) 不明?
- `bookmarkData`: (`?`) 不明?
- `alt`: (`String`) 简略信息
- `isAdContainer`: (`Boolean`) 广告标识?
- `titleCaptionTranslation`: (`Object`) 不明?
- (待完善)
- `urls`: (`Object`) 其他封面图尺寸的链接
- (略)
- `seriesId`: (`?`) 不明?
- `seriesTitle`: (`?`) 不明?
- `profileImageUrl`: (`String`) 用户头像图链接
- `users`: (`Object[]`) 用户信?
- `userId`: (`String` -> `Number`) 用户Id
- `name`: (`String`) 用户名
- `image`: (`String`) 用户头像图链接
- `imageBig`: (`String`) 用户大尺寸头像图链接
- `premium`: (`Boolean`) 是否为Pixiv高级会员
- `isFollowed`: (`Boolean`) 是否关注
- `isMypixiv`: (`Boolean`) 不明?
- `isBlocking`: (`Boolean`) 是否为黑名单?
- `background`: (`?`) 不明?
- `partial`: (`Number`) 不明?
- `tagTranslation`: 标签翻译名
- `key=${标签原始名}`: (`Object`) 标签翻译信息
- `${语言代码}`: (`String`) 对应语言代码的翻译, 不一定有
- `boothItems`: (`Object[]`) 商品信息
- (待完善)
- `sketchLives`: (`Object[]`) 不明?
- (待完善)
- `zoneConfig`: (`Object`) 不明?
- (待完善)

View File

@ -6,7 +6,7 @@
<groupId>net.lamgc</groupId>
<artifactId>ContentGrabbingJi</artifactId>
<version>2.5.0-20200506.1-SNAPSHOT</version>
<version>2.5.2-20200520.1-SNAPSHOT</version>
<repositories>
<repository>
@ -19,7 +19,7 @@
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<maven.compiler.encoding>UTF-8</maven.compiler.encoding>
<mirai.CoreVersion>0.39.4</mirai.CoreVersion>
<mirai.CoreVersion>1.0-RC2-1</mirai.CoreVersion>
<mirai.JaptVersion>1.1.1</mirai.JaptVersion>
<kotlin.version>1.3.71</kotlin.version>
<ktor.version>1.3.2</ktor.version>
@ -86,7 +86,7 @@
<dependency>
<groupId>net.lamgc</groupId>
<artifactId>java-utils</artifactId>
<version>1.2.0_20200505.1-SNAPSHOT</version>
<version>1.2.0_20200514.1-SNAPSHOT</version>
</dependency>
<dependency>

View File

@ -86,13 +86,15 @@ public class Main {
File cookieStoreFile = new File(System.getProperty("cgj.botDataDir"), "cookies.store");
if(!cookieStoreFile.exists()) {
log.warn("未找到cookies.store文件, 是否启动PixivLoginProxyServer? (yes/no)");
Scanner scanner = new Scanner(System.in);
if(scanner.nextLine().trim().equalsIgnoreCase("yes")) {
startPixivLoginProxyServer();
} else {
System.exit(1);
return;
try(Scanner scanner = new Scanner(System.in)) {
if(scanner.nextLine().trim().equalsIgnoreCase("yes")) {
startPixivLoginProxyServer();
} else {
System.exit(1);
return;
}
}
}
ObjectInputStream ois = new ObjectInputStream(new FileInputStream(cookieStoreFile));
cookieStore = (CookieStore) ois.readObject();
@ -137,7 +139,9 @@ public class Main {
@Command
public static void botMode(@Argument(name = "args", force = false) String argsStr) {
new MiraiMain().init();
MiraiMain main = new MiraiMain();
main.init();
main.close();
}
@Command
@ -398,7 +402,7 @@ public class Main {
private static void saveCookieStoreToFile() throws IOException {
log.info("正在保存CookieStore...");
File outputFile = new File(System.getProperty("cgj.botDataDir"), "cookies.store");
if(!outputFile.exists() && !outputFile.delete() && !outputFile.createNewFile()){
if(!outputFile.exists() && !outputFile.createNewFile()){
log.error("保存CookieStore失败.");
return;
}
@ -423,23 +427,25 @@ public class Main {
proxyServerStartThread.setName("LoginProxyServerThread");
proxyServerStartThread.start();
//System.console().readLine();
Scanner scanner = new Scanner(System.in);
log.info("登录完成后, 使用\"done\"命令结束登录过程.");
while(true) {
if (scanner.nextLine().equalsIgnoreCase("done")) {
log.info("关闭PLPS服务器...");
proxyServer.close();
cookieStore = proxyServer.getCookieStore();
try {
log.info("正在保存CookieStore...");
saveCookieStoreToFile();
log.info("CookieStore保存完成.");
} catch (IOException e) {
log.error("CookieStore保存时发生异常, 本次CookieStore仅可在本次运行使用.", e);
try(Scanner scanner = new Scanner(System.in)) {
while(true) {
if (scanner.nextLine().equalsIgnoreCase("done")) {
log.info("关闭PLPS服务器...");
proxyServer.close();
cookieStore = proxyServer.getCookieStore();
try {
log.info("正在保存CookieStore...");
saveCookieStoreToFile();
log.info("CookieStore保存完成.");
} catch (IOException e) {
log.error("CookieStore保存时发生异常, 本次CookieStore仅可在本次运行使用.", e);
}
break;
} else {
log.warn("要结束登录过程, 请使用\"done\"命令.");
}
break;
} else {
log.warn("要结束登录过程, 请使用\"done\"命令.");
}
}
}

View File

@ -1,57 +0,0 @@
package net.lamgc.cgj.bot;
import net.lz1998.cq.robot.CoolQ;
import org.apache.http.client.methods.HttpGet;
import java.util.Random;
import java.util.Timer;
import java.util.TimerTask;
public class AutoArtworksSender {
private final CoolQ CQ;
private final ReceiveType receiveType;
private final long targetReceiveId;
private Timer timer = new Timer();
private TimerTask task = new TimerTask() {
@Override
public void run() {
HttpGet request = new HttpGet();
// https://api.imjad.cn/pixiv/v2/?type=tags
}
};
public AutoArtworksSender(CoolQ cq, ReceiveType receiveType, long receiveId) {
this.CQ = cq;
this.receiveType = receiveType;
this.targetReceiveId = receiveId;
}
public void reset(long time) {
if(time <= 0) {
timer.schedule(task, new Random().nextInt(10 * 60 * 60 * 1000) + 7200000L); //2H ~ 12H
} else {
timer.schedule(task, time);
}
}
public void sendMessage(String message, boolean auto_escape) {
switch (receiveType) {
case GROUP:
CQ.sendGroupMsg(targetReceiveId, message, auto_escape);
break;
case Discuss:
CQ.sendDiscussMsg(targetReceiveId, message, auto_escape);
break;
case PRIVATE:
CQ.sendPrivateMsg(targetReceiveId, message, auto_escape);
break;
}
}
public enum ReceiveType {
PRIVATE, GROUP, Discuss
}
}

View File

@ -111,6 +111,30 @@ public class BotAdminCommandProcess {
@Argument(name = "type", force = false, defaultValue = "ILLUST") String rankingContentType,
@Argument(name = "original", force = false, defaultValue = "false") boolean original
) {
if(minTime <= 0 || floatTime <= 0) {
return "时间不能为0或负数";
} else if(rankingStart <= 0 || rankingStop - rankingStart <= 0) {
return "排行榜范围选取错误!";
}
PixivURL.RankingContentType type;
PixivURL.RankingMode mode;
try {
type = PixivURL.RankingContentType.valueOf("TYPE_" + rankingContentType.toUpperCase());
} catch(IllegalArgumentException e) {
return "无效的排行榜类型参数!";
}
try {
mode = PixivURL.RankingMode.valueOf("MODE_" + rankingMode.toUpperCase());
} catch(IllegalArgumentException e) {
return "无效的排行榜模式参数!";
}
if(!type.isSupportedMode(mode)) {
return "不兼容的排行榜模式与类型!";
}
long group = groupId <= 0 ? fromGroup : groupId;
JsonObject setting = new JsonObject();
setting.addProperty(RANKING_SETTING_TIME_MIN, minTime);
@ -127,7 +151,8 @@ public class BotAdminCommandProcess {
removePushGroup(fromGroup, groupId);
}
log.info("正在增加Timer...(Setting: {})", setting);
log.info("群组 {} 新推送配置: {}", group, setting);
log.info("正在增加Timer...");
pushInfoMap.put(group, setting);
addPushTimer(group, setting);
return "已在 " + group + " 开启定时推送功能。";

View File

@ -2,20 +2,22 @@ package net.lamgc.cgj.bot;
import com.google.common.base.Strings;
import com.google.common.base.Throwables;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import com.google.gson.*;
import io.netty.handler.codec.http.HttpHeaderNames;
import net.lamgc.cgj.Main;
import net.lamgc.cgj.bot.cache.*;
import net.lamgc.cgj.bot.event.BotEventHandler;
import net.lamgc.cgj.bot.event.BufferMessageEvent;
import net.lamgc.cgj.bot.sort.PreLoadDataComparator;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivSearchBuilder;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.cgj.pixiv.PixivDownload.PageQuality;
import net.lamgc.cgj.pixiv.PixivURL.RankingContentType;
import net.lamgc.cgj.pixiv.PixivURL.RankingMode;
import net.lamgc.cgj.util.URLs;
import net.lamgc.utils.base.runner.Argument;
import net.lamgc.utils.base.runner.Command;
import net.lamgc.utils.event.EventExecutor;
import net.lz1998.cq.utils.CQCode;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;
@ -29,9 +31,6 @@ import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
@SuppressWarnings({"SynchronizationOnLocalVariableOrMethodParameter", "SameParameterValue"})
@ -46,29 +45,46 @@ public class BotCommandProcess {
.serializeNulls()
.create();
/* -------------------- 缓存 -------------------- */
private final static Hashtable<String, File> imageCache = new Hashtable<>();
private final static CacheStore<JsonElement> illustInfoCache = new JsonRedisCacheStore(BotEventHandler.redisServer, "illustInfo", gson);
private final static CacheStore<JsonElement> illustPreLoadDataCache = new HotDataCacheStore<>(
new JsonRedisCacheStore(BotEventHandler.redisServer, "illustPreLoadData", gson),
new LocalHashCacheStore<>(), 3600000, 900000);
private final static CacheStore<JsonElement> searchBodyCache = new JsonRedisCacheStore(BotEventHandler.redisServer, "searchBody", gson);
private final static CacheStore<List<JsonObject>> rankingCache = new JsonObjectRedisListCacheStore(BotEventHandler.redisServer, "ranking", gson);
private final static CacheStore<List<String>> pagesCache = new StringListRedisCacheStore(BotEventHandler.redisServer, "imagePages");
public final static CacheStore<JsonElement> reportStore = new JsonRedisCacheStore(BotEventHandler.redisServer, "report", gson);
/**
* 图片异步缓存执行器
* 作品信息缓存 - 不过期
*/
private final static EventExecutor imageCacheExecutor = new EventExecutor(new ThreadPoolExecutor(
Runtime.getRuntime().availableProcessors() >= 2 ? 2 : 1,
(int) Math.ceil(Runtime.getRuntime().availableProcessors() / 2F),
5L, TimeUnit.SECONDS,
new LinkedBlockingQueue<>(128),
new ThreadFactoryBuilder()
.setNameFormat("imageCacheThread-%d")
.build(),
new ThreadPoolExecutor.DiscardOldestPolicy()
));
private final static CacheStore<JsonElement> illustInfoCache =
new JsonRedisCacheStore(BotEventHandler.redisServer, "illustInfo", gson);
/**
* 作品信息预加载数据 - 有效期 2 小时, 本地缓存有效期1 ± 0.25
*/
private final static CacheStore<JsonElement> illustPreLoadDataCache =
CacheStoreUtils.hashLocalHotDataStore(
new JsonRedisCacheStore(BotEventHandler.redisServer, "illustPreLoadData", gson),
3600000, 900000);
/**
* 搜索内容缓存, 有效期 2 小时
*/
private final static CacheStore<JsonElement> searchBodyCache =
new JsonRedisCacheStore(BotEventHandler.redisServer, "searchBody", gson);
/**
* 排行榜缓存, 不过期
*/
private final static CacheStore<List<JsonObject>> rankingCache =
new JsonObjectRedisListCacheStore(BotEventHandler.redisServer, "ranking", gson);
/**
* 作品页面下载链接缓存 - 不过期
*/
private final static CacheStore<List<String>> pagesCache =
new StringListRedisCacheStore(BotEventHandler.redisServer, "imagePages");
/**
* 作品报告存储 - 不过期
*/
public final static CacheStore<JsonElement> reportStore =
new JsonRedisCacheStore(BotEventHandler.redisServer, "report", gson);
private final static RankingUpdateTimer updateTimer = new RankingUpdateTimer();
@ -77,12 +93,6 @@ public class BotCommandProcess {
SettingProperties.loadProperties();
try {
imageCacheExecutor.addHandler(new ImageCacheHandler());
} catch (IllegalAccessException e) {
e.printStackTrace();
}
updateTimer.schedule(null);
log.info("初始化完成.");
}
@ -103,7 +113,7 @@ public class BotCommandProcess {
helpStrBuilder.append("CGJ Bot使用指南").append("\n");
helpStrBuilder.append("使用方法:.cgj <命令> [参数...]").append("\n");
helpStrBuilder.append("例如查询作品信息功能:").append("\n");
helpStrBuilder.append(".cgj artwork -id 80846159").append("\n");
helpStrBuilder.append(".cgj info -id 80846159").append("\n");
helpStrBuilder.append("目前可用的命令:").append("\n");
helpStrBuilder.append("\t").append("ranking - 获取今天或指定日期排行榜的前10名作品").append("\n");
helpStrBuilder.append("\t\t").append("-date - 指定查询日期(年-月-日)").append("\n");
@ -129,6 +139,12 @@ public class BotCommandProcess {
return helpStrBuilder.toString();
}
/**
* 作品信息查询
* @param fromGroup 来源群(系统提供)
* @param illustId 作品Id
* @return 返回作品信息
*/
@Command(commandName = "info")
public static String artworkInfo(@Argument(name = "$fromGroup") long fromGroup, @Argument(name = "id") int illustId) {
if(illustId <= 0) {
@ -141,16 +157,18 @@ public class BotCommandProcess {
}
JsonObject illustPreLoadData = getIllustPreLoadData(illustId, false);
StringBuilder builder = new StringBuilder("---------------- 作品信息 ----------------\n");
builder.append("作品Id: ").append(illustId).append("\n");
builder.append("作品标题:").append(illustPreLoadData.get("illustTitle").getAsString()).append("\n");
builder.append("作者(作者Id)").append(illustPreLoadData.get("userName").getAsString())
.append("(").append(illustPreLoadData.get("userId").getAsInt()).append(")\n");
builder.append("点赞数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.LIKE.attrName).getAsInt()).append("\n");
builder.append("收藏数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.BOOKMARK.attrName).getAsInt()).append("\n");
builder.append("围观数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.VIEW.attrName).getAsInt()).append("\n");
builder.append("评论数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.COMMENT.attrName).getAsInt()).append("\n");
builder.append("数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.PAGE.attrName).getAsInt()).append("\n");
StringBuilder builder = new StringBuilder("色图姬帮你了解了这个作品信息\n");
builder.append("---------------- 作品信息 ----------------");
builder.append("\n作品Id: ").append(illustId);
builder.append("\n作品标题").append(illustPreLoadData.get("illustTitle").getAsString());
builder.append("\n作者(作者Id)").append(illustPreLoadData.get("userName").getAsString())
.append("(").append(illustPreLoadData.get("userId").getAsInt()).append(")");
builder.append("\n点赞数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.LIKE.attrName).getAsInt());
builder.append("\n收藏数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.BOOKMARK.attrName).getAsInt());
builder.append("\n围观数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.VIEW.attrName).getAsInt());
builder.append("\n评论数:").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.COMMENT.attrName).getAsInt());
builder.append("\n页数").append(illustPreLoadData.get(PreLoadDataComparator.Attribute.PAGE.attrName).getAsInt()).append("");
builder.append("\n作品链接").append(artworksLink(fromGroup, illustId)).append("\n");
builder.append("---------------- 作品图片 ----------------\n");
builder.append(getImageById(fromGroup, illustId, PixivDownload.PageQuality.REGULAR, 1)).append("\n");
builder.append("使用 \".cgj image -id ")
@ -164,10 +182,20 @@ public class BotCommandProcess {
return "尚未支持";
}
/**
* 排行榜命令
* @param fromGroup 来源群(系统提供)
* @param queryTime 查询时间, 格式: 年-月-日
* @param force 是否强制查询, 当主动提供的时间不在查询范围时, 是否强制查询, 仅系统可用
* @param contentMode 内容模式
* @param contentType 排行榜类型
* @return 返回排行榜信息
*/
@Command
public static String ranking(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(force = false, name = "date") Date queryTime,
@Argument(force = false, name = "force") boolean force,
@Argument(force = false, name = "mode", defaultValue = "DAILY") String contentMode,
@Argument(force = false, name = "type", defaultValue = "ILLUST") String contentType
) {
@ -184,7 +212,7 @@ public class BotCommandProcess {
}
queryDate = gregorianCalendar.getTime();
} else {
if(new Date().before(queryDate)) {
if(new Date().before(queryDate) && !force) {
log.warn("查询的日期过早, 无法查询排行榜.");
return "查询日期过早, 暂未更新指定日期的排行榜!";
}
@ -221,7 +249,7 @@ public class BotCommandProcess {
String itemLimitPropertyKey = "ranking.itemCountLimit";
try {
itemLimit = Integer.parseInt(SettingProperties
.getProperty(SettingProperties.GLOBAL, itemLimitPropertyKey, "10"));
.getProperty(fromGroup, itemLimitPropertyKey, "10"));
} catch(NumberFormatException e) {
log.warn("配置项 {} 的参数值格式有误!", itemLimitPropertyKey);
}
@ -230,7 +258,7 @@ public class BotCommandProcess {
String imageLimitPropertyKey = "ranking.imageCountLimit";
try {
imageLimit = Integer.parseInt(
SettingProperties.getProperty(SettingProperties.GLOBAL, imageLimitPropertyKey, "3"));
SettingProperties.getProperty(fromGroup, imageLimitPropertyKey, "3"));
} catch(NumberFormatException e) {
log.warn("配置项 {} 的参数值格式有误!", imageLimitPropertyKey);
}
@ -261,14 +289,31 @@ public class BotCommandProcess {
return resultBuilder.append("如查询当前时间获取到昨天时间,则今日排名榜未更新。\n如有不当作品可使用\".cgj report -id 作品id\"向色图姬反馈。").toString();
}
/**
* 查询指定作者的作品(尚未完成)
* @return 返回作者信息和部分作品
*/
@Command(commandName = "userArt")
public static String userArtworks() {
return "功能未完成";
}
@Command(commandName = "st")
public static String r18Image() {
BufferMessageEvent event = new BufferMessageEvent();
RandomRankingArtworksSender artworksSender =
new RandomRankingArtworksSender(event, 1, 200,
RankingMode.MODE_MALE,
RankingContentType.TYPE_ALL,
PageQuality.ORIGINAL);
artworksSender.send();
return event.getBufferMessage();
}
/**
* 搜索命令
* @param fromGroup 来源群(系统提供)
* @param content 搜索内容
* @param type 搜索类型
* @param area 搜索区域
@ -379,7 +424,7 @@ public class BotCommandProcess {
int limit = 8;
try {
limit = Integer.parseInt(SettingProperties.
getProperty(SettingProperties.GLOBAL, "search.itemCountLimit", "8"));
getProperty(fromGroup, "search.itemCountLimit", "8"));
} catch (Exception e) {
log.warn("参数转换异常!将使用默认值(" + limit + ")", e);
}
@ -408,7 +453,7 @@ public class BotCommandProcess {
StringBuilder builder = new StringBuilder("[");
illustObj.get("tags").getAsJsonArray().forEach(el -> builder.append(el.getAsString()).append(", "));
builder.replace(builder.length() - 2, builder.length(), "]");
log.debug("{} ({} / {})\n\t作品id: {}, \n\t作者名(作者id): {} ({}), \n\t作品标题: {}, \n\t作品Tags: {}, \n\t页数: {}, \n\t作品链接: {}",
log.debug("{} ({} / {})\n\t作品id: {}, \n\t作者名(作者id): {} ({}), \n\t作品标题: {}, \n\t作品Tags: {}, \n\t页数: {}, \n\t作品链接: {}",
searchArea.name(),
count,
illustsList.size(),
@ -432,10 +477,20 @@ public class BotCommandProcess {
continue;
}
result.append(searchArea.name()).append(" (").append(count).append(" / ").append(limit).append(")\n\t作品id: ").append(illustId)
JsonObject illustPreLoadData = getIllustPreLoadData(illustId, false);
result.append(searchArea.name()).append(" (").append(count).append(" / ")
.append(limit).append(")\n\t作品id: ").append(illustId)
.append(", \n\t作者名: ").append(illustObj.get("userName").getAsString())
.append("\n\t作品标题: ").append(illustObj.get("illustTitle").getAsString())
.append("\n\t作品页数: ").append(illustObj.get("pageCount").getAsInt())
.append("\n\t作品页数: ").append(illustObj.get("pageCount").getAsInt()).append("")
.append("\n\t点赞数")
.append(illustPreLoadData.get(PreLoadDataComparator.Attribute.LIKE.attrName).getAsInt())
.append("\n\t收藏数")
.append(illustPreLoadData.get(PreLoadDataComparator.Attribute.BOOKMARK.attrName).getAsInt())
.append("\n\t围观数")
.append(illustPreLoadData.get(PreLoadDataComparator.Attribute.VIEW.attrName).getAsInt())
.append("\n\t评论数")
.append(illustPreLoadData.get(PreLoadDataComparator.Attribute.COMMENT.attrName).getAsInt())
.append("\n").append(imageMsg).append("\n");
count++;
}
@ -446,11 +501,22 @@ public class BotCommandProcess {
return Strings.nullToEmpty(result.toString()) + "预览图片并非原图,使用“.cgj image -id 作品id”获取原图\n如有不当作品可使用\".cgj report -id 作品id\"向色图姬反馈。";
}
/**
* 获取作品页面的下载链接
* @param illustId 作品Id
* @param quality 画质类型
* @return 返回作品所有页面在Pixiv的下载链接(有防盗链, 考虑要不要设置镜像站)
*/
@Command(commandName = "pages")
public static String getPagesList(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "id") int illustId,
@Argument(name = "quality", force = false) PixivDownload.PageQuality quality) {
try {
if(isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) {
log.warn("来源群 {} 查询的作品Id {} 为R18作品, 根据配置设定, 屏蔽该作品.", fromGroup, illustId);
return "该作品已被封印!";
}
List<String> pagesList = PixivDownload.getIllustAllPageDownload(pixivDownload.getHttpClient(), pixivDownload.getCookieStore(), illustId, quality);
StringBuilder builder = new StringBuilder("作品ID ").append(illustId).append(" 共有").append(pagesList.size()).append("页:").append("\n");
int index = 0;
@ -464,6 +530,12 @@ public class BotCommandProcess {
}
}
/**
* 获取作品链接
* @param fromGroup 来源群(系统提供)
* @param illustId 作品Id
* @return 返回作品在Pixiv的链接
*/
@Command(commandName = "link")
public static String artworksLink(@Argument(name = "$fromGroup") long fromGroup, @Argument(name = "id") int illustId) {
try {
@ -483,6 +555,7 @@ public class BotCommandProcess {
/**
* 通过illustId获取作品图片
* @param fromGroup 来源群(系统提供)
* @param illustId 作品Id
* @param quality 图片质量
* @param pageIndex 指定页面索引, 从1开始
@ -495,25 +568,6 @@ public class BotCommandProcess {
@Argument(name = "quality", force = false) PixivDownload.PageQuality quality,
@Argument(name = "page", force = false, defaultValue = "1") int pageIndex) {
log.debug("IllustId: {}, Quality: {}, PageIndex: {}", illustId, quality.name(), pageIndex);
List<String> pagesList;
try {
pagesList = getIllustPages(illustId, quality, false);
} catch (IOException e) {
log.error("获取下载链接列表时发生异常", e);
return "发生网络异常,无法获取图片!";
}
if(log.isDebugEnabled()) {
StringBuilder logBuilder = new StringBuilder("作品Id {} 所有页面下载链接: \n");
AtomicInteger index = new AtomicInteger();
pagesList.forEach(item -> logBuilder.append(index.incrementAndGet()).append(". ").append(item).append("\n"));
log.debug(logBuilder.toString());
}
if (pagesList.size() < pageIndex || pageIndex <= 0) {
log.warn("指定的页数超出了总页数({} / {})", pageIndex, pagesList.size());
return "指定的页数超出了范围(总共 " + pagesList.size() + " 页)";
}
try {
if (isNoSafe(illustId, SettingProperties.getProperties(fromGroup), false)) {
@ -528,6 +582,26 @@ public class BotCommandProcess {
return "发生网络异常,无法获取图片!";
}
List<String> pagesList;
try {
pagesList = getIllustPages(illustId, quality, false);
} catch (IOException e) {
log.error("获取下载链接列表时发生异常", e);
return "发生网络异常,无法获取图片!";
}
if(log.isDebugEnabled()) {
StringBuilder logBuilder = new StringBuilder("作品Id " + illustId + " 所有页面下载链接: \n");
AtomicInteger index = new AtomicInteger();
pagesList.forEach(item -> logBuilder.append(index.incrementAndGet()).append(". ").append(item).append("\n"));
log.debug(logBuilder.toString());
}
if (pagesList.size() < pageIndex || pageIndex <= 0) {
log.warn("指定的页数超出了总页数({} / {})", pageIndex, pagesList.size());
return "指定的页数超出了范围(总共 " + pagesList.size() + " 页)";
}
String downloadLink = pagesList.get(pageIndex - 1);
String fileName = URLs.getResourceName(Strings.nullToEmpty(downloadLink));
File imageFile = new File(getImageStoreDir(), downloadLink.substring(downloadLink.lastIndexOf("/") + 1));
@ -552,12 +626,11 @@ public class BotCommandProcess {
}
}
ImageCacheObject taskObject = new ImageCacheObject(imageCache, illustId, downloadLink, imageFile);
try {
imageCacheExecutor.executorSync(taskObject);
ImageCacheStore.executeCacheRequest(new ImageCacheObject(imageCache, illustId, downloadLink, imageFile));
} catch (InterruptedException e) {
log.error("等待图片下载时发生中断", e);
return "图片获取失败!";
log.warn("图片缓存被中断", e);
return "(错误:图片获取超时)";
}
} else {
log.debug("图片 {} 缓存命中.", fileName);
@ -603,12 +676,18 @@ public class BotCommandProcess {
/**
* 举报某一作品
* @param fromGroup 来源群(系统提供)
* @param illustId 需要举报的作品id
* @param reason 举报原因
* @return 返回提示信息
*/
@Command
public static String report(@Argument(name = "$fromGroup") long fromGroup, @Argument(name = "$fromQQ") long fromQQ, @Argument(name = "id") int illustId, @Argument(name = "msg", force = false) String reason) {
public static String report(
@Argument(name = "$fromGroup") long fromGroup,
@Argument(name = "$fromQQ") long fromQQ,
@Argument(name = "id") int illustId,
@Argument(name = "msg", force = false) String reason
) {
log.warn("收到作品反馈(IllustId: {}, 原因: {})", illustId, reason);
JsonObject reportJson = new JsonObject();
reportJson.addProperty("illustId", illustId);

View File

@ -6,6 +6,7 @@ import net.lamgc.cgj.pixiv.PixivURL;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.text.SimpleDateFormat;
import java.time.LocalDate;
import java.util.Calendar;
import java.util.Date;
@ -26,21 +27,23 @@ public class RankingUpdateTimer {
Calendar cal = Calendar.getInstance();
cal.setTime(firstRunDate == null ? new Date() : firstRunDate);
LocalDate currentLocalDate = LocalDate.now();
if(cal.get(Calendar.DAY_OF_YEAR) <= currentLocalDate.getDayOfYear() && cal.get(Calendar.HOUR_OF_DAY) >= 12) {
if(cal.get(Calendar.DAY_OF_YEAR) <= currentLocalDate.getDayOfYear() &&
cal.get(Calendar.HOUR_OF_DAY) >= 11 && cal.get(Calendar.MINUTE) >= 30) {
cal.set(Calendar.DAY_OF_YEAR, currentLocalDate.getDayOfYear() + 1);
}
cal.set(Calendar.HOUR_OF_DAY, 12);
cal.set(Calendar.MINUTE, 0);
cal.set(Calendar.HOUR_OF_DAY, 11);
cal.set(Calendar.MINUTE, 30);
cal.set(Calendar.SECOND, 0);
cal.set(Calendar.MILLISECOND, 0);
log.warn("已设置排行榜定时更新, 首次运行时间: {}", cal.getTime());
long delay = cal.getTime().getTime() - (System.currentTimeMillis());
log.warn("已设置排行榜定时更新, 首次运行时间: {} ({}min)", cal.getTime(), delay / 1000 / 60);
timer.schedule(new TimerTask() {
@Override
public void run() {
now(null);
}
}, cal.getTime(), 86400000); // 1 Day
}, delay, 86400000); // 1 Day
}
public void now(Date queryDate) {
@ -51,13 +54,14 @@ public class RankingUpdateTimer {
LocalDate currentLocalDate = LocalDate.now();
if(calendar.get(Calendar.DAY_OF_YEAR) == currentLocalDate.getDayOfYear() ||
calendar.get(Calendar.DAY_OF_YEAR) == currentLocalDate.getDayOfYear() - 1) {
if(calendar.get(Calendar.HOUR_OF_DAY) < 12) {
if(calendar.get(Calendar.HOUR_OF_DAY) < 11) {
calendar.add(Calendar.DAY_OF_YEAR, -2);
} else {
calendar.add(Calendar.DAY_OF_YEAR, -1);
}
}
String dateStr = new SimpleDateFormat("yyyy-MM-dd").format(calendar.getTime());
log.info("正在获取 {} 期排行榜数据...", calendar.getTime());
for (PixivURL.RankingMode rankingMode : PixivURL.RankingMode.values()) {
for (PixivURL.RankingContentType contentType : PixivURL.RankingContentType.values()) {
@ -66,7 +70,7 @@ public class RankingUpdateTimer {
}
log.info("当前排行榜类型: {}.{}, 正在更新...", rankingMode.name(), contentType.name());
BotEventHandler.executeMessageEvent(new VirtualLoadMessageEvent(0,0,
".cgj ranking -type=" + contentType.name() + " -mode=" + rankingMode.name()));
".cgj ranking -type=" + contentType.name() + " -mode=" + rankingMode.name() + " -force -date " + dateStr));
log.info("排行榜 {}.{} 负载指令已投递.", rankingMode.name(), contentType.name());
}
}

View File

@ -10,7 +10,7 @@ import java.util.*;
public final class SettingProperties {
private final static Logger log = LoggerFactory.getLogger("SettingProperties");
private final static Logger log = LoggerFactory.getLogger(SettingProperties.class.getName());
private final static File globalPropFile = new File(getPropertiesDir(), "global.properties");
private final static Properties globalProp = new Properties();
@ -95,7 +95,7 @@ public final class SettingProperties {
private static void saveGroupProperties(Long groupId, Properties properties) throws IOException {
File groupPropFile = new File(getPropertiesDir(), "group." + groupId + ".properties");
if((!groupPropFile.exists() || !groupPropFile.isFile()) && (!groupPropFile.delete() || !groupPropFile.createNewFile())) {
if(!groupPropFile.exists() && !groupPropFile.createNewFile()) {
log.error("群组 {} 配置文件创建失败!", groupId);
return;
}
@ -122,7 +122,7 @@ public final class SettingProperties {
*/
private static void saveGlobalProperties() {
try {
if((!globalPropFile.exists() || !globalPropFile.isFile()) && (!globalPropFile.delete() || !globalPropFile.createNewFile())) {
if(!globalPropFile.exists() && !globalPropFile.createNewFile()) {
log.error("创建全局配置文件失败.");
return;
}
@ -158,7 +158,7 @@ public final class SettingProperties {
*/
private static File getPropertiesDir() {
File propDir = new File(System.getProperty("cgj.botDataDir"), "/setting/");
if((!propDir.exists() || !propDir.isDirectory()) && (!propDir.delete() || !propDir.mkdirs())) {
if(!propDir.exists() && !propDir.mkdirs()) {
log.warn("Setting文件夹创建失败!");
}
return propDir;
@ -188,10 +188,10 @@ public final class SettingProperties {
Objects.requireNonNull(key);
Properties targetProperties;
if(groupId <= 0) {
targetProperties = globalProp;
} else {
changeList.add(groupId);
targetProperties = getGroupProperties(groupId);
} else {
targetProperties = globalProp;
}
String lastValue = targetProperties.getProperty(key);
if(value != null) {

View File

@ -0,0 +1,52 @@
package net.lamgc.cgj.bot.cache;
import com.google.common.base.Throwables;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Set;
import java.util.Timer;
import java.util.TimerTask;
import java.util.concurrent.CopyOnWriteArraySet;
public class AutoCleanTimer extends TimerTask {
private final static Set<Cleanable> cleanSet = new CopyOnWriteArraySet<>();
private final static Timer cleanTimer = new Timer("Thread-AutoClean", true);
private final static Logger log = LoggerFactory.getLogger(AutoCleanTimer.class.getName());
static {
cleanTimer.schedule(new AutoCleanTimer(), 100L);
}
/**
* 增加需要定时执行清理的缓存库
* @param store 已实现Cleanable的对象
*/
public static void add(Cleanable store) {
cleanSet.add(store);
}
/**
* 移除已添加的缓存库
* @param store 需要从AutoCleanTimer移除的对象
*/
public static void remove(Cleanable store) {
cleanSet.remove(store);
}
private AutoCleanTimer() {}
@Override
public void run() {
cleanSet.forEach(cleanable -> {
try {
cleanable.clean();
} catch (Exception e) {
log.error("{} 执行清理动作时发生异常:\n{}", cleanable.toString(), Throwables.getStackTraceAsString(e));
}
});
}
}

View File

@ -0,0 +1,35 @@
package net.lamgc.cgj.bot.cache;
public final class CacheStoreUtils {
private CacheStoreUtils() {}
/**
* 将 CacheStore 转换成 基于 {@link LocalHashCacheStore} 的 {@link HotDataCacheStore}
* <p>通过该方法转换, 会自动启用 自动清理</p>
* @param cacheStore 上游缓存库
* @param expireTime 热点缓存最小有效期
* @param floatRange 缓存浮动最大范围
* @param <T> 缓存库数据类型
* @return 返回 {@link HotDataCacheStore}
*/
public static <T> CacheStore<T> hashLocalHotDataStore(CacheStore<T> cacheStore, long expireTime, int floatRange) {
return hashLocalHotDataStore(cacheStore, expireTime, floatRange, true);
}
/**
* 将 CacheStore 转换成 基于 {@link LocalHashCacheStore} 的 {@link HotDataCacheStore}
* @param cacheStore 上游缓存库
* @param expireTime 热点缓存最小有效期
* @param floatRange 缓存浮动最大范围
* @param autoClean 是否启用自动清理
* @param <T> 缓存库数据类型
* @return 返回 {@link HotDataCacheStore}
*/
public static <T> CacheStore<T> hashLocalHotDataStore(CacheStore<T> cacheStore,
long expireTime, int floatRange, boolean autoClean) {
return new HotDataCacheStore<>(cacheStore, new LocalHashCacheStore<>(), expireTime, floatRange, autoClean);
}
}

View File

@ -0,0 +1,10 @@
package net.lamgc.cgj.bot.cache;
/**
* 可清理接口, 实现该接口代表该类拥有清理动作.
*/
public interface Cleanable {
void clean() throws Exception;
}

View File

@ -10,7 +10,7 @@ import java.util.*;
* @param <T> 存储类型
* @author LamGC
*/
public class HotDataCacheStore<T> implements CacheStore<T> {
public class HotDataCacheStore<T> implements CacheStore<T>, Cleanable {
private final CacheStore<T> parent;
private final CacheStore<T> current;
@ -24,17 +24,22 @@ public class HotDataCacheStore<T> implements CacheStore<T> {
* @param parent 上级缓存存储库
* @param current 热点缓存存储库, 最好使用本地缓存(例如 {@linkplain LocalHashCacheStore LocalHashCacheStore})
* @param expireTime 本地缓存库的缓存项过期时间, 单位毫秒;
* 该时间并不是所有缓存项的最终过期时间, 还需要根据expireFloatRange的设定随机设置, 公式:
* {@code expireTime + new Random().nextInt(expireFloatRange)}
* 该时间并不是所有缓存项的最终过期时间, 还需要根据expireFloatRange的设定随机设置, 公式:
* {@code expireTime + new Random().nextInt(expireFloatRange)}
* @param expireFloatRange 过期时间的浮动范围(单位毫秒), 用于防止短时间内大量缓存项失效导致的缓存雪崩
* @param autoClean 是否交由{@link AutoCleanTimer}自动执行清理
*/
public HotDataCacheStore(CacheStore<T> parent, CacheStore<T> current, long expireTime, int expireFloatRange) {
public HotDataCacheStore(CacheStore<T> parent, CacheStore<T> current, long expireTime, int expireFloatRange, boolean autoClean) {
this.parent = parent;
this.current = current;
this.expireTime = expireTime;
this.expireFloatRange = expireFloatRange;
log.debug("HotDataCacheStore初始化完成. (Parent: {}, Current: {}, expireTime: {}, expireFloatRange: {})",
parent, current, expireTime, expireFloatRange);
if(autoClean) {
AutoCleanTimer.add(this);
}
log.debug("HotDataCacheStore初始化完成. (Parent: {}, Current: {}, expireTime: {}, expireFloatRange: {}, autoClean: {})",
parent, current, expireTime, expireFloatRange, autoClean);
}
@Override
@ -121,4 +126,13 @@ public class HotDataCacheStore<T> implements CacheStore<T> {
public boolean supportedList() {
return false;
}
@Override
public void clean() {
for(String key : this.current.keys()) {
if(current.exists(key)) {
current.remove(key);
}
}
}
}

View File

@ -28,9 +28,9 @@ public class ImageCacheHandler implements EventHandler {
private final static Set<ImageCacheObject> cacheQueue = Collections.synchronizedSet(new HashSet<>());
@SuppressWarnings("unused")
public void getImageToCache(ImageCacheObject event) {
public void getImageToCache(ImageCacheObject event) throws Exception {
if(cacheQueue.contains(event)) {
log.debug("图片 {} 已存在相同缓存任务, 跳过.", event.getStoreFile().getName());
log.warn("图片 {} 已存在相同缓存任务, 跳过.", event.getStoreFile().getName());
return;
} else {
cacheQueue.add(event);
@ -43,11 +43,11 @@ public class ImageCacheHandler implements EventHandler {
try {
if(!storeFile.exists() && !storeFile.createNewFile()) {
log.error("无法创建文件(Path: {})", storeFile.getAbsolutePath());
return;
throw new IOException("Failed to create file");
}
} catch (IOException e) {
log.error("无法创建文件(Path: {})", storeFile.getAbsolutePath());
e.printStackTrace();
throw e;
}
HttpGet request = new HttpGet(event.getDownloadLink());
@ -57,11 +57,11 @@ public class ImageCacheHandler implements EventHandler {
response = httpClient.execute(request);
} catch (IOException e) {
log.error("Http请求时发生异常", e);
return;
throw e;
}
if(response.getStatusLine().getStatusCode() != 200) {
log.warn("Http请求异常{}", response.getStatusLine());
return;
throw new IOException("Http Response Error: " + response.getStatusLine());
}
log.debug("正在下载...(Content-Length: {}KB)", response.getEntity().getContentLength() / 1024);
@ -69,7 +69,7 @@ public class ImageCacheHandler implements EventHandler {
IOUtils.copy(response.getEntity().getContent(), fos);
} catch (IOException e) {
log.error("下载图片时发生异常", e);
return;
throw e;
}
event.getImageCache().put(URLs.getResourceName(event.getDownloadLink()), storeFile);
} finally {

View File

@ -0,0 +1,106 @@
package net.lamgc.cgj.bot.cache;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Hashtable;
import java.util.Map;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicReference;
import java.util.concurrent.locks.ReentrantLock;
public final class ImageCacheStore {
private final static Logger log = LoggerFactory.getLogger(ImageCacheStore.class.getName());
private final static Map<ImageCacheObject, Task> cacheMap = new Hashtable<>();
private final static ThreadPoolExecutor imageCacheExecutor = new ThreadPoolExecutor(
4, 6,
30L,
TimeUnit.SECONDS,
new LinkedBlockingQueue<>(100),
new ThreadFactoryBuilder()
.setNameFormat("ImageCacheThread-%d")
.build()
);
private final static ImageCacheHandler handler = new ImageCacheHandler();
private ImageCacheStore() {}
/**
* 传递图片缓存任务, 并等待缓存完成.
* @param cacheObject 缓存任务组
*/
public static void executeCacheRequest(ImageCacheObject cacheObject) throws InterruptedException {
Task task = getTaskState(cacheObject);
if(task.taskState.get() == TaskState.COMPLETE) {
return;
}
boolean locked = false;
try {
if(task.taskState.get() == TaskState.COMPLETE) {
return;
}
task.lock.lock();
locked = true;
// 双重检查
if(task.taskState.get() == TaskState.COMPLETE) {
return;
}
// 置任务状态
task.taskState.set(TaskState.RUNNING);
try {
Throwable throwable = imageCacheExecutor.submit(() -> {
try {
handler.getImageToCache(cacheObject);
} catch (Throwable e) {
return e;
}
return null;
}).get();
if(throwable == null) {
task.taskState.set(TaskState.COMPLETE);
} else {
task.taskState.set(TaskState.ERROR);
}
} catch (ExecutionException e) {
log.error("执行图片缓存任务时发生异常", e);
}
} finally {
if(locked) {
task.lock.unlock();
}
}
}
private static Task getTaskState(ImageCacheObject cacheObject) {
if(!cacheMap.containsKey(cacheObject)) {
cacheMap.put(cacheObject, new Task());
}
return cacheMap.get(cacheObject);
}
/**
* 任务状态
*/
private enum TaskState {
READY, RUNNING, COMPLETE, ERROR
}
private static class Task {
public final ReentrantLock lock = new ReentrantLock(true);
public final AtomicReference<TaskState> taskState = new AtomicReference<>(TaskState.READY);
}
}

View File

@ -12,19 +12,45 @@ import java.util.concurrent.atomic.AtomicReference;
* 基于Hashtable的本地缓存库
* @param <T> 缓存类型
*/
public class LocalHashCacheStore<T> implements CacheStore<T> {
public class LocalHashCacheStore<T> implements CacheStore<T>, Cleanable {
private final Hashtable<String, CacheObject<T>> cache;
/**
* 构造一个基于Hashtable的本地缓存库
* @see Hashtable
*/
public LocalHashCacheStore() {
this(0);
}
/**
* 构造一个基于Hashtable的本地缓存库
* @param initialCapacity 初始容量
* @see Hashtable
*/
public LocalHashCacheStore(int initialCapacity) {
this(initialCapacity, 0F);
}
/**
* 构造一个基于Hashtable的本地缓存库
* @param initialCapacity 初始容量
* @param loadFactor 重载因子
* @see Hashtable
*/
public LocalHashCacheStore(int initialCapacity, float loadFactor) {
this(initialCapacity, loadFactor, false);
}
/**
* 构造一个基于Hashtable的本地缓存库
* @param initialCapacity 初始容量
* @param loadFactor 重载因子
* @param autoClean 是否自动清理
* @see Hashtable
*/
public LocalHashCacheStore(int initialCapacity, float loadFactor, boolean autoClean) {
if(initialCapacity != 0) {
if(loadFactor <= 0F) {
cache = new Hashtable<>(initialCapacity);
@ -34,6 +60,10 @@ public class LocalHashCacheStore<T> implements CacheStore<T> {
} else {
cache = new Hashtable<>();
}
if(autoClean) {
AutoCleanTimer.add(this);
}
}
@Override
@ -118,6 +148,15 @@ public class LocalHashCacheStore<T> implements CacheStore<T> {
return false;
}
@Override
public void clean() throws Exception {
Date currentDate = new Date();
cache.forEach((key, value) -> {
if(value.isExpire(currentDate)) {
cache.remove(key);
}
});
}
public static class CacheObject<T> implements Comparable<CacheObject<T>> {

View File

@ -13,7 +13,7 @@ import java.util.Set;
import java.util.function.Consumer;
import java.util.function.Function;
public abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
abstract class RedisPoolCacheStore<T> implements CacheStore<T> {
private final JedisPool jedisPool;
private final String keyPrefix;

View File

@ -41,6 +41,8 @@ public class BotEventHandler implements EventHandler {
private final static Logger log = LoggerFactory.getLogger(BotEventHandler.class.getName());
private final static Map<Long, AtomicBoolean> muteStateMap = new Hashtable<>();
/**
* 所有缓存共用的JedisPool
*/
@ -158,6 +160,9 @@ public class BotEventHandler implements EventHandler {
log.debug(event.toString());
if(!match(msg)) {
return;
} else if(isMute(event.getFromGroup())) {
log.debug("机器人已被禁言, 忽略请求.");
return;
}
Pattern pattern = Pattern.compile("/\\s*(\".+?\"|[^:\\s])+((\\s*:\\s*(\".+?\"|[^\\s])+)|)|(\".+?\"|[^\"\\s])+");
@ -219,12 +224,14 @@ public class BotEventHandler implements EventHandler {
}
}
long processTime = System.currentTimeMillis() - time;
if(Objects.requireNonNull(result) instanceof String) {
if(!Objects.isNull(result) && result instanceof String && !isMute(event.getFromGroup())) {
try {
event.sendMessage((String) result);
} catch (Exception e) {
log.error("发送消息时发生异常", e);
}
} else if(isMute(event.getFromGroup())) {
log.warn("命令反馈时机器人已被禁言, 跳过反馈.");
}
long totalTime = System.currentTimeMillis() - time;
log.info("命令反馈完成.(事件耗时: {}ms, P: {}%({}ms), R: {}%({}ms))", totalTime,
@ -241,4 +248,40 @@ public class BotEventHandler implements EventHandler {
return message.startsWith(COMMAND_PREFIX) || message.startsWith(ADMIN_COMMAND_PREFIX);
}
private static boolean isMute(long groupId) {
Boolean mute = isMute(groupId, false);
return mute != null && mute;
}
/**
* 查询某群是否被禁言.
* @param groupId 群组Id
* @param rawValue 是否返回原始值(当没有该群状态, 且本参数为true时, 将返回null)
* @return 返回状态值, 如无该群禁言记录且rawValue = true, 则返回null
*/
public static Boolean isMute(long groupId, boolean rawValue) {
if(groupId <= 0) {
return false;
}
AtomicBoolean state = muteStateMap.get(groupId);
if(state == null && rawValue) {
return null;
}
return state != null && state.get();
}
/**
* 设置机器人禁言状态.
* <p>设置该项可防止因机器人在禁言期间反馈请求导致被封号.</p>
* @param mute 如果被禁言, 传入true
*/
public static void setMuteState(long groupId, boolean mute) {
if(!muteStateMap.containsKey(groupId)) {
muteStateMap.put(groupId, new AtomicBoolean(mute));
} else {
muteStateMap.get(groupId).set(mute);
}
log.warn("群组 {} 机器人禁言状态已变更: {}", groupId, mute ? "已禁言" : "已解除");
}
}

View File

@ -0,0 +1,59 @@
package net.lamgc.cgj.bot.event;
import java.util.Objects;
public class BufferMessageEvent extends MessageEvent {
StringBuffer buffer = new StringBuffer();
public final MessageEvent parent;
/**
* 以空消息空Id生成BufferMessageEvent
*/
public BufferMessageEvent() {
super(0, 0, "");
parent = null;
}
/**
* 提供消息内容构造BufferMessageEvent
* @param message 传入的消息内容
*/
public BufferMessageEvent(String message) {
super(0, 0, message);
parent = null;
}
/**
* 使用事件构造BufferMessageEvent
* @param parentEvent 父级消息事件对象
*/
public BufferMessageEvent(MessageEvent parentEvent) {
super(parentEvent.getFromGroup(), parentEvent.getFromQQ(), parentEvent.getMessage());
parent = parentEvent;
}
@Override
public int sendMessage(String message) {
buffer.append(message);
return 0;
}
/**
* 当提供了父级消息事件时, 本方法调用父级消息事件对象的{@code getImageUrl(String)}, 如果没有, 返回{@code null}
*/
@Override
public String getImageUrl(String image) {
return Objects.isNull(this.parent) ? null : this.parent.getImageUrl(image);
}
/**
* 获取缓冲区消息内容
* @return 消息内容
*/
public String getBufferMessage() {
return buffer.toString();
}
}

View File

@ -12,7 +12,7 @@ public abstract class MessageEvent implements EventObject, MessageSender {
public MessageEvent(long fromGroup, long fromQQ, String message) {
this.fromGroup = fromGroup;
this.fromQQ = fromQQ;
this.message = message;
this.message = message.trim();
}
/**

View File

@ -6,9 +6,10 @@ import net.lamgc.cgj.bot.message.MessageSenderBuilder;
import net.lamgc.cgj.bot.framework.mirai.message.MiraiMessageSenderFactory;
import net.mamoe.mirai.Bot;
import net.mamoe.mirai.BotFactoryJvm;
import net.mamoe.mirai.event.events.BotMuteEvent;
import net.mamoe.mirai.event.events.BotUnmuteEvent;
import net.mamoe.mirai.japt.Events;
import net.mamoe.mirai.message.FriendMessage;
import net.mamoe.mirai.message.GroupMessage;
import net.mamoe.mirai.message.*;
import net.mamoe.mirai.utils.BotConfiguration;
import org.apache.commons.net.util.Base64;
import org.slf4j.Logger;
@ -42,18 +43,46 @@ public class MiraiMain implements Closeable {
return;
}
bot = BotFactoryJvm.newBot(Long.parseLong(botProperties.getProperty("bot.qq", "0")), Base64.decodeBase64(botProperties.getProperty("bot.password", "")), new BotConfiguration());
Events.subscribeAlways(GroupMessage.class, (msg) -> BotEventHandler.executeMessageEvent(new MiraiMessageEvent(msg)));
Events.subscribeAlways(FriendMessage.class, (msg) -> BotEventHandler.executeMessageEvent(new MiraiMessageEvent(msg)));
BotConfiguration configuration = new BotConfiguration();
configuration.setProtocol(BotConfiguration.MiraiProtocol.ANDROID_PAD);
bot = BotFactoryJvm.newBot(Long.parseLong(botProperties.getProperty("bot.qq", "0")), Base64.decodeBase64(botProperties.getProperty("bot.password", "")), configuration);
Events.subscribeAlways(GroupMessageEvent.class, this::executeMessageEvent);
Events.subscribeAlways(FriendMessageEvent.class, this::executeMessageEvent);
Events.subscribeAlways(BotMuteEvent.class,
event -> BotEventHandler.setMuteState(event.getGroup().getId(), true));
Events.subscribeAlways(BotUnmuteEvent.class,
event -> BotEventHandler.setMuteState(event.getGroup().getId(), false));
bot.login();
MessageSenderBuilder.setCurrentMessageSenderFactory(new MiraiMessageSenderFactory(bot));
BotEventHandler.preLoad();
bot.join();
}
/**
* 处理消息事件
* @param message 消息事件对象
*/
private void executeMessageEvent(MessageEvent message) {
if(message instanceof GroupMessageEvent) {
GroupMessageEvent GroupMessageEvent = (GroupMessageEvent) message;
if(BotEventHandler.isMute(GroupMessageEvent.getGroup().getId(), true) == null) {
BotEventHandler.setMuteState(GroupMessageEvent.getGroup().getId(),
((GroupMessageEvent) message).getGroup().getBotMuteRemaining() != 0);
}
}
BotEventHandler.executeMessageEvent(new MiraiMessageEvent(message));
}
/**
* 关闭机器人
*/
public void close() {
if(bot == null) {
return;
}
log.warn("正在关闭机器人...");
bot.close(null);
bot = null;
log.warn("机器人已关闭.");
}

View File

@ -1,29 +1,40 @@
package net.lamgc.cgj.bot.framework.mirai.message;
import net.lamgc.cgj.bot.event.MessageEvent;
import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.message.ContactMessage;
import net.mamoe.mirai.message.GroupMessage;
import net.mamoe.mirai.message.GroupMessageEvent;
import net.mamoe.mirai.message.MessageEvent;
import net.mamoe.mirai.message.data.MessageUtils;
import java.util.Objects;
public class MiraiMessageEvent extends MessageEvent {
public class MiraiMessageEvent extends net.lamgc.cgj.bot.event.MessageEvent {
private final ContactMessage messageObject;
private final MessageEvent messageObject;
private final MessageSender messageSender;
public MiraiMessageEvent(ContactMessage message) {
super(message instanceof GroupMessage ? ((GroupMessage) message).getGroup().getId() : 0,
message.getSender().getId(), message.getMessage().contentToString());
public MiraiMessageEvent(MessageEvent message) {
super(message instanceof GroupMessageEvent ? ((GroupMessageEvent) message).getGroup().getId() : 0,
message.getSender().getId(), getMessageBodyWithoutSource(message.getMessage().toString()));
this.messageObject = Objects.requireNonNull(message);
if(message instanceof GroupMessage) {
messageSender = new MiraiMessageSender(((GroupMessage) message).getGroup(), MessageSource.Group);
if(message instanceof GroupMessageEvent) {
messageSender = new MiraiMessageSender(((GroupMessageEvent) message).getGroup(), MessageSource.Group);
} else {
messageSender = new MiraiMessageSender(message.getSender(), MessageSource.Private);
}
}
/**
* 将ContactMessage获得的消息内容删除 Mirai:source 并返回.
* <p>该做法比较保守, 防止Mirai:source位置出现变动.</p>
* @param message ContactMessage的消息内容;
* @return 返回删除了Mirai:source的消息
*/
private static String getMessageBodyWithoutSource(String message) {
StringBuilder builder = new StringBuilder(message);
int startIndex = builder.indexOf("[mirai:source:");
int endIndex = builder.indexOf("]", startIndex) + 1;
return builder.delete(startIndex, endIndex).toString();
}
@Override

View File

@ -11,10 +11,7 @@ import net.lamgc.cgj.bot.message.MessageSender;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.Bot;
import net.mamoe.mirai.contact.Contact;
import net.mamoe.mirai.message.data.Image;
import net.mamoe.mirai.message.data.Message;
import net.mamoe.mirai.message.data.MessageChain;
import net.mamoe.mirai.message.data.MessageUtils;
import net.mamoe.mirai.message.data.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@ -33,7 +30,7 @@ public class MiraiMessageSender implements MessageSender {
private final static CacheStore<String> imageIdCache = new HotDataCacheStore<>(
new StringRedisCacheStore(BotEventHandler.redisServer, "mirai.imageId"),
new LocalHashCacheStore<>(),
5400000, 1800000);
5400000, 1800000, true);
/**
* 使用id构造发送器
@ -60,7 +57,7 @@ public class MiraiMessageSender implements MessageSender {
public int sendMessage(final String message) {
log.debug("处理前的消息内容:\n{}", message);
Message msgBody = processMessage(Objects.requireNonNull(message));
log.debug("处理后的消息内容(可能出现乱序的情况, 但实际上顺序是没问题的):\n{}", msgBody);
log.debug("处理后的消息内容(可能出现乱序的情况, 但实际上顺序是没问题的):\n{}", msgBody.contentToString());
member.sendMessage(msgBody);
return 0;
}
@ -117,6 +114,15 @@ public class MiraiMessageSender implements MessageSender {
} else {
return img;
}
case "face":
if(!code.containsParameter("id")) {
return MessageUtils.newChain("(无效的表情Id)");
}
int faceId = Integer.parseInt(code.getParameter("id"));
if(faceId <= 0) {
return MessageUtils.newChain("(无效的表情Id)");
}
return new Face(faceId);
default:
log.warn("解析到不支持的BotCode: {}", code);
return MessageUtils.newChain("(不支持的BotCode)");

View File

@ -5,6 +5,8 @@ import net.lamgc.cgj.bot.message.MessageSenderFactory;
import net.lamgc.cgj.bot.message.MessageSource;
import net.mamoe.mirai.Bot;
import java.util.Objects;
public class MiraiMessageSenderFactory implements MessageSenderFactory {
private final Bot bot;
@ -14,15 +16,11 @@ public class MiraiMessageSenderFactory implements MessageSenderFactory {
}
@Override
public MessageSender createMessageSender(MessageSource source, long id) throws Exception {
switch(source) {
case Group:
case Discuss:
return new MiraiMessageSender(bot.getGroup(id), source);
case Private:
return new MiraiMessageSender(bot.getFriend(id), source);
default:
throw new NoSuchFieldException(source.toString());
public MessageSender createMessageSender(MessageSource source, long id) {
Objects.requireNonNull(source);
if(id <= 0) {
throw new IllegalArgumentException("id cannot be 0 or negative: " + id);
}
return new MiraiMessageSender(bot, source, id);
}
}

View File

@ -99,7 +99,7 @@ public class PixivAccessProxyServer {
}*/
log.info("Response Cookie: " + value);
BasicClientCookie cookie = parseRawCookie(value);
cookieStore.addCookie(null);
cookieStore.addCookie(cookie);
});
httpResponse.headers().remove(HttpHeaderNames.SET_COOKIE);
super.afterResponse(clientChannel, proxyChannel, httpResponse, pipeline);

View File

@ -1,7 +1,6 @@
package net.lamgc.cgj.util;
import net.lamgc.cgj.pixiv.PixivDownload;
import net.lamgc.cgj.pixiv.PixivURL;
import net.lamgc.utils.base.runner.StringParameterParser;
public class PagesQualityParser implements StringParameterParser<PixivDownload.PageQuality> {