Lxcloud/core
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(ai): integrate new AI SDK and refactor AI services

Browse Source 
- Replaced @langchain/openai with @ai-sdk/openai for improved functionality.
- Updated AI service methods to utilize the new SDK, including getOpenAiProvider and getOpenAiModel.
- Refactored AI agent and deep reading services to implement new tool definitions and execution logic.
- Enhanced comment service to utilize the new AI model for content review and spam detection.
- Added new AI prompts for summarization and title generation.

Signed-off-by: Innei <tukon479@gmail.com>
This commit is contained in:
Innei
2025-07-10 22:11:08 +08:00
parent a4635519b8
commit 3e12044fce
13 changed files with 1336 additions and 1166 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
apps/core/package.json
View File

@@ -48,6 +48,7 @@
"changelog": true
},
"dependencies": {
"@ai-sdk/openai": "1.3.23",
"@algolia/client-search": "^4.22.1",
"@babel/core": "7.27.4",
"@babel/plugin-transform-modules-commonjs": "7.27.1",
@@ -59,7 +60,6 @@
"@innei/next-async": "0.3.0",
"@innei/pretty-logger-nestjs": "0.3.4",
"@keyv/redis": "4.4.0",
"@langchain/openai": "0.5.10",
"@mx-space/compiled": "workspace:*",
"@nestjs/cache-manager": "3.0.1",
"@nestjs/common": "11.1.3",
@@ -77,6 +77,7 @@
"@typegoose/auto-increment": "4.13.0",
"@typegoose/typegoose": "12.16.0",
"@types/jsonwebtoken": "9.0.9",
"ai": "4.3.17",
"algoliasearch": "4.24.0",
"axios": "^1.9.0",
"axios-retry": "4.5.0",
@@ -98,7 +99,6 @@
"jsonwebtoken": "9.0.2",
"jszip": "3.10.1",
"keyv": "5.3.4",
"langchain": "0.3.25",
"linkedom": "0.18.11",
"lodash": "^4.17.21",
"lru-cache": "11.1.0",
@@ -131,8 +131,6 @@
"zx-cjs": "7.0.7-0"
},
"devDependencies": {
"@langchain/core": "0.3.55",
"@langchain/langgraph": "0.2.72",
"@nestjs/cli": "11.0.7",
"@nestjs/schematics": "11.0.5",
"@nestjs/testing": "11.1.3",

148
apps/core/src/modules/ai/ai-agent/README.md Normal file
View File

@@ -0,0 +1,148 @@
# AI Agent API Documentation
## Overview
The AI Agent module provides a chat bot API that can answer questions about your blog content using AI. It supports both streaming and synchronous responses.
## Configuration
To enable the AI Agent, you need to configure the following in the admin panel:
1. Go to Settings > AI Settings
2. Set `openAiKey` - Your OpenAI API key
3. Set `openAiEndpoint` (optional) - Custom OpenAI endpoint if needed
4. Set `openAiPreferredModel` - Default: `gpt-4o-mini`
5. Enable `enableAIAgent` - Toggle to enable the AI Agent feature
## API Endpoints
### POST `/api/ai/agent/chat` (Streaming)
Chat with the AI agent about your blog content with streaming responses.
**Authentication**: Required (must be logged in)
**Request Body**:
```json
{
"message": "What are your latest blog posts?",
"context": [ // Optional conversation history
{
"role": "user",
"content": "Previous question"
},
{
"role": "assistant",
"content": "Previous answer"
}
]
}
```
**Response**: Server-Sent Events (SSE) stream
```
data: {"type":"text","text":"Here are the latest blog posts"}
data: {"type":"text","text":"..."}
data: {"type":"finish","finishReason":"stop"}
```
### POST `/api/ai/agent/chat/sync` (Non-Streaming)
Chat with the AI agent synchronously (waits for complete response).
**Authentication**: Required
**Request Body**: Same as streaming endpoint
**Response**:
```json
{
"message": "Here are the latest blog posts...",
"timestamp": "2024-01-01T12:00:00.000Z"
}
```
### GET `/api/ai/agent/status`
Check if the AI Agent is enabled and configured.
**Authentication**: Required
**Response**:
```json
{
"enabled": true,
"service": "AI Agent Chat"
}
```
## Available Tools
The AI Agent has access to the following tools to query your blog data:
- **Posts**: Get posts by ID, list posts, get latest post, get posts by category/tag
- **Notes**: Get notes by ID, list notes, get latest notes
- **Categories**: Get category info, list all categories
- **Tags**: Get tag summary, get posts by tag
- **Pages**: Get page by ID, list all pages
- **Says**: Get all says, get random say
- **Recently**: Get recent activities
- **Comments**: Get comments, get comments by content
## Example Usage
### Streaming Chat (JavaScript/TypeScript)
```javascript
const response = await fetch('http://localhost:2333/api/ai/agent/chat', {
method: 'POST',
headers: {
'Authorization': 'Bearer YOUR_TOKEN',
'Content-Type': 'application/json'
},
body: JSON.stringify({
message: 'What blog posts do you have about TypeScript?'
})
})
const reader = response.body.getReader()
const decoder = new TextDecoder()
while (true) {
const { done, value } = await reader.read()
if (done) break
const text = decoder.decode(value)
const lines = text.split('\n')
for (const line of lines) {
if (line.startsWith('data: ')) {
const data = JSON.parse(line.slice(6))
if (data.type === 'text') {
console.log(data.text)
}
}
}
}
```
### Synchronous Chat
```bash
curl -X POST http://localhost:2333/api/ai/agent/chat/sync \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"message": "What blog posts do you have about TypeScript?"
}'
```
### Check AI Agent Status
```bash
curl -X GET http://localhost:2333/api/ai/agent/status \
-H "Authorization: Bearer YOUR_TOKEN"
```
## Streaming Response Format
The streaming endpoint uses Server-Sent Events (SSE) format. Each message is prefixed with `data: ` and contains a JSON object:
- `{"type":"text","text":"..."}` - Partial text response
- `{"type":"tool-call","toolCall":{...}}` - Tool invocation information
- `{"type":"tool-result","toolResult":{...}}` - Tool execution result
- `{"type":"finish","finishReason":"stop"}` - Stream completion
## Error Codes
- `ErrorCodeEnum.AINotEnabled` - AI Agent is not enabled in the configuration
- Authentication errors - User must be logged in to use the AI Agent

44
apps/core/src/modules/ai/ai-agent/ai-agent.controller.ts Normal file
View File

@@ -0,0 +1,44 @@
import { Readable } from 'node:stream'
import { FastifyReply } from 'fastify'
import { Body, Get, Post, Res } from '@nestjs/common'
import { ApiController } from '~/common/decorators/api-controller.decorator'
import { Auth } from '~/common/decorators/auth.decorator'
import { BizException } from '~/common/exceptions/biz.exception'
import { ErrorCodeEnum } from '~/constants/error-code.constant'
import { ConfigsService } from '../../configs/configs.service'
import { ChatRequestDto, ChatResponseDto } from './ai-agent.dto'
import { AIAgentService } from './ai-agent.service'
@ApiController('ai/agent')
export class AIAgentController {
constructor(
private readonly aiAgentService: AIAgentService,
private readonly configService: ConfigsService,
) {}
@Post('chat')
@Auth()
async chat(
@Body() chatRequest: ChatRequestDto,
@Res() reply: FastifyReply,
): Promise<void> {
const result = await this.aiAgentService.chatWithTools(chatRequest.message)
// 使用 toDataStreamResponse 获取 SSE 格式的响应
const response = result.toDataStreamResponse()
// 设置响应头
response.headers.forEach((value, key) => {
reply.header(key, value)
})
// 将 Web Response 的 ReadableStream 转换为 Node.js Readable 流
if (response.body) {
const nodeStream = Readable.fromWeb(response.body as any)
return reply.send(nodeStream)
}
}
}

19
apps/core/src/modules/ai/ai-agent/ai-agent.dto.ts Normal file
View File

@@ -0,0 +1,19 @@
import { IsArray, IsNotEmpty, IsOptional, IsString } from 'class-validator'
export class ChatRequestDto {
@IsString()
@IsNotEmpty()
message: string
@IsArray()
@IsOptional()
context?: Array<{
role: 'user' | 'assistant'
content: string
}>
}
export class ChatResponseDto {
message: string
timestamp: Date
}

1093
apps/core/src/modules/ai/ai-agent/ai-agent.service.ts
View File

File diff suppressed because it is too large Load Diff

149
apps/core/src/modules/ai/ai-deep-reading/ai-deep-reading.service.ts
View File

@@ -1,9 +1,6 @@
import { generateText, tool } from 'ai'
import type { PagerDto } from '~/shared/dto/pager.dto'
import { JsonOutputToolsParser } from '@langchain/core/output_parsers/openai_tools'
import { DynamicStructuredTool } from '@langchain/core/tools'
import { GraphRecursionError } from '@langchain/langgraph'
import { createReactAgent } from '@langchain/langgraph/prebuilt'
import { z } from '@mx-space/compiled/zod'
import { Injectable, Logger } from '@nestjs/common'
import { OnEvent } from '@nestjs/event-emitter'
@@ -17,6 +14,7 @@ import { InjectModel } from '~/transformers/model.transformer'
import { md5 } from '~/utils/tool.util'
import { ConfigsService } from '../../configs/configs.service'
import { AI_PROMPTS } from '../ai.prompts'
import { AiService } from '../ai.service'
import { AIDeepReadingModel } from './ai-deep-reading.model'
@@ -51,9 +49,7 @@ export class AiDeepReadingService {
throw new BizException(ErrorCodeEnum.ContentNotFoundCantProcess)
}
const llm = await this.aiService.getOpenAiChain({
maxTokens: 8192,
})
const model = await this.aiService.getOpenAiModel()
const dataModel = {
keyPoints: [] as string[],
@@ -61,126 +57,60 @@ export class AiDeepReadingService {
content: '',
}
// 创建分析文章的工具
const tools = [
new DynamicStructuredTool({
name: 'deep_reading',
description: `获取深度阅读内容`,
schema: z.object({}),
func: async () => {
const llm = await this.aiService.getOpenAiChain({
maxTokens: 1024 * 10,
// 定义工具
const tools = {
deep_reading: tool({
description: '获取深度阅读内容',
parameters: z.object({}),
execute: async () => {
const { text } = await generateText({
model,
system: AI_PROMPTS.deepReading.deepReadingSystem,
prompt: AI_PROMPTS.deepReading.getDeepReadingPrompt(
article.document.text,
),
maxTokens: 10240,
})
const result = await llm
.bind({
tool_choice: {
type: 'function',
function: { name: 'deep_reading' },
},
tools: [
{
name: 'deep_reading',
type: 'function',
function: {
name: 'deep_reading',
parameters: {
type: 'object',
properties: {
content: {
type: 'string',
description: '深度阅读内容',
},
},
required: ['content'],
},
description: `创建一个全面的深度阅读Markdown文本，保持文章的原始结构但提供扩展的解释和见解。
内容应该：
1. 遵循原文的流程和主要论点
2. 包含原文的所有关键技术细节
3. 扩展未充分解释的复杂概念
4. 在需要的地方提供额外背景和解释
5. 保持文章的原始语调和语言风格
6. 使用适当的Markdown格式，包括标题、代码块、列表等
7. 输出的语言必须与原文的语言匹配`,
},
},
],
})
.pipe(new JsonOutputToolsParser())
.invoke([
{
content: `分析以下文章：${article.document.text}\n\n创建一个全面的深度阅读Markdown文本，保持文章的原始结构但提供扩展的解释和见解。`,
role: 'system',
},
])
.then((result: any[]) => {
const content = result[0]?.args?.content
dataModel.content = content
return content
})
return result
dataModel.content = text
return text
},
}),
new DynamicStructuredTool({
name: 'save_key_points',
save_key_points: tool({
description: '保存关键点到数据库',
schema: z.object({
parameters: z.object({
keyPoints: z.array(z.string()).describe('关键点数组'),
}),
func: async (data: { keyPoints: string[] }) => {
dataModel.keyPoints = data.keyPoints
execute: async ({ keyPoints }) => {
dataModel.keyPoints = keyPoints
return '关键点已保存'
},
}),
new DynamicStructuredTool({
name: 'save_critical_analysis',
save_critical_analysis: tool({
description: '保存批判性分析到数据库',
schema: z.object({
parameters: z.object({
criticalAnalysis: z.string().describe('批判性分析'),
}),
func: async (data: { criticalAnalysis: string }) => {
dataModel.criticalAnalysis = data.criticalAnalysis
execute: async ({ criticalAnalysis }) => {
dataModel.criticalAnalysis = criticalAnalysis
return '批判性分析已保存'
},
}),
]
// 创建系统提示模板
const systemPrompt = `你是一个专门进行文章深度阅读的AI助手，需要分析文章并提供详细的解读。
分析过程：
1. 首先提取文章关键点，然后使用 save_key_points 保存到数据库
2. 然后进行批判性分析，包括文章的优点、缺点和改进建议，然后使用 save_critical_analysis 保存到数据库
3. 最后使用 deep_reading 生成完整的深度阅读内容
4. 返回完整结果，包括关键点、批判性分析和深度阅读内容`
}
try {
// 创建LangGraph React Agent
const agent = createReactAgent({
llm,
// 使用Vercel AI SDK执行多步骤工具调用
await generateText({
model,
tools,
system: AI_PROMPTS.deepReading.systemPrompt,
prompt: AI_PROMPTS.deepReading.getUserPrompt(
article.document.title,
article.document.text,
),
maxSteps: 10,
})
await agent.invoke(
{
messages: [
{
role: 'system',
content: systemPrompt,
},
{
role: 'user',
content: `文章标题: ${article.document.title}\n文章内容: ${article.document.text}`,
},
],
},
{
recursionLimit: 10, // 相当于以前的maxIterations
},
)
return {
keyPoints: dataModel.keyPoints,
criticalAnalysis: dataModel.criticalAnalysis,
@@ -261,9 +191,12 @@ export class AiDeepReadingService {
} catch (error) {
console.error(error)
if (error instanceof GraphRecursionError) {
if (
error.message?.includes('limit reached') ||
error.message?.includes('maximum')
) {
this.logger.error(
`LangGraph recursion limit reached for article ${articleId}: ${error.message}`,
`AI processing iteration limit reached for article ${articleId}: ${error.message}`,
)
throw new BizException(
ErrorCodeEnum.AIException,

58
apps/core/src/modules/ai/ai-summary/ai-summary.service.ts
View File

@@ -1,7 +1,8 @@
import { generateObject } from 'ai'
import removeMdCodeblock from 'remove-md-codeblock'
import type { PagerDto } from '~/shared/dto/pager.dto'
import { JsonOutputToolsParser } from '@langchain/core/output_parsers/openai_tools'
import { z } from '@mx-space/compiled/zod'
import { Injectable, Logger } from '@nestjs/common'
import { OnEvent } from '@nestjs/event-emitter'
@@ -16,7 +17,8 @@ import { transformDataToPaginate } from '~/transformers/paginate.transformer'
import { md5 } from '~/utils/tool.util'
import { ConfigsService } from '../../configs/configs.service'
import { DEFAULT_SUMMARY_LANG, LANGUAGE_CODE_TO_NAME } from '../ai.constants'
import { DEFAULT_SUMMARY_LANG } from '../ai.constants'
import { AI_PROMPTS } from '../ai.prompts'
import { AiService } from '../ai.service'
import { AISummaryModel } from './ai-summary.model'
@@ -50,49 +52,29 @@ export class AiSummaryService {
throw new BizException(ErrorCodeEnum.AINotEnabled)
}
const openai = await this.aiService.getOpenAiChain()
const model = await this.aiService.getOpenAiModel()
const article = await this.databaseService.findGlobalById(articleId)
if (!article || article.type === CollectionRefTypes.Recently) {
throw new BizException(ErrorCodeEnum.ContentNotFoundCantProcess)
}
const parser = new JsonOutputToolsParser()
const { object } = await generateObject({
model,
schema: z.object({
summary: z
.string()
.describe(AI_PROMPTS.summary.getSummaryDescription(lang)),
}),
prompt: AI_PROMPTS.summary.getSummaryPrompt(
lang,
this.serializeText(article.document.text),
),
temperature: 0.5,
maxRetries: 2,
})
const runnable = openai
.bind({
tools: [
{
type: 'function',
function: {
name: 'extractor',
description: `Extract the summary of the input text in the ${LANGUAGE_CODE_TO_NAME[lang] || LANGUAGE_CODE_TO_NAME[DEFAULT_SUMMARY_LANG]}, and the length of the summary is less than 150 words.`,
parameters: {
type: 'object',
properties: {
summary: {
type: 'string',
description: `The summary of the input text in the ${LANGUAGE_CODE_TO_NAME[lang] || LANGUAGE_CODE_TO_NAME[DEFAULT_SUMMARY_LANG]}, and the length of the summary is less than 150 words.`,
},
},
required: ['summary'],
},
},
},
],
tool_choice: { type: 'function', function: { name: 'extractor' } },
})
.pipe(parser)
const result = (await runnable.invoke([
this.serializeText(article.document.text),
])) as any[]
if (result.length === 0) {
return {}
}
return result[0]?.args?.summary
return object.summary
}
async generateSummaryByOpenAI(articleId: string, lang: string) {
const {

151
apps/core/src/modules/ai/ai-writer/ai-writer.service.ts
View File

@@ -1,11 +1,9 @@
import type {
FunctionDefinition,
ToolDefinition,
} from '@langchain/core/language_models/base'
import { generateObject } from 'ai'
import { JsonOutputToolsParser } from '@langchain/core/output_parsers/openai_tools'
import { z } from '@mx-space/compiled/zod'
import { Injectable, Logger } from '@nestjs/common'
import { AI_PROMPTS } from '../ai.prompts'
import { AiService } from '../ai.service'
@Injectable()
@@ -15,78 +13,83 @@ export class AiWriterService {
this.logger = new Logger(AiWriterService.name)
}
async queryByFunctionSchema(
text: string,
parameters: FunctionDefinition['parameters'],
) {
const toolDefinition: ToolDefinition = {
type: 'function',
function: {
name: 'extractor',
description: 'Extracts fields from the input.',
parameters,
},
}
const model = await this.aiService.getOpenAiChain()
const parser = new JsonOutputToolsParser()
const runnable = model
.bind({
tools: [toolDefinition],
tool_choice: { type: 'function', function: { name: 'extractor' } },
})
.pipe(parser)
const result = (await runnable.invoke([text])) as any[]
if (result.length === 0) {
return {}
}
// Extract just the args object from the first tool call response
return result[0]?.args || {}
}
async generateTitleAndSlugByOpenAI(text: string) {
return this.queryByFunctionSchema(text, {
type: 'object',
properties: {
title: {
type: 'string',
description:
'Generate a concise, engaging title from the input text. The title should be in the same language as the input text and capture the main topic effectively.',
},
slug: {
type: 'string',
description:
'Create an SEO-friendly slug in English based on the title. The slug should be lowercase, use hyphens to separate words, contain only alphanumeric characters and hyphens, and include relevant keywords for better search engine ranking.',
},
lang: {
type: 'string',
description:
'Identify the natural language of the input text (e.g., "en", "zh", "es", "fr", etc.).',
},
keywords: {
type: 'array',
items: {
type: 'string',
},
description:
'Extract 3-5 relevant keywords or key phrases from the input text that represent its main topics.',
},
},
required: ['title', 'slug', 'lang', 'keywords'],
})
const model = await this.aiService.getOpenAiModel()
try {
const { object } = await generateObject({
model,
schema: z.object({
title: z
.string()
.describe(AI_PROMPTS.writer.titleAndSlug.schema.title),
slug: z.string().describe(AI_PROMPTS.writer.titleAndSlug.schema.slug),
lang: z.string().describe(AI_PROMPTS.writer.titleAndSlug.schema.lang),
keywords: z
.array(z.string())
.describe(AI_PROMPTS.writer.titleAndSlug.schema.keywords),
}),
prompt: AI_PROMPTS.writer.titleAndSlug.prompt(text),
temperature: 0.3, // Lower temperature for more consistent output
maxRetries: 2, // Allow retries on failure
})
return object
} catch (error) {
this.logger.error(
`Failed to generate title and slug: ${error.message}`,
error.stack,
)
// Fallback response if AI fails
const fallbackTitle =
text.slice(0, 50).trim() + (text.length > 50 ? '...' : '')
const fallbackSlug = fallbackTitle
.toLowerCase()
.replaceAll(/[^a-z0-9]+/g, '-')
.replaceAll(/^-+|-+$/g, '')
.slice(0, 50)
return {
title: fallbackTitle,
slug: fallbackSlug || 'untitled',
lang: 'en',
keywords: [],
}
}
}
async generateSlugByTitleViaOpenAI(title: string) {
return this.queryByFunctionSchema(title, {
type: 'object',
properties: {
slug: {
type: 'string',
description:
'An SEO-friendly slug in English based on the title. The slug should be lowercase, use hyphens to separate words, contain only alphanumeric characters and hyphens, and be concise while including relevant keywords from the title.',
},
},
required: ['slug'],
})
const model = await this.aiService.getOpenAiModel()
try {
const { object } = await generateObject({
model,
schema: z.object({
slug: z.string().describe(AI_PROMPTS.writer.slug.schema.slug),
}),
prompt: AI_PROMPTS.writer.slug.prompt(title),
temperature: 0.3, // Lower temperature for more consistent output
maxRetries: 2, // Allow retries on failure
})
return object
} catch (error) {
this.logger.error(
`Failed to generate slug from title: ${error.message}`,
error.stack,
)
// Fallback slug generation
const fallbackSlug = title
.toLowerCase()
.replaceAll(/[^a-z0-9]+/g, '-')
.replaceAll(/^-+|-+$/g, '')
.slice(0, 50)
return {
slug: fallbackSlug || 'untitled',
}
}
}
}

2
apps/core/src/modules/ai/ai.module.ts
View File

@@ -1,6 +1,7 @@
import { forwardRef, Module } from '@nestjs/common'
import { McpModule } from '../mcp/mcp.module'
import { AIAgentController } from './ai-agent/ai-agent.controller'
import { AIAgentService } from './ai-agent/ai-agent.service'
import { AiDeepReadingController } from './ai-deep-reading/ai-deep-reading.controller'
import { AiDeepReadingService } from './ai-deep-reading/ai-deep-reading.service'
@@ -23,6 +24,7 @@ import { AiService } from './ai.service'
AiSummaryController,
AiWriterController,
AiDeepReadingController,
AIAgentController,
],
exports: [AiService],
})

125
apps/core/src/modules/ai/ai.prompts.ts Normal file
View File

@@ -0,0 +1,125 @@
import { DEFAULT_SUMMARY_LANG, LANGUAGE_CODE_TO_NAME } from './ai.constants'
export const AI_PROMPTS = {
// AI Summary Prompts
summary: {
getSummaryPrompt: (lang: string, text: string) =>
`Extract the summary of the following text in the ${LANGUAGE_CODE_TO_NAME[lang] || LANGUAGE_CODE_TO_NAME[DEFAULT_SUMMARY_LANG]}, and the length of the summary is less than 150 words:\n\n${text}`,
getSummaryDescription: (lang: string) =>
`The summary of the input text in the ${LANGUAGE_CODE_TO_NAME[lang] || LANGUAGE_CODE_TO_NAME[DEFAULT_SUMMARY_LANG]}, and the length of the summary is less than 150 words.`,
},
// AI Agent Prompts
agent: {
systemPrompt: `你是一个可以访问博客数据库的智能助手。使用提供的工具来获取和分析数据。
当你需要回答用户问题时，请遵循以下步骤：
1. 分析用户的问题，确定需要获取什么数据
2. 使用合适的工具获取数据
3. 检查和分析获取的数据
4. 如需更多信息，继续使用工具获取
5. 根据所有收集到的数据提供完整回答
你可以查询的内容包括：
- 博客文章（posts）
- 笔记（notes）
- 分类（categories）
- 标签（tags）
- 自定义页面（pages）
- 说说/状态更新（says）
- 动态/活动（recently）
- 评论（comments）
不要编造信息，只使用通过工具获得的真实数据。`,
},
// AI Writer Prompts
writer: {
titleAndSlug: {
prompt: (text: string) =>
`Based on the following text content, generate a title, slug, language, and keywords.
Text content:
${text}
Please generate:
1. A concise, engaging title that captures the main topic
2. An SEO-friendly slug (lowercase, hyphens, alphanumeric only)
3. The language code of the text (e.g., "en" for English, "zh" for Chinese)
4. 3-5 relevant keywords that represent the main topics
Respond with a JSON object containing these fields.`,
schema: {
title:
'Generate a concise, engaging title from the input text. The title should be in the same language as the input text and capture the main topic effectively.',
slug: 'Create an SEO-friendly slug in English based on the title. The slug should be lowercase, use hyphens to separate words, contain only alphanumeric characters and hyphens, and include relevant keywords for better search engine ranking.',
lang: 'Identify the natural language of the input text (e.g., "en", "zh", "es", "fr", etc.).',
keywords:
'Extract 3-5 relevant keywords or key phrases from the input text that represent its main topics.',
},
},
slug: {
prompt: (title: string) =>
`Generate an SEO-friendly slug from the following title: "${title}"
The slug should:
- Be in lowercase
- Use hyphens to separate words
- Contain only alphanumeric characters and hyphens
- Be concise while including relevant keywords
- Be in English regardless of the title language
Respond with a JSON object containing the slug field.`,
schema: {
slug: 'An SEO-friendly slug in English based on the title. The slug should be lowercase, use hyphens to separate words, contain only alphanumeric characters and hyphens, and be concise while including relevant keywords from the title.',
},
},
},
// AI Deep Reading Prompts
deepReading: {
systemPrompt: `你是一个专门进行文章深度阅读的AI助手，需要分析文章并提供详细的解读。
分析过程：
1. 首先提取文章关键点，然后使用 save_key_points 保存到数据库
2. 然后进行批判性分析，包括文章的优点、缺点和改进建议，然后使用 save_critical_analysis 保存到数据库
3. 最后使用 deep_reading 生成完整的深度阅读内容
4. 返回完整结果，包括关键点、批判性分析和深度阅读内容`,
deepReadingSystem: `创建一个全面的深度阅读Markdown文本，保持文章的原始结构但提供扩展的解释和见解。
内容应该：
1. 遵循原文的流程和主要论点
2. 包含原文的所有关键技术细节
3. 扩展未充分解释的复杂概念
4. 在需要的地方提供额外背景和解释
5. 保持文章的原始语调和语言风格
6. 使用适当的Markdown格式，包括标题、代码块、列表等
7. 输出的语言必须与原文的语言匹配`,
getDeepReadingPrompt: (text: string) =>
`分析以下文章：${text}\n\n创建一个全面的深度阅读Markdown文本，保持文章的原始结构但提供扩展的解释和见解。`,
getUserPrompt: (title: string, text: string) =>
`文章标题: ${title}\n文章内容: ${text}`,
},
// Comment Review Prompts
comment: {
score: {
prompt: (text: string) =>
`分析以下评论是否包含不适当内容：${text}\n\n评估其是否包含垃圾信息、诈骗、广告、有毒内容及整体质量。`,
schema: {
score: '风险评分，1-10，越高越危险',
hasSensitiveContent: '是否包含政治敏感、色情、暴力或恐吓内容',
},
},
spam: {
prompt: (text: string) =>
`检查以下评论是否不适当：${text}\n\n分析其是否包含垃圾信息、广告、政治敏感内容、色情、暴力或低质量内容。`,
schema: {
isSpam: '是否为垃圾内容',
hasSensitiveContent: '是否包含政治敏感、色情、暴力或恐吓内容',
},
},
},
} as const

29
apps/core/src/modules/ai/ai.service.ts
View File

@@ -1,4 +1,4 @@
import { ChatOpenAI } from '@langchain/openai'
import { createOpenAI } from '@ai-sdk/openai'
import { Injectable } from '@nestjs/common'
import { BizException } from '~/common/exceptions/biz.exception'
@@ -10,26 +10,31 @@ import { ConfigsService } from '../configs/configs.service'
export class AiService {
constructor(private readonly configService: ConfigsService) {}
public async getOpenAiChain(options?: { maxTokens?: number }) {
public async getOpenAiProvider() {
const {
ai: { openAiKey, openAiEndpoint, openAiPreferredModel },
ai: { openAiKey, openAiEndpoint },
url: { webUrl },
} = await this.configService.waitForConfigReady()
if (!openAiKey) {
throw new BizException(ErrorCodeEnum.AINotEnabled, 'Key not found')
}
return new ChatOpenAI({
model: openAiPreferredModel,
return createOpenAI({
apiKey: openAiKey,
configuration: {
baseURL: openAiEndpoint || void 0,
defaultHeaders: {
'X-Title': 'Mix Space AI Client',
'HTTP-Referer': webUrl,
},
baseURL: openAiEndpoint || undefined,
headers: {
'X-Title': 'Mix Space AI Client',
'HTTP-Referer': webUrl,
},
maxTokens: options?.maxTokens,
})
}
public async getOpenAiModel() {
const {
ai: { openAiPreferredModel },
} = await this.configService.waitForConfigReady()
const provider = await this.getOpenAiProvider()
return provider(openAiPreferredModel)
}
}

110
apps/core/src/modules/comment/comment.service.ts
View File

@@ -1,4 +1,5 @@
import { URL } from 'node:url'
import { generateObject } from 'ai'
import { render } from 'ejs'
import { omit, pick } from 'lodash'
import { isObjectIdOrHexString, Types } from 'mongoose'
@@ -11,7 +12,7 @@ import type {
CommentModelRenderProps,
} from './comment.email.default'
import { JsonOutputToolsParser } from '@langchain/core/output_parsers/openai_tools'
import { z } from '@mx-space/compiled/zod'
import {
BadRequestException,
forwardRef,
@@ -34,6 +35,7 @@ import { InjectModel } from '~/transformers/model.transformer'
import { scheduleManager } from '~/utils/schedule.util'
import { getAvatar, hasChinese } from '~/utils/tool.util'
import { AI_PROMPTS } from '../ai/ai.prompts'
import { AiService } from '../ai/ai.service'
import { ConfigsService } from '../configs/configs.service'
import { ReaderModel } from '../reader/reader.model'
@@ -126,56 +128,28 @@ export class CommentService implements OnModuleInit {
aiReviewType: 'binary' | 'score',
aiReviewThreshold: number,
): Promise<boolean> {
const runnable = await this.aiService.getOpenAiChain()
const model = await this.aiService.getOpenAiModel()
// 评分模式
if (aiReviewType === 'score') {
const scorePrompt = {
content: `分析以下评论是否包含不适当内容：${text}\n\n评估其是否包含垃圾信息、诈骗、广告、有毒内容及整体质量。`,
role: 'user',
}
try {
const response = await runnable
.bind({
tools: [
{
type: 'function',
function: {
name: 'comment_review',
description: '分析评论内容并给出风险评分',
parameters: {
type: 'object',
properties: {
score: {
type: 'number',
description: '风险评分，1-10，越高越危险',
},
hasSensitiveContent: {
type: 'boolean',
description: '是否包含政治敏感、色情、暴力或恐吓内容',
},
},
required: ['score', 'hasSensitiveContent'],
},
},
},
],
})
.pipe(new JsonOutputToolsParser())
.invoke([scorePrompt])
if (!response) {
return false
}
const responseData = (response[0] as any)?.args
const { object } = await generateObject({
model,
schema: z.object({
score: z.number().describe(AI_PROMPTS.comment.score.schema.score),
hasSensitiveContent: z
.boolean()
.describe(AI_PROMPTS.comment.score.schema.hasSensitiveContent),
}),
prompt: AI_PROMPTS.comment.score.prompt(text),
})
// 如果包含敏感内容直接拒绝
if (responseData.hasSensitiveContent) {
if (object.hasSensitiveContent) {
return true
}
// 否则根据评分判断
return responseData.score > aiReviewThreshold
return object.score > aiReviewThreshold
} catch (error) {
this.logger.error('AI评审评分模式出错', error)
return false
@@ -183,52 +157,24 @@ export class CommentService implements OnModuleInit {
}
// 垃圾检测模式
else {
const spamPrompt = {
content: `检查以下评论是否不适当：${text}\n\n分析其是否包含垃圾信息、广告、政治敏感内容、色情、暴力或低质量内容。`,
role: 'user',
}
try {
const response = await runnable
.bind({
tools: [
{
type: 'function',
function: {
name: 'spam_check',
description: '检查评论是否为垃圾内容',
parameters: {
type: 'object',
properties: {
isSpam: {
type: 'boolean',
description: '是否为垃圾内容',
},
hasSensitiveContent: {
type: 'boolean',
description: '是否包含政治敏感、色情、暴力或恐吓内容',
},
},
required: ['isSpam', 'hasSensitiveContent'],
},
},
},
],
})
.pipe(new JsonOutputToolsParser())
.invoke([spamPrompt])
if (!response) {
return false
}
const responseData = (response[0] as any)?.args
const { object } = await generateObject({
model,
schema: z.object({
isSpam: z.boolean().describe(AI_PROMPTS.comment.spam.schema.isSpam),
hasSensitiveContent: z
.boolean()
.describe(AI_PROMPTS.comment.spam.schema.hasSensitiveContent),
}),
prompt: AI_PROMPTS.comment.spam.prompt(text),
})
// 如果包含敏感内容直接拒绝
if (responseData.hasSensitiveContent) {
if (object.hasSensitiveContent) {
return true
}
// 否则按照是否spam判断
return responseData.isSpam
return object.isSpam
} catch (error) {
this.logger.error('AI评审垃圾检测模式出错', error)
return false

568
pnpm-lock.yaml generated
View File

File diff suppressed because it is too large Load Diff