22 Commits

Author SHA1 Message Date
4295293a21 Align branding, prompts, and deployment tooling 2026-03-28 22:16:56 +08:00
4aa69650e8 feat: update openclaw workspace integration 2026-03-27 22:27:16 +08:00
5c08c1865c chore: sync current workspace changes 2026-03-27 11:27:26 +08:00
6ecc224427 feat: OpenClaw WebSocket integration with workspace file preview
- Migrate OpenClaw from HTTP (port 8004) to WebSocket (port 18789)
- Add workspace file list and content preview handlers
- Add OpenClawStatus component with agent/skills view
- Add OpenClawView panel in trader interface
- Add Zustand store for OpenClaw state management
- Fix gateway logging noise (yfinance, websockets)
- Fix RunWorkspaceManager.get_agent_asset_dir attribute error
- Handle missing workspace files gracefully in preview

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 11:08:15 +08:00
9bcc4221a4 refactor: remove mock trading functionality from backend and frontend
Removes all mock price simulation features:
- Delete MockPriceManager from backend/data/
- Remove mock_mode, enable_mock, is_mock_mode flags from services
- Remove mock CLI options and config
- Remove mock mode UI components and state from frontend
- Update tests to remove mock references

Now system supports only live and backtest modes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 13:38:51 +08:00
fecf8a9466 Fix agent workspace file handlers 2026-03-26 13:07:47 +08:00
86eb8c37a9 Fix empty dashboard fallback and websocket gating 2026-03-26 12:39:13 +08:00
1f9063edad Fix provider router import for news ingest 2026-03-26 11:29:14 +08:00
7e7a58769a feat: 添加新闻增量刷新和前端组件修复
- 新增 refresh_news_incremental/refresh_news_for_symbols 函数支持增量新闻获取
- 在 live cycle 中集成新闻刷新逻辑
- AgentFeed 支持 agentProfilesByAgent 显示模型信息
- StatisticsView 修复 stats 计算逻辑,使用 portfolioData 作为 fallback
- StockExplainView 修复 useEffect 依赖项问题
- AppShell/RoomView 传递 agentProfilesByAgent 属性
- start-dev.sh 调整日志级别为 warning 减少噪音

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 10:50:45 +08:00
16bb3c4211 docs(README): 更新项目说明文档,完善架构与启动指南
- 调整 CLAUDE.md 内容,增加使用指导和架构概览
- 补充微服务启动说明及单服务启动命令
- 明确 Gateway 服务器四阶段启动流程和职责划分
- 细化后端目录结构说明,补充主要文件职责描述
- 新增系统分层结构图,优化整体架构理解
- 更新 .gitignore,添加 runs 目录忽略规则
- 同步 .omc 相关状态文件,更新项目状态跟踪信息
2026-03-24 17:19:31 +08:00
da6d642aaa Migrate agent control reads and writes to REST 2026-03-24 16:30:36 +08:00
8d6c3c5647 Add restore-mode task launch flow 2026-03-24 15:27:35 +08:00
6413edf8c9 Refine runtime data flow and UI layering 2026-03-24 15:00:35 +08:00
c5eaf2b5ad Fix runtime logging and frontend app regressions 2026-03-24 10:58:41 +08:00
032c37538f fix(frontend): 添加缺失的 lastDayHistory 字段到 runtimeStore
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 20:15:27 +08:00
456748b01e fix(frontend): 修复 now 变量重复声明导致的空白页面
- 从 runtimeStore 移除 now(应从 uiStore 获取)
- 修复重复声明错误

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 18:12:32 +08:00
609b509446 feat(frontend): 完成 Zustand 状态管理迁移
- 将 App.jsx 中的 useState 迁移到 5 个 Zustand stores
- useRuntimeStore: 连接状态、运行时配置
- useMarketStore: 市场数据、股票价格
- usePortfolioStore: 组合、持仓、交易
- useAgentStore: Agent 技能,工作区
- useUIStore: UI 状态、视图切换
- 保留 tickers useState(需与 INITIAL_TICKERS 同步)
- 恢复 newsApi.js 和 tradingApi.js

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 18:03:23 +08:00
38102d0805 feat(backend): Agent 系统和 API 增强
- task_delegator: 完善团队任务分发逻辑
- runtime API: 增强运行时管理功能
- skills_manager: 技能管理改进
- tool_guard: 工具调用守卫优化
- evo_agent: 核心 Agent 改进

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 17:46:06 +08:00
3448667b79 feat: 微服务架构拆分和前后端优化
后端:
- 拆分出 agent_service, runtime_service, trading_service, news_service
- Gateway 模块化拆分 (gateway_*.py)
- 添加 domains/ 领域层
- 新增 control_client, runtime_client
- 更新 start-dev.sh 支持 split 服务模式

前端:
- 完善 API 服务层 (newsApi, tradingApi)
- 更新 vite.config.js
- Explain 组件优化

测试:
- 添加多个服务 app 测试

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 17:45:39 +08:00
0f1bc2bb39 feat(frontend): 添加 Zustand store 架构
- 创建 5 个领域 store:runtimeStore, marketStore, portfolioStore, agentStore, uiStore
- 更新 CLAUDE.md 记录架构改进
- Zustand 已安装但 stores 尚未在 App.jsx 中使用(渐进迁移)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 17:44:17 +08:00
06a23c32a4 refactor: Fix code quality issues identified in analysis
1. Rename factory.py's EvoAgent data class to AgentConfig
   - Avoids naming conflict with base/evo_agent.py's EvoAgent

2. Export pipeline_runner functions in backend/core/__init__.py
   - Add create_agents, create_long_term_memory, stop_gateway

3. Consolidate PromptLoader to singleton pattern
   - Add get_prompt_loader() singleton function
   - Update all usages to use singleton

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 01:07:53 +08:00
5b925fbe02 feat: Refactor services architecture and update project structure
- Remove Docker-based microservices (docker-compose.yml, Makefile, Dockerfiles)
- Update start-dev.sh to use backend.app:app entry point
- Add shared schema and client modules for service communication
- Add team coordination modules (messenger, registry, task_delegator, coordinator)
- Add evaluation hooks and skill adaptation hooks
- Add skill template and gateway server
- Update frontend WebSocket URL configuration
- Add explain components for insider and technical analysis

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 00:57:09 +08:00
214 changed files with 26758 additions and 8657 deletions

41
.env.example Normal file
View File

@@ -0,0 +1,41 @@
# Copy this file to `.env` for local development.
# Keep `.env` untracked and never paste real secrets into tracked files.
# ================== General Configuration | 通用配置 ==================
TICKERS=AAPL,MSFT,GOOGL,AMZN,NVDA,META,TSLA,AMD,NFLX,AVGO,PLTR,COIN
# Financial Data API
# At least `FINANCIAL_DATASETS_API_KEY` is required when using `FIN_DATA_SOURCE=financial_datasets`.
# `FINNHUB_API_KEY` is recommended for `FIN_DATA_SOURCE=finnhub` and required for live mode.
FIN_DATA_SOURCE=finnhub
ENABLED_DATA_SOURCES=financial_datasets,finnhub,yfinance,local_csv
FINANCIAL_DATASETS_API_KEY=
FINNHUB_API_KEY=
POLYGON_API_KEY=
MARKET_DB_PATH=
# Model API
OPENAI_API_KEY=
OPENAI_BASE_URL=
MODEL_NAME=qwen3-max-preview
EXPLAIN_ENRICH_USE_LLM=false
EXPLAIN_ENRICH_MODEL_PROVIDER=
EXPLAIN_ENRICH_MODEL_NAME=
EXPLAIN_RANGE_USE_LLM=
# Memory module
MEMORY_API_KEY=
# ================== Agent-Specific Model Configuration | Agent特定模型配置 ==================
AGENT_SENTIMENT_ANALYST_MODEL_NAME=deepseek-v3.2-exp
AGENT_TECHNICAL_ANALYST_MODEL_NAME=glm-4.6
AGENT_FUNDAMENTALS_ANALYST_MODEL_NAME=qwen3-max-preview
AGENT_VALUATION_ANALYST_MODEL_NAME=Moonshot-Kimi-K2-Instruct
AGENT_RISK_MANAGER_MODEL_NAME=qwen3-max-preview
AGENT_PORTFOLIO_MANAGER_MODEL_NAME=qwen3-max-preview
# ================== Advanced Configuration | 高阶配置 ==================
MAX_COMM_CYCLES=2
MARGIN_REQUIREMENT=0.5
DATA_START_DATE=2022-01-01
AUTO_UPDATE_DATA=true

6
.gitignore vendored
View File

@@ -51,13 +51,15 @@ node_modules
outputs/
/production/
/smoke_test/
/smoke_live_mock/
# Local tooling state
/.omc/
.omc/
/.pydeps/
/referance/
# Run outputs
/runs/
# Data files
backend/data/ret_data/

View File

@@ -1,7 +1,7 @@
{
"version": "1.0.0",
"lastScanned": 1773304964541,
"projectRoot": "/Users/cillin/workspeace/agentscope-samples/evotraders",
"lastScanned": 1774515151036,
"projectRoot": "/Users/cillin/workspeace/evotraders",
"techStack": {
"languages": [
{
@@ -40,7 +40,8 @@
"isMonorepo": false,
"workspaces": [],
"mainDirectories": [
"docs"
"docs",
"scripts"
],
"gitBranches": {
"defaultBranch": "main",
@@ -52,26 +53,54 @@
"backend": {
"path": "backend",
"purpose": null,
"fileCount": 3,
"lastAccessed": 1773304964533,
"fileCount": 4,
"lastAccessed": 1774515151025,
"keyFiles": [
"__init__.py",
"cli.py",
"gateway_server.py",
"main.py"
]
},
"backtest": {
"path": "backtest",
"purpose": null,
"fileCount": 0,
"lastAccessed": 1774515151026,
"keyFiles": []
},
"data": {
"path": "data",
"purpose": "Data files",
"fileCount": 3,
"lastAccessed": 1774515151027,
"keyFiles": [
"market_research.db",
"market_research.db-shm",
"market_research.db-wal"
]
},
"deploy": {
"path": "deploy",
"purpose": null,
"fileCount": 0,
"lastAccessed": 1774515151027,
"keyFiles": []
},
"docs": {
"path": "docs",
"purpose": "Documentation",
"fileCount": 0,
"lastAccessed": 1773304964533,
"keyFiles": []
"fileCount": 1,
"lastAccessed": 1774515151027,
"keyFiles": [
"compat-removal-plan.md"
]
},
"evotraders.egg-info": {
"path": "evotraders.egg-info",
"purpose": null,
"fileCount": 6,
"lastAccessed": 1773304964534,
"lastAccessed": 1774515151028,
"keyFiles": [
"PKG-INFO",
"SOURCES.txt",
@@ -83,8 +112,8 @@
"frontend": {
"path": "frontend",
"purpose": null,
"fileCount": 12,
"lastAccessed": 1773304964535,
"fileCount": 13,
"lastAccessed": 1774515151028,
"keyFiles": [
"README.md",
"components.json",
@@ -93,239 +122,414 @@
"index.css"
]
},
"live": {
"path": "live",
"purpose": null,
"fileCount": 0,
"lastAccessed": 1774515151028,
"keyFiles": []
},
"reference": {
"path": "reference",
"purpose": null,
"fileCount": 0,
"lastAccessed": 1774515151028,
"keyFiles": []
},
"runs": {
"path": "runs",
"purpose": null,
"fileCount": 0,
"lastAccessed": 1774515151029,
"keyFiles": []
},
"scripts": {
"path": "scripts",
"purpose": "Build/utility scripts",
"fileCount": 1,
"lastAccessed": 1774515151030,
"keyFiles": [
"run_prod.sh"
]
},
"services": {
"path": "services",
"purpose": "Business logic services",
"fileCount": 1,
"lastAccessed": 1774515151030,
"keyFiles": [
"README.md"
]
},
"shared": {
"path": "shared",
"purpose": null,
"fileCount": 0,
"lastAccessed": 1774515151030,
"keyFiles": []
},
"backend/api": {
"path": "backend/api",
"purpose": "API routes",
"fileCount": 5,
"lastAccessed": 1774515151030,
"keyFiles": [
"__init__.py",
"agents.py",
"guard.py"
]
},
"backend/config": {
"path": "backend/config",
"purpose": "Configuration files",
"fileCount": 4,
"lastAccessed": 1773304964535,
"fileCount": 6,
"lastAccessed": 1774515151030,
"keyFiles": [
"__init__.py",
"constants.py",
"data_config.py"
"agent_profiles.yaml",
"bootstrap_config.py"
]
},
"backend/data": {
"path": "backend/data",
"purpose": "Data files",
"fileCount": 7,
"lastAccessed": 1773304964536,
"fileCount": 12,
"lastAccessed": 1774515151031,
"keyFiles": [
"__init__.py",
"cache.py",
"historical_price_manager.py"
]
},
"backend/services": {
"path": "backend/services",
"purpose": "Business logic services",
"fileCount": 4,
"lastAccessed": 1773304964536,
"keyFiles": [
"__init__.py",
"gateway.py",
"market.py"
]
},
"backend/tests": {
"path": "backend/tests",
"purpose": "Test files",
"fileCount": 4,
"lastAccessed": 1773304964536,
"keyFiles": [
"__init__.py",
"test_agents.py",
"test_market_service.py"
]
},
"docs/assets": {
"path": "docs/assets",
"purpose": "Static assets",
"fileCount": 5,
"lastAccessed": 1773304964536,
"lastAccessed": 1774515151031,
"keyFiles": [
"dashboard.jpg",
"evotraders_demo.gif",
"evotraders_logo.jpg"
]
},
"frontend/public": {
"path": "frontend/public",
"purpose": "Public files",
"fileCount": 1,
"lastAccessed": 1773304964538,
"frontend/dist": {
"path": "frontend/dist",
"purpose": "Distribution/build output",
"fileCount": 2,
"lastAccessed": 1774515151031,
"keyFiles": [
"index.html",
"trading_logo.png"
]
},
"frontend/node_modules": {
"path": "frontend/node_modules",
"purpose": "Dependencies",
"fileCount": 1,
"lastAccessed": 1774515151036,
"keyFiles": []
}
},
"hotPaths": [
{
"path": "frontend/src/components/StatisticsView.jsx",
"accessCount": 22,
"lastAccessed": 1773310044545,
"path": "frontend/src/hooks/useWebSocketConnection.js",
"accessCount": 100,
"lastAccessed": 1774550862686,
"type": "file"
},
{
"path": "frontend/src/components/AgentCard.jsx",
"accessCount": 17,
"lastAccessed": 1773309995177,
"path": "backend/services/gateway.py",
"accessCount": 98,
"lastAccessed": 1774550272354,
"type": "file"
},
{
"path": "backend/services/gateway_openclaw_handlers.py",
"accessCount": 91,
"lastAccessed": 1774550256325,
"type": "file"
},
{
"path": "backend/api/openclaw.py",
"accessCount": 48,
"lastAccessed": 1774545375555,
"type": "file"
},
{
"path": "frontend/src/hooks/useOpenClawPanel.js",
"accessCount": 42,
"lastAccessed": 1774550688926,
"type": "file"
},
{
"path": "shared/client/openclaw_client.py",
"accessCount": 39,
"lastAccessed": 1774545484770,
"type": "file"
},
{
"path": "frontend/src",
"accessCount": 35,
"lastAccessed": 1774550715529,
"type": "directory"
},
{
"path": "reference/openclaw/src",
"accessCount": 33,
"lastAccessed": 1774550840611,
"type": "directory"
},
{
"path": "backend/services/openclaw_cli.py",
"accessCount": 31,
"lastAccessed": 1774545484887,
"type": "file"
},
{
"path": "frontend/src/components/TraderView.jsx",
"accessCount": 23,
"lastAccessed": 1774543366574,
"type": "file"
},
{
"path": "shared/models/openclaw.py",
"accessCount": 22,
"lastAccessed": 1774545419541,
"type": "file"
},
{
"path": "frontend/src/store/openclawStore.js",
"accessCount": 20,
"lastAccessed": 1774550319533,
"type": "file"
},
{
"path": "frontend/src/App.jsx",
"accessCount": 12,
"lastAccessed": 1773309849392,
"type": "file"
},
{
"path": "frontend/src/components/AgentFeed.jsx",
"accessCount": 12,
"lastAccessed": 1773309960022,
"type": "file"
},
{
"path": ".env",
"accessCount": 7,
"lastAccessed": 1773308950505,
"type": "file"
},
{
"path": "frontend/src/components/RoomView.jsx",
"accessCount": 7,
"lastAccessed": 1773309864236,
"type": "file"
},
{
"path": "backend/tools/analysis_tools.py",
"accessCount": 5,
"lastAccessed": 1773312271446,
"type": "file"
},
{
"path": "frontend/src/components/Header.jsx",
"accessCount": 4,
"lastAccessed": 1773309827069,
"type": "file"
},
{
"path": "frontend/src/components/AboutModal.jsx",
"accessCount": 4,
"lastAccessed": 1773310093371,
"type": "file"
},
{
"path": "backend/agents/prompts/analyst/personas.yaml",
"accessCount": 4,
"lastAccessed": 1773312049213,
"type": "file"
},
{
"path": "backend/agents/prompts/analyst/system.md",
"accessCount": 4,
"lastAccessed": 1773312049696,
"type": "file"
},
{
"path": "backend/agents/prompts/portfolio_manager/system.md",
"accessCount": 4,
"lastAccessed": 1773312050326,
"type": "file"
},
{
"path": "backend/agents/prompts/risk_manager/system.md",
"accessCount": 4,
"lastAccessed": 1773312050782,
"type": "file"
},
{
"path": "frontend/src/config/constants.js",
"accessCount": 3,
"lastAccessed": 1773309824671,
"type": "file"
},
{
"path": "frontend/src/components/RulesView.jsx",
"accessCount": 3,
"lastAccessed": 1773310061939,
"type": "file"
},
{
"path": "backend",
"accessCount": 3,
"lastAccessed": 1773312200721,
"type": "directory"
},
{
"path": "backend/services/gateway.py",
"accessCount": 2,
"lastAccessed": 1773312232905,
"type": "directory"
},
{
"path": "README.md",
"accessCount": 1,
"lastAccessed": 1773305013217,
"type": "file"
},
{
"path": "README_zh.md",
"accessCount": 1,
"lastAccessed": 1773305013274,
"type": "file"
},
{
"path": "env.template",
"accessCount": 1,
"lastAccessed": 1773305019965,
"accessCount": 18,
"lastAccessed": 1774544542524,
"type": "file"
},
{
"path": "frontend/src/services/websocket.js",
"accessCount": 1,
"lastAccessed": 1773309324302,
"accessCount": 18,
"lastAccessed": 1774549669596,
"type": "file"
},
{
"path": "backend/config/data_config.py",
"accessCount": 1,
"lastAccessed": 1773309324414,
"path": "start-dev.sh",
"accessCount": 15,
"lastAccessed": 1774548224246,
"type": "file"
},
{
"path": "backend/cli.py",
"accessCount": 1,
"lastAccessed": 1773309336899,
"path": "frontend/src/components/RuntimeView.jsx",
"accessCount": 14,
"lastAccessed": 1774518525793,
"type": "file"
},
{
"path": "frontend/src/components/AppShell.jsx",
"accessCount": 13,
"lastAccessed": 1774533781725,
"type": "file"
},
{
"path": "backend/main.py",
"accessCount": 13,
"lastAccessed": 1774548236340,
"type": "directory"
},
{
"path": "backend/agents/portfolio_manager.py",
"accessCount": 1,
"lastAccessed": 1773311956562,
"path": "backend/apps/openclaw_service.py",
"accessCount": 10,
"lastAccessed": 1774547900186,
"type": "file"
},
{
"path": "backend/agents/risk_manager.py",
"accessCount": 1,
"lastAccessed": 1773311956760,
"path": "frontend/src/components/OpenClawStatusPanel.jsx",
"accessCount": 8,
"lastAccessed": 1774533622019,
"type": "file"
},
{
"path": "backend/agents/analyst.py",
"accessCount": 1,
"lastAccessed": 1773311963222,
"type": "file"
},
{
"path": "backend/tools",
"accessCount": 1,
"lastAccessed": 1773312289643,
"path": "reference/openclaw/src/commands",
"accessCount": 7,
"lastAccessed": 1774530402019,
"type": "directory"
},
{
"path": "backend/tools/data_tools.py",
"accessCount": 1,
"lastAccessed": 1773312293851,
"path": "frontend/src/config/constants.js",
"accessCount": 7,
"lastAccessed": 1774544689658,
"type": "file"
},
{
"path": "",
"accessCount": 6,
"lastAccessed": 1774550700047,
"type": "directory"
},
{
"path": "backend/services",
"accessCount": 5,
"lastAccessed": 1774550692490,
"type": "directory"
},
{
"path": "frontend/src/store/uiStore.js",
"accessCount": 4,
"lastAccessed": 1774533747700,
"type": "file"
},
{
"path": "frontend/src/styles/GlobalStyles.jsx",
"accessCount": 4,
"lastAccessed": 1774533753657,
"type": "file"
},
{
"path": "frontend/src/store/agentStore.js",
"accessCount": 3,
"lastAccessed": 1774517930592,
"type": "file"
},
{
"path": "reference/openclaw/src/cli/skills-cli.ts",
"accessCount": 3,
"lastAccessed": 1774527140107,
"type": "file"
},
{
"path": "reference/openclaw/src/commands/agents.commands.list.ts",
"accessCount": 3,
"lastAccessed": 1774533427441,
"type": "file"
},
{
"path": "frontend/src/store/runtimeStore.js",
"accessCount": 2,
"lastAccessed": 1774517930660,
"type": "file"
},
{
"path": "frontend/src/hooks/useAgentWorkspacePanel.js",
"accessCount": 2,
"lastAccessed": 1774518021290,
"type": "file"
},
{
"path": "frontend/src/services/runtimeApi.js",
"accessCount": 2,
"lastAccessed": 1774518025465,
"type": "file"
},
{
"path": "reference/openclaw/src/commands/agents.commands.delete.ts",
"accessCount": 2,
"lastAccessed": 1774530389553,
"type": "file"
},
{
"path": "reference/openclaw/src/commands/agents.commands.add.ts",
"accessCount": 2,
"lastAccessed": 1774530389605,
"type": "file"
},
{
"path": "backend/api/__init__.py",
"accessCount": 2,
"lastAccessed": 1774542416191,
"type": "file"
},
{
"path": "frontend/vite.config.js",
"accessCount": 2,
"lastAccessed": 1774544772960,
"type": "file"
},
{
"path": "frontend/src/store/index.js",
"accessCount": 1,
"lastAccessed": 1774515811752,
"type": "file"
},
{
"path": "frontend/src/store/marketStore.js",
"accessCount": 1,
"lastAccessed": 1774515838923,
"type": "file"
},
{
"path": "frontend/src/store/portfolioStore.js",
"accessCount": 1,
"lastAccessed": 1774515839687,
"type": "file"
},
{
"path": "frontend/src/index.css",
"accessCount": 1,
"lastAccessed": 1774515988837,
"type": "file"
},
{
"path": "frontend/src/App.css",
"accessCount": 1,
"lastAccessed": 1774515998423,
"type": "file"
},
{
"path": "frontend/package.json",
"accessCount": 1,
"lastAccessed": 1774516005569,
"type": "file"
},
{
"path": "frontend/src/hooks/useAgentDataRequests.js",
"accessCount": 1,
"lastAccessed": 1774517930219,
"type": "file"
},
{
"path": "backend/services/gateway_admin_handlers.py",
"accessCount": 1,
"lastAccessed": 1774517937966,
"type": "file"
},
{
"path": "backend/apps/agent_service.py",
"accessCount": 1,
"lastAccessed": 1774517946208,
"type": "file"
},
{
"path": "frontend/src/hooks",
"accessCount": 1,
"lastAccessed": 1774517946260,
"type": "directory"
},
{
"path": "frontend/src/hooks/useFeedProcessor.js",
"accessCount": 1,
"lastAccessed": 1774517952115,
"type": "file"
},
{
"path": "reference/openclaw/src/commands/models/set.ts",
"accessCount": 1,
"lastAccessed": 1774526963526,
"type": "file"
},
{
"path": "reference/openclaw/src/commands/models/list.ts",
"accessCount": 1,
"lastAccessed": 1774526963632,
"type": "file"
},
{
"path": "reference/openclaw/src/cli/skills-cli.format.ts",
"accessCount": 1,
"lastAccessed": 1774526963684,
"type": "file"
}
],
"userDirectives": []

View File

@@ -1,6 +1,6 @@
{
"timestamp": "2026-03-12T20:33:59.497Z",
"timestamp": "2026-03-27T04:53:52.906Z",
"backgroundTasks": [],
"sessionStartTimestamp": "2026-03-12T14:19:33.615Z",
"sessionId": "73b0d597-0141-4873-9d0e-2b60e4e0635e"
"sessionStartTimestamp": "2026-03-27T04:53:21.944Z",
"sessionId": "cbb9004e-771b-4e82-95d4-cea6d9753642"
}

View File

@@ -1 +1 @@
{"session_id":"73b0d597-0141-4873-9d0e-2b60e4e0635e","transcript_path":"/Users/cillin/.claude/projects/-Users-cillin-workspeace-agentscope-samples-evotraders/73b0d597-0141-4873-9d0e-2b60e4e0635e.jsonl","cwd":"/Users/cillin/workspeace/agentscope-samples/evotraders","model":{"id":"kimi-for-coding","display_name":"kimi-for-coding"},"workspace":{"current_dir":"/Users/cillin/workspeace/agentscope-samples/evotraders","project_dir":"/Users/cillin/workspeace/agentscope-samples/evotraders","added_dirs":["/Users/cillin/workspeace/agentscope-samples/EvoTraders","/Users/cillin/workspeace/agentscope-samples/evotraders"]},"version":"2.1.63","output_style":{"name":"default"},"cost":{"total_cost_usd":6.822239999999999,"total_duration_ms":42679588,"total_api_duration_ms":1223637,"total_lines_added":275,"total_lines_removed":240},"context_window":{"total_input_tokens":654274,"total_output_tokens":27014,"context_window_size":200000,"current_usage":{"input_tokens":48465,"output_tokens":0,"cache_creation_input_tokens":0,"cache_read_input_tokens":0},"used_percentage":24,"remaining_percentage":76},"exceeds_200k_tokens":false}
{"session_id":"cbb9004e-771b-4e82-95d4-cea6d9753642","transcript_path":"/Users/cillin/.claude/projects/-Users-cillin-workspeace-evotraders/cbb9004e-771b-4e82-95d4-cea6d9753642.jsonl","cwd":"/Users/cillin/workspeace/evotraders","model":{"id":"MiniMax-M2.7-highspeed","display_name":"MiniMax-M2.7-highspeed"},"workspace":{"current_dir":"/Users/cillin/workspeace/evotraders","project_dir":"/Users/cillin/workspeace/evotraders","added_dirs":[]},"version":"2.1.78","output_style":{"name":"default"},"cost":{"total_cost_usd":0.660433,"total_duration_ms":168502,"total_api_duration_ms":37670,"total_lines_added":0,"total_lines_removed":0},"context_window":{"total_input_tokens":14416,"total_output_tokens":1705,"context_window_size":200000,"current_usage":{"input_tokens":461,"output_tokens":214,"cache_creation_input_tokens":0,"cache_read_input_tokens":53991},"used_percentage":27,"remaining_percentage":73},"exceeds_200k_tokens":false}

View File

@@ -1,3 +1,3 @@
{
"lastSentAt": "2026-03-12T20:31:37.362Z"
"lastSentAt": "2026-03-27T04:55:49.635Z"
}

View File

@@ -1,26 +0,0 @@
{
"agents": [
{
"agent_id": "a4090d26a45ac828d",
"agent_type": "oh-my-claudecode:executor",
"started_at": "2026-03-12T10:02:38.238Z",
"parent_mode": "none",
"status": "completed",
"completed_at": "2026-03-12T10:10:59.192Z",
"duration_ms": 500954
},
{
"agent_id": "af87583ef76a4df30",
"agent_type": "oh-my-claudecode:executor",
"started_at": "2026-03-12T10:40:04.409Z",
"parent_mode": "none",
"status": "completed",
"completed_at": "2026-03-12T10:41:17.387Z",
"duration_ms": 72978
}
],
"total_spawned": 2,
"total_completed": 2,
"total_failed": 0,
"last_updated": "2026-03-12T10:41:17.490Z"
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 144 KiB

376
CLAUDE.md Normal file
View File

@@ -0,0 +1,376 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
本文件为 Claude Code (claude.ai/code) 在此代码库中工作时提供指导。
## 项目概述
大时代 是一个自进化多智能体交易系统,由 6 个 AI Agent4 名分析师 + 投资经理 + 风控经理协作完成交易决策。Agent 基于 AgentScope 框架构建,配合 ReMe 记忆系统实现持续学习。
## 常用命令
### Backend (Python)
```bash
# 安装依赖
uv pip install -e .
# 运行命令
evotraders backtest --start 2025-11-01 --end 2025-12-01 # 回测模式
evotraders backtest --start 2025-11-01 --end 2025-12-01 --enable-memory # 带记忆回测
evotraders live # 实盘交易
evotraders live -t 22:30 # 定时每日交易
evotraders frontend # 启动可视化界面
# 开发服务器
./start-dev.sh # 启动全部 4 个微服务 (agent, runtime, trading, news)
# Gateway WebSocket 服务器
python backend/main.py --mode live --config-name live
# 单独启动微服务
python -m uvicorn backend.apps.runtime_service:app --host 0.0.0.0 --port 8003 --reload
python -m uvicorn backend.apps.agent_service:app --host 0.0.0.0 --port 8000 --reload
python -m uvicorn backend.apps.trading_service:app --host 0.0.0.0 --port 8001 --reload
python -m uvicorn backend.apps.news_service:app --host 0.0.0.0 --port 8002 --reload
# 测试
pytest backend/tests # 运行全部测试
pytest backend/tests/test_news_service_app.py -v # 运行单个测试
```
### Frontend (React)
```bash
cd frontend
npm run dev # Vite 开发服务器 (http://localhost:5173)
npm run build # 生产构建
npm run lint # ESLint 检查
npm run lint:fix # ESLint 自动修复
npm run test # Vitest 单元测试
```
## 架构概览
### 系统分层
```
┌─────────────────────────────────────────────────────────────┐
│ Frontend (React) │
│ WebSocket ws://localhost:8765 连接 Gateway │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ Gateway (backend/services/gateway.py) │
│ WebSocket 服务器,编排 Pipeline4 阶段启动 │
└─────────────────────────────────────────────────────────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐
│ Market │ │ Storage │ │ Pipeline │ │ Scheduler │
│ Service │ │ Service │ │ │ │ │
└────────────┘ └────────────┘ └────────────┘ └────────────┘
┌──────────────────────┼──────────────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Analysts │ │ PM │ │ Risk │
│ (4 个) │ │ │ │ Manager │
└──────────┘ └──────────┘ └──────────┘
```
### 微服务架构 (`backend/apps/`)
| 服务 | 端口 | 职责 |
|------|------|------|
| runtime_service | 8003 | 运行时配置、任务启动、Pipeline Runner |
| agent_service | 8000 | Agent 生命周期、工作区管理 |
| trading_service | 8001 | 市场数据、交易操作 |
| news_service | 8002 | 新闻、新闻富化、解释功能 |
### Gateway 4 阶段启动 (`backend/services/gateway.py`)
1. **WebSocket Server** - 前端立即可连接
2. **Market Service** - 价格数据开始推送
3. **Market Status Monitor** - 市场状态监控
4. **Scheduler** - 交易周期开始
### 运行时管理层 (`backend/runtime/`)
| 文件 | 职责 |
|------|------|
| `manager.py` | TradingRuntimeManager - 全局运行时管理器agent 注册、会话、事件快照 |
| `agent_runtime.py` | AgentRuntimeState - 单 agent 状态status、last_session |
| `context.py` | TradingRunContext - 运行上下文 |
| `session.py` | TradingSessionKey - 交易日会话键 |
| `registry.py` | RuntimeRegistry - agent 状态注册表 |
快照持久化到 `runs/<run_id>/state/runtime_state.json`
### Pipeline 执行 (`backend/core/`)
| 文件 | 职责 |
|------|------|
| `pipeline.py` | TradingPipeline - 核心编排器(分析→沟通→决策→执行→评估) |
| `pipeline_runner.py` | REST API 触发的独立执行5 阶段启动 |
| `scheduler.py` | BacktestScheduler、Scheduler - 回测/实盘调度 |
| `state_sync.py` | StateSync - 状态同步和广播 |
## 后端结构
```
backend/
├── agents/ # 多智能体实现
│ ├── analyst.py # AnalystAgent 基类
│ ├── portfolio_manager.py # PMAgent 投资经理
│ ├── risk_manager.py # RiskAgent 风控经理
│ ├── factory.py # Agent 实例工厂
│ ├── toolkit_factory.py # 工具集工厂
│ ├── skills_manager.py # 技能加载管理
│ ├── workspace_manager.py # 工作区管理
│ ├── skill_loader.py # 技能加载器
│ ├── agent_workspace.py # Agent 工作区
│ ├── prompt_loader.py # Prompt 加载器
│ ├── prompt_factory.py # Prompt 工厂
│ ├── skill_metadata.py # 技能元数据
│ ├── registry.py # Agent 注册表
│ ├── team_pipeline_config.py # 团队 Pipeline 配置
│ ├── compat.py # 兼容性层
│ ├── templates.py # 模板
│ ├── workspace.py # 工作区
│ ├── base/ # 核心类、Hooks
│ │ ├── evo_agent.py # 基于 AgentScope 的核心实现
│ │ └── hooks.py # 生命周期 Hooks
│ └── prompts/ # Agent 提示词
│ └── analyst/personas.yaml
├── apps/ # 微服务入口
│ ├── runtime_service.py # 运行时服务(端口 8003
│ ├── agent_service.py # Agent 服务(端口 8000
│ ├── trading_service.py # 交易服务(端口 8001
│ ├── news_service.py # 新闻服务(端口 8002
│ └── cors.py
├── runtime/ # 运行时管理层
│ ├── manager.py # TradingRuntimeManager
│ ├── agent_runtime.py # AgentRuntimeState
│ ├── context.py # TradingRunContext
│ ├── session.py # TradingSessionKey
│ └── registry.py # RuntimeRegistry
├── process/ # 进程监管层
│ ├── supervisor.py # ProcessSupervisor
│ ├── registry.py # RunRegistry
│ └── models.py # ProcessRun、ProcessRunState
├── core/ # Pipeline 执行
│ ├── pipeline.py # TradingPipeline核心编排器
│ ├── pipeline_runner.py # 独立 Pipeline 执行
│ ├── scheduler.py # 调度器
│ └── state_sync.py # 状态同步
├── services/ # Gateway 和服务
│ ├── gateway.py # WebSocket 网关
│ ├── gateway_*.py # Gateway 子模块
│ ├── market.py # 市场数据服务
│ ├── storage.py # 存储服务
│ ├── runtime_db.py # 运行时数据库
│ └── research_db.py # 研究数据库
├── data/ # 市场数据处理
│ ├── provider_router.py # 数据源路由
│ ├── provider_utils.py # 数据源工具
│ ├── market_store.py # 市场数据存储
│ ├── market_ingest.py # 数据采集
│ ├── cache.py # 缓存
│ ├── schema.py # 数据 schema
│ ├── historical_price_manager.py # 历史价格管理
│ ├── polling_price_manager.py # 轮询价格管理
│ ├── news_alignment.py # 新闻对齐
│ ├── polygon_client.py # Polygon.io 客户端
│ └── ret_data_updater.py # 离线数据更新
├── config/ # 配置
│ ├── constants.py # Agent 配置、显示名称
│ ├── bootstrap_config.py # 启动配置解析
│ ├── env_config.py # 环境变量配置
│ ├── data_config.py # 数据源配置
│ └── agent_profiles.yaml # Agent Profile 配置
├── domains/ # 领域业务逻辑
│ ├── news.py
│ └── trading.py
├── llm/ # LLM 集成
│ └── models.py # RetryChatModel、TokenRecordingModelWrapper
├── skills/ # 技能定义
├── tools/ # 交易和分析工具
├── enrich/ # LLM 响应富化
├── explain/ # 交易决策解释
├── utils/ # 工具函数
│ ├── settlement.py # 结算协调器
│ ├── trade_executor.py # 交易执行器
│ ├── terminal_dashboard.py # 终端仪表板
│ ├── analyst_tracker.py # 分析师追踪
│ ├── baselines.py # 基准线
│ ├── msg_adapter.py # 消息适配器
│ └── progress.py # 进度追踪
├── api/ # FastAPI 端点
│ └── runtime.py
└── main.py # 主入口点
```
## 前端结构
```
frontend/src/
├── App.jsx # 主应用LiveTradingApp
├── AppShell.jsx # App 外壳(布局、侧边栏)
├── components/
│ ├── RuntimeView.jsx # 交易运行时 UI
│ ├── TraderView.jsx # 交易员界面
│ ├── RoomView.jsx # 聊天室视图
│ ├── StockExplainView.jsx # 股票解释视图
│ ├── RuntimeSettingsPanel.jsx # 运行时设置面板
│ ├── RuntimeLogsModal.jsx # 运行时日志弹窗
│ ├── WatchlistPanel.jsx # 关注列表
│ ├── PerformanceView.jsx # 绩效视图
│ ├── StatisticsView.jsx # 统计视图
│ ├── NetValueChart.jsx # 净值曲线图
│ ├── AgentCard.jsx # Agent 卡片
│ ├── AgentFeed.jsx # Agent 动态
│ ├── Header.jsx # 头部
│ ├── MarkdownModal.jsx # Markdown 弹窗
│ ├── StockLogo.jsx # 股票 Logo
│ └── explain/ # 解释组件
│ ├── ExplainNewsSection.jsx
│ ├── ExplainRangeSection.jsx
│ ├── ExplainSimilarDaysSection.jsx
│ ├── ExplainStorySection.jsx
│ └── useExplainModel.js
├── hooks/ # React Hooks
│ ├── useWebSocketConnection.js # WebSocket 连接管理
│ ├── useRuntimeControls.js # 运行时配置管理
│ ├── useAgentDataRequests.js # Agent 数据请求
│ ├── useStockDataRequests.js # 股票数据请求
│ ├── useStockExplainData.js # 股票解释数据
│ ├── useAgentWorkspacePanel.js # Agent 工作区面板
│ ├── useWebsocketSessionSync.js # WebSocket 会话同步
│ └── useFeedProcessor.js # Feed 事件处理
├── store/ # Zustand 状态管理
│ ├── runtimeStore.js # 连接状态、运行时配置
│ ├── marketStore.js # 市场数据、股票价格
│ ├── portfolioStore.js # 组合、持仓、交易
│ ├── agentStore.js # Agent 技能、工作区
│ └── uiStore.js # UI 状态、视图切换
├── services/
│ ├── websocket.js # WebSocket 客户端
│ ├── runtimeApi.js # 运行时 API
│ ├── runtimeControls.js # 运行时控制
│ ├── newsApi.js # 新闻 API
│ └── tradingApi.js # 交易 API
├── utils/
│ ├── formatters.js # 格式化工具
│ └── modelIcons.js # 模型图标
└── config/
└── constants.js # Agent 定义、配置
```
## Agent 系统
### 6 种 Agent 角色
| 角色 ID | 名称 | 职责 |
|---------|------|------|
| `fundamentals_analyst` | 基本面分析师 | 财务健康、盈利能力、成长质量 |
| `technical_analyst` | 技术分析师 | 价格趋势、技术指标、动量分析 |
| `sentiment_analyst` | 情绪分析师 | 市场情绪、新闻情绪、内幕交易 |
| `valuation_analyst` | 估值分析师 | DCF、EV/EBITDA、intrinsic value |
| `portfolio_manager` | 投资经理 | 决策执行、交易协调 |
| `risk_manager` | 风控经理 | 实时价格/波动率监控、仓位限制 |
### 添加自定义分析师
1. `backend/agents/prompts/analyst/personas.yaml` 注册
2. `backend/config/constants.py``ANALYST_TYPES` 字典添加
3. `frontend/src/config/constants.js` 可选更新
### LLM 模型封装 (`backend/llm/models.py`)
- **RetryChatModel**: 自动重试瞬态 LLM 错误,指数退避
- **TokenRecordingModelWrapper**: 追踪 token 消耗和成本
## 技能系统 (`backend/skills/`)
技能定义在 `SKILL.md`,包含 `instructions``triggers``parameters``available_tools`
技能管理器支持 6 种作用域builtin、customized、installed、active、disabled、local。
## 运行时数据布局
- `data/market_research.db` - 持久研究数据
- `runs/<run_id>/` - 每次任务运行的状态
- `runs/<run_id>/team_dashboard/*.json` - 仪表板导出层(非权威源)
- `runs/<run_id>/state/runtime_state.json` - 运行时快照
- 运行时 API 优先使用 `server_state.json``runtime.db`
```bash
RUNS_RETENTION_COUNT=20 # 时间戳格式文件夹自动清理
```
## 环境配置
### Backend (`env.template`)
```bash
# 金融数据源支持多源fallback
FIN_DATA_SOURCE=finnhub|financial_datasets|yfinance|local_csv
ENABLED_DATA_SOURCES=financial_datasets,finnhub,yfinance,local_csv
FINANCIAL_DATASETS_API_KEY= # 回测必需
FINNHUB_API_KEY= # 实盘必需
POLYGON_API_KEY= # Polygon市场库采集可选
# LLM 配置
OPENAI_API_KEY=
OPENAI_BASE_URL=
MODEL_NAME=qwen3-max-preview
# Agent 特定模型
AGENT_SENTIMENT_ANALYST_MODEL_NAME=deepseek-v3.2-exp
AGENT_TECHNICAL_ANALYST_MODEL_NAME=glm-4.6
AGENT_FUNDAMENTALS_ANALYST_MODEL_NAME=qwen3-max-preview
AGENT_VALUATION_ANALYST_MODEL_NAME=Moonshot-Kimi-K2-Instruct
AGENT_RISK_MANAGER_MODEL_NAME=qwen3-max-preview
AGENT_PORTFOLIO_MANAGER_MODEL_NAME=qwen3-max-preview
# ReMe 记忆系统
MEMORY_API_KEY=
MEMORY_MODEL_NAME=qwen3-max
MEMORY_EMBEDDING_MODEL=text-embedding-v4
# 交易参数
MAX_COMM_CYCLES=2
MARGIN_REQUIREMENT=0.5
DATA_START_DATE=2022-01-01
AUTO_UPDATE_DATA=true
```
### Frontend (`frontend/env.template`)
```bash
VITE_WS_URL=ws://localhost:8765
```
## 关键依赖
- **AgentScope** - 多智能体框架
- **ReMe** - 持续学习记忆系统
- **FastAPI** + **uvicorn** - 后端 API
- **websockets** - 实时通信
- **React 19** + **Vite** + **TailwindCSS** - 前端
- **Zustand** - 状态管理

415
README.md
View File

@@ -1,36 +1,34 @@
<p align="center">
<img src="./docs/assets/evotraders_logo.jpg" width="45%">
<img src="./docs/assets/bigtime_logo.jpg" width="45%">
</p>
<h2 align="center">EvoTraders: A Self-Evolving Multi-Agent Trading System</h2>
<h2 align="center">大时代:自进化多智能体交易系统</h2>
<p align="center">
📌 <a href="http://trading.evoagents.cn">Visit us at EvoTraders website !</a>
📌 <a href="http://trading.evoagents.cn">Visit the 大时代 website</a>
</p>
![System Demo](./docs/assets/evotraders_demo.gif)
![System Demo](./docs/assets/bigtime_demo.gif)
EvoTraders is an open-source financial trading agent framework that builds a trading system capable of continuous learning and evolution in real markets through multi-agent collaboration and memory systems.
大时代 is an open-source financial trading agent framework that combines multi-agent collaboration, run-scoped workspaces, and memory to support both backtests and live trading workflows.
The repository name and CLI entrypoints still use `evotraders` for compatibility, but the product-facing branding now follows the 大时代 naming used by the reference branch.
---
## Core Features
**Multi-Agent Collaborative Trading**
A team of 6 members, including 4 specialized analyst roles (fundamentals, technical, sentiment, valuation) + portfolio manager + risk management, collaborating to make decisions like a real trading team.
**Multi-agent trading team**
Six roles collaborate like a real desk: four specialist analysts (fundamentals, technical, sentiment, valuation), one portfolio manager, and one risk manager.
You can customize your Agents here: [Custom Configuration](#custom-configuration)
**Continuous learning**
Agents can persist long-term memory with ReMe, reflect after each cycle, and evolve their decision patterns over time.
**Continuous Learning and Evolution**
Based on the ReMe memory framework, agents reflect and summarize after each trade, preserving experience across rounds, and forming unique investment methodologies.
**Backtest and live modes**
The same runtime model supports historical simulation and live execution with real-time market data.
Through this design, we hope that when AI Agents form a team and enter the real-time market, they will gradually develop their own trading styles and decision preferences, rather than one-time random inference.
**Real-Time Market Trading**
Supports real-time market data integration, providing backtesting mode and live trading mode, allowing AI Agents to learn and make decisions in real market fluctuations.
**Visualized Trading Information**
Observe agents' analysis processes, communication records, and decision evolution in real-time, with complete tracking of return curves and analyst performance.
**Operator-facing UI**
The frontend exposes the trading room, runtime controls, logs, approvals, agent workspaces, and explain/news views.
<p>
<img src="docs/assets/performance.jpg" width="45%">
@@ -39,198 +37,325 @@ Observe agents' analysis processes, communication records, and decision evolutio
---
## Current Architecture
The repository is currently in a transition from a modular monolith to split service surfaces. The split-service path is the default local development mode.
Current app surfaces:
- `backend.apps.agent_service` on `:8000`: control plane for workspaces, agents, skills, and guard/approval APIs
- `backend.apps.trading_service` on `:8001`: read-only trading data APIs
- `backend.apps.news_service` on `:8002`: read-only explain/news APIs
- `backend.apps.runtime_service` on `:8003`: runtime lifecycle APIs
- `backend.apps.openclaw_service` on `:8004`: read-only OpenClaw facade
- WebSocket gateway on `:8765`: live event/feed channel for the frontend
The most important runtime path today is:
`frontend -> runtime_service/control APIs -> gateway/runtime manager -> market service + pipeline + storage`
Reference notes for the migration live in [services/README.md](./services/README.md).
---
## Quick Start
### Installation
### 1. Install
```bash
# Clone repository
git clone https://github.com/agentscope-ai/agentscope-samples
cd agentscope-samples/EvoTraders
# clone this repository, then:
cd evotraders
# Install dependencies (Recommend uv!)
# backend runtime dependencies
uv pip install -r requirements.txt
# install package entrypoint in editable mode
uv pip install -e .
# optional: pip install -e .
# optional
# uv pip install -e ".[dev]"
# pip install -e .
```
# Configure environment variables
Frontend dependencies:
```bash
cd frontend
npm ci
cd ..
```
Production deployment should prefer `requirements.txt` for backend and `npm ci` for frontend so the pulled environment matches the checked-in lockfiles and version pins.
### 2. Configure environment
```bash
cp env.template .env
# Edit .env file and add your API Keys. The following config are required:
```
# finance data API: At minimum, FINANCIAL_DATASETS_API_KEY is required, corresponding to FIN_DATA_SOURCE=financial_datasets; It is recommended to add FINNHUB_API_KEY, corresponding to FIN_DATA_SOURCE=finnhub; If using live mode, FINNHUB_API_KEY must be added
FIN_DATA_SOURCE = #finnhub or financial_datasets
FINANCIAL_DATASETS_API_KEY= #Required
FINNHUB_API_KEY= #Optional
The root `env.template` is the canonical local template. A `.env.example` is also kept in the repo for reference.
# LLM API for Agents
Minimum useful variables:
```bash
# watchlist
TICKERS=AAPL,MSFT,GOOGL,NVDA,TSLA,META,AMZN
# market data
FIN_DATA_SOURCE=finnhub
FINANCIAL_DATASETS_API_KEY=
FINNHUB_API_KEY=
POLYGON_API_KEY=
# agent model
OPENAI_API_KEY=
OPENAI_BASE_URL=
MODEL_NAME=qwen3-max-preview
# LLM & embedding API for Memory
# memory (optional unless --enable-memory is used)
MEMORY_API_KEY=
```
### Running
Notes:
- `FINNHUB_API_KEY` is required for live mode.
- `POLYGON_API_KEY` enables long-lived market-store ingestion and refresh helpers.
- `MEMORY_API_KEY` is only required when long-term memory is enabled.
For a production-style local start flow, you can also use:
**Backtest Mode:**
```bash
evotraders backtest --start 2025-11-01 --end 2025-12-01
evotraders backtest --start 2025-11-01 --end 2025-12-01 --enable-memory # Use Memory
./start.sh
```
If you do not have market data APIs and just want to try the backtest demo, download the offline data and unzip it into `backend/data`:
### 3. Start the stack
Recommended local development flow:
```bash
./start-dev.sh
```
This starts:
- `agent_service` at `http://localhost:8000`
- `trading_service` at `http://localhost:8001`
- `news_service` at `http://localhost:8002`
- `runtime_service` at `http://localhost:8003`
- gateway WebSocket at `ws://localhost:8765`
Then start the frontend in another terminal:
```bash
evotraders frontend
```
Open `http://localhost:5173`.
You can also run services manually:
```bash
python -m uvicorn backend.apps.agent_service:app --host 0.0.0.0 --port 8000 --reload
python -m uvicorn backend.apps.trading_service:app --host 0.0.0.0 --port 8001 --reload
python -m uvicorn backend.apps.news_service:app --host 0.0.0.0 --port 8002 --reload
python -m uvicorn backend.apps.runtime_service:app --host 0.0.0.0 --port 8003 --reload
python -m backend.main --mode live --host 0.0.0.0 --port 8765
```
### 4. Run backtest or live mode from CLI
Backtest:
```bash
evotraders backtest --start 2025-11-01 --end 2025-12-01
evotraders backtest --start 2025-11-01 --end 2025-12-01 --enable-memory
evotraders backtest --config-name smoke_fullstack --start 2025-11-01 --end 2025-12-01
```
Live:
```bash
evotraders live
evotraders live --enable-memory
evotraders live --schedule-mode intraday --interval-minutes 60
evotraders live --trigger-time 22:30
```
Help:
```bash
evotraders --help
evotraders backtest --help
evotraders live --help
evotraders frontend --help
```
### Offline backtest data
If you want a quick backtest demo without external market APIs, download the offline bundle and unzip it into `backend/data`:
```bash
wget "https://agentscope-open.oss-cn-beijing.aliyuncs.com/ret_data.zip"
unzip ret_data.zip -d backend/data
```
The zip includes basic stock price data so you can run the backtest demo out of the box.
**Live Trading:**
```bash
evotraders live # Run immediately (default)
evotraders live --enable-memory # Use memory
evotraders live --mock # Mock mode (testing)
evotraders live -t 22:30 # Run daily at 22:30 local time (auto-converts to NYSE timezone)
```
**Get Help:**
```bash
evotraders --help # View global CLI help
evotraders backtest --help # View backtest mode parameters
evotraders live --help # View live/mock run parameters
```
**Launch Visualization Interface:**
```bash
# Ensure npm is installed, otherwise install it:
# npm install
evotraders frontend # Default connects to port 8765, you can modify the address in ./frontend/env.local to change the port number
```
Visit `http://localhost:5173/` to view the trading room, select a date and click Run/Replay to observe the decision-making process.
---
## System Architecture
## Runtime Data Layout
![Architecture Diagram](docs/assets/evotraders_pipeline.jpg)
- Long-lived research data lives in `data/market_research.db`
- Each run writes run-scoped state under `runs/<run_id>/`
- `runs/<run_id>/BOOTSTRAP.md` stores run-specific bootstrap values and prompt body
- `runs/<run_id>/state/runtime_state.json` stores runtime snapshot state
- `runs/<run_id>/team_dashboard/*.json` is a compatibility/export layer for dashboard consumers, not the primary runtime source of truth
### Agent Design
Optional retention control:
**Analyst Team:**
- **Fundamentals Analyst**: Financial health, profitability, growth quality
- **Technical Analyst**: Price trends, technical indicators, momentum analysis
- **Sentiment Analyst**: Market sentiment, news sentiment, insider trading
- **Valuation Analyst**: DCF, residual income, EV/EBITDA
**Decision Layer:**
- **Portfolio Manager**: Integrates analysis signals from analysts, executes communication strategies, combines analyst and team historical performance, recent investment memories, and long-term investment experience to make final decisions
- **Risk Management**: Real-time price and volatility monitoring, position limits, multi-layer risk warnings
### Decision Process
```
Real-time Market Data → Independent Analysis → Intelligent Communication (1v1/1vN/NvN) → Decision Execution → Performance Evaluation → Learning and Evolution (Memory Update)
```bash
RUNS_RETENTION_COUNT=20
```
Each trading day goes through five stages:
1. **Analysis Stage**: Each agent independently analyzes based on their respective tools and historical experience
2. **Communication Stage**: Exchange views through private chats, notifications, meetings, etc.
3. **Decision Stage**: Portfolio manager makes comprehensive judgments and provides final trades
4. **Evaluation Stage**
- **Performance Charts**: Track portfolio return curves vs. benchmark strategies (equal-weighted, market-cap weighted, momentum). Used to evaluate overall strategy effectiveness.
- **Analyst Rankings**: Click on avatars in the Trading Room to view analyst performance (win rate, bull/bear market win rate). Used to understand which analysts provide the most valuable insights.
- **Statistics**: Detailed position and trading history. Used for in-depth analysis of position management and execution quality.
5. **Review Stage**: Agents reflect on decisions and summarize experiences based on actual returns of the day, and store them in the ReMe memory framework for continuous improvement
Only timestamped run folders like `YYYYMMDD_HHMMSS` are pruned automatically. Named runs such as `live`, `smoke_fullstack`, or `reload_demo_*` are preserved.
---
### Module Support
## Frontend Service Routing
- **Agent Framework**: [AgentScope](https://github.com/agentscope-ai/agentscope)
- **Memory System**: [ReMe](https://github.com/agentscope-ai/reme)
- **LLM Support**: OpenAI, DeepSeek, Qwen, Moonshot, Zhipu AI, etc.
The frontend always uses the control plane and runtime APIs, and can optionally call split services directly for read-only data.
Useful frontend env vars:
```bash
VITE_CONTROL_API_BASE_URL=http://localhost:8000/api
VITE_RUNTIME_API_BASE_URL=http://localhost:8003/api/runtime
VITE_NEWS_SERVICE_URL=http://localhost:8002
VITE_TRADING_SERVICE_URL=http://localhost:8001
VITE_WS_URL=ws://localhost:8765
```
If these are not set, the frontend falls back to its local defaults and compatibility paths where available.
---
## Decision Flow
```text
Market data -> independent analyst work -> team communication -> portfolio decision ->
risk review -> execution/settlement -> reflection/memory update
```
The runtime manager also tracks:
- agent registration and status
- pending approvals
- run events
- current session key
---
## Custom Configuration
### Custom Analyst Roles
### Add or change analyst roles
1. Register role information in [./backend/agents/prompts/analyst/personas.yaml](./backend/agents/prompts/analyst/personas.yaml), for example:
1. Define the analyst persona in [backend/agents/prompts/analyst/personas.yaml](./backend/agents/prompts/analyst/personas.yaml)
2. Register the role in [backend/config/constants.py](./backend/config/constants.py)
3. Optionally add/update the frontend seat metadata in [frontend/src/config/constants.js](./frontend/src/config/constants.js)
Example persona entry:
```yaml
comprehensive_analyst:
name: "Comprehensive Analyst"
focus:
- ...
preferred_tools: # Flexibly select based on situation
- multi-factor synthesis
preferred_tools:
- get_stock_price
- get_company_financials
description: |
As a comprehensive analyst ...
A generalist analyst that combines multiple signals.
```
2. Add role definition in [./backend/config/constants.py](./backend/config/constants.py)
```python
ANALYST_TYPES = {
# Add new analyst
"comprehensive_analyst": {
"display_name": "Comprehensive Analyst",
"agent_id": "comprehensive_analyst",
"description": "Uses LLM to intelligently select analysis tools, performs comprehensive analysis",
"order": 15
}
}
```
### Configure per-agent models
3. Introduce new role in frontend configuration [./frontend/src/config/constants.js](./frontend/src/config/constants.js) (optional)
```javascript
export const AGENTS = [
// Override one of the agents
{
id: "comprehensive_analyst",
name: "Comprehensive Analyst",
role: "Comprehensive Analyst",
avatar: `${ASSET_BASE_URL}/...`,
colors: { bg: '#F9FDFF', text: '#1565C0', accent: '#1565C0' }
}
]
```
### Custom Models
Configure models used by different agents in the [.env](.env) file:
Model overrides are configured in `.env`:
```bash
AGENT_SENTIMENT_ANALYST_MODEL_NAME=qwen3-max-preview
AGENT_FUNDAMENTALS_ANALYST_MODEL_NAME=deepseek-chat
AGENT_TECHNICAL_ANALYST_MODEL_NAME=glm-4-plus
AGENT_VALUATION_ANALYST_MODEL_NAME=moonshot-v1-32k
AGENT_SENTIMENT_ANALYST_MODEL_NAME=deepseek-v3.2-exp
AGENT_TECHNICAL_ANALYST_MODEL_NAME=glm-4.6
AGENT_FUNDAMENTALS_ANALYST_MODEL_NAME=qwen3-max-preview
AGENT_VALUATION_ANALYST_MODEL_NAME=Moonshot-Kimi-K2-Instruct
AGENT_RISK_MANAGER_MODEL_NAME=qwen3-max-preview
AGENT_PORTFOLIO_MANAGER_MODEL_NAME=qwen3-max-preview
```
### Project Structure
### Run-scoped bootstrap config
Each run can override defaults through `runs/<run_id>/BOOTSTRAP.md`. The front matter is parsed by [backend/config/bootstrap_config.py](./backend/config/bootstrap_config.py) and can define values such as:
```yaml
tickers:
- AAPL
- MSFT
initial_cash: 100000
margin_requirement: 0.5
max_comm_cycles: 2
schedule_mode: daily
trigger_time: "09:30"
enable_memory: false
```
EvoTraders/
Initialize a run workspace with:
```bash
evotraders init-workspace --config-name my_run
```
---
## Project Structure
```text
evotraders/
├── backend/
│ ├── agents/ # Agent implementation
│ ├── communication/ # Communication system
│ ├── memory/ # Memory system (ReMe)
│ ├── tools/ # Analysis toolset
│ ├── servers/ # WebSocket services
── cli.py # CLI entry point
├── frontend/ # React visualization interface
── logs_and_memory/ # Logs and memory data
│ ├── agents/ # agent roles, prompts, skills, workspaces
│ ├── api/ # FastAPI routers
│ ├── apps/ # split service surfaces
│ ├── core/ # pipeline, scheduler, state sync
│ ├── runtime/ # runtime manager and agent runtime state
── services/ # gateway, market/storage/db services
│ └── cli.py # Typer CLI entrypoint
── frontend/ # React + Vite UI
├── shared/ # shared clients and schemas for split services
├── runs/ # run-scoped state and dashboards
├── data/ # long-lived research artifacts
└── services/README.md
```
---
## Testing
Backend tests live under `backend/tests` and cover service apps, shared clients, domains, routing, enrichment, gateway support, and runtime support.
Typical commands:
```bash
pytest
pytest backend/tests/test_runtime_service_app.py
pytest backend/tests/test_trading_service_app.py
```
Frontend tests:
```bash
cd frontend
npm test
```
---
## License and Disclaimer
EvoTraders is a research and educational project, open-sourced under the Apache 2.0 license.
大时代 is a research and educational project. Review the repository license before redistribution or commercial use.
**Risk Warning**: Before trading with real funds, please conduct thorough testing and risk assessment. Past performance does not guarantee future returns. Investment involves risks, and decisions should be made with caution.
**Risk warning**: this project is not investment advice. Test thoroughly before any real-money deployment. Past performance does not guarantee future returns.

View File

@@ -1,246 +1,359 @@
<p align="center">
<img src="./docs/assets/evotraders_logo.jpg" width="45%">
<img src="./docs/assets/bigtime_logo.jpg" width="45%">
</p>
<h2 align="center">EvoTraders:自进化多智能体交易系统</h2>
<h2 align="center">大时代:自进化多智能体交易系统</h2>
<p align="center">
📌 <a href="http://trading.evoagents.cn">Visit us at EvoTraders website !</a>
📌 <a href="http://trading.evoagents.cn">访问大时代官网</a>
</p>
![系统演示](./docs/assets/evotraders_demo.gif)
![系统演示](./docs/assets/bigtime_demo.gif)
EvoTraders是一个开源的金融交易智能体框架,通过多智能体协作和记忆系统,构建能够在真实市场中持续学习与进化的交易系统
大时代 是一个开源的金融交易智能体框架,结合多智能体协作、run 级工作区和记忆机制,支持回测与实盘两类交易运行模式
---
## 核心特性
**多智能体协作交易**
6名成员包含4种专业分析师角色(基本面、技术面、情绪、估值)+ 投资组合经理 + 风险管理,像真实交易团队一样协作决策
**多智能体交易团队**
系统默认包含 6 个角色4 个分析师(基本面、技术面、情绪、估值)+ 投资经理 + 风控经理
你可以在这里自定义你的Agents支持配置不同大模型如 Qwen、DeepSeek、GPT、Claude等协同分析[自定义配置](#自定义配置)
**持续学习**
可选接入 ReMe 长期记忆,智能体会在每轮结束后反思、复盘并沉淀经验。
**持续学习与进化**
基于 ReMe 记忆框架,智能体在每次交易后反思总结,跨回合保留经验,形成独特的投资方法论
通过这样的设计,我们希望当 AI Agents 组成团队进入实时市场,它们会逐渐形成自己的交易风格和决策偏好,而不是一次性的随机推理
**实时市场交易**
支持实时行情接入,提供回测模式和实盘模式,让 AI Agents 在真实市场波动中学习和决策。
**可视化交易信息**
实时观察 Agents 的分析过程、沟通记录和决策演化,完整追踪收益曲线和分析师表现。
**统一运行时**
同一套运行时模型支持历史回测和实时行情驱动的实盘流程
**可操作前端**
前端不只是展示层还包含交易室、运行控制、日志、审批、Agent 工作区和 explain/news 视图。
<p>
<img src="docs/assets/performance.jpg" width="45%">
<img src="./docs/assets/dashboard.jpg" width="45%">
</p>
---
## 当前架构
仓库目前处于“模块化单体 -> 拆分服务”的迁移阶段,本地开发默认走 split-service 路径。
当前 app surface
- `backend.apps.agent_service`,端口 `8000`:控制面,负责 workspaces、agents、skills、审批接口
- `backend.apps.trading_service`,端口 `8001`:只读交易数据接口
- `backend.apps.news_service`,端口 `8002`:只读 explain/news 接口
- `backend.apps.runtime_service`,端口 `8003`:运行时生命周期接口
- `backend.apps.openclaw_service`,端口 `8004`:只读 OpenClaw facade
- WebSocket gateway端口 `8765`:前端实时事件和 feed 通道
当前最关键的主链路是:
`frontend -> runtime_service/control APIs -> gateway/runtime manager -> market service + pipeline + storage`
迁移背景可参考 [services/README.md](./services/README.md)。
---
## 快速开始
### 安装
### 1. 安装
```bash
# 克隆仓库
git clone https://github.com/agentscope-ai/agentscope-samples
cd agentscope-samples/EvoTraders
# 克隆仓库后进入项目目录
cd evotraders
# 安装依赖(推荐使用uv
# 安装后端运行时依赖
uv pip install -r requirements.txt
# 安装项目入口(可编辑模式)
uv pip install -e .
# (可选pip install -e .
# 配置环境变量
# 可选
# uv pip install -e ".[dev]"
# pip install -e .
```
前端依赖:
```bash
cd frontend
npm ci
cd ..
```
生产环境部署建议后端使用 `requirements.txt`,前端使用 `npm ci`,这样拉起的环境会严格跟随仓库中锁定的依赖版本。
### 2. 配置环境变量
```bash
cp env.template .env
# 编辑 .env 文件,添加你的 API Keys,以下的配置项为必填项
```
# finance data API:至少需要FINANCIAL_DATASETS_API_KEY对应FIN_DATA_SOURCE=financial_datasets推荐添加FINNHUB_API_KEY对应至少需要FINANCIAL_DATASETS_API_KEY对应FIN_DATA_SOURCE填为finnhub;如果使用live 模式必须添加FINNHUB_API_KEY
FIN_DATA_SOURCE= #finnhub or financial_datasets
FINANCIAL_DATASETS_API_KEY= #必需
FINNHUB_API_KEY= #可选
根目录 `env.template` 是当前本地开发的主模板,仓库里也保留了 `.env.example` 作为参考。
# LLM API for Agents
最常用的配置项:
```bash
# 自选股
TICKERS=AAPL,MSFT,GOOGL,NVDA,TSLA,META,AMZN
# 行情数据
FIN_DATA_SOURCE=finnhub
FINANCIAL_DATASETS_API_KEY=
FINNHUB_API_KEY=
POLYGON_API_KEY=
# Agent 模型
OPENAI_API_KEY=
OPENAI_BASE_URL=
MODEL_NAME=qwen3-max-preview
# LLM & embedding API for Memory
# 长期记忆(只有启用 --enable-memory 才需要)
MEMORY_API_KEY=
```
### 运行
说明:
- live 模式必须配置 `FINNHUB_API_KEY`
- `POLYGON_API_KEY` 用于长期 market store 的补数和刷新
- `MEMORY_API_KEY` 仅在启用长期记忆时需要
如果要用更接近生产的本地启动方式,也可以直接执行:
**回测模式:**
```bash
evotraders backtest --start 2025-11-01 --end 2025-12-01
evotraders backtest --start 2025-11-01 --end 2025-12-01 --enable-memory # 使用记忆
./start.sh
```
如果没有可用的行情 API想快速体验回测 demo可直接下载离线数据并解压到 `backend/data`
### 3. 启动服务栈
本地开发推荐直接使用:
```bash
./start-dev.sh
```
该脚本会启动:
- `agent_service``http://localhost:8000`
- `trading_service``http://localhost:8001`
- `news_service``http://localhost:8002`
- `runtime_service``http://localhost:8003`
- gateway WebSocket`ws://localhost:8765`
然后在另一个终端启动前端:
```bash
evotraders frontend
```
访问 `http://localhost:5173`
也可以手动分别启动:
```bash
python -m uvicorn backend.apps.agent_service:app --host 0.0.0.0 --port 8000 --reload
python -m uvicorn backend.apps.trading_service:app --host 0.0.0.0 --port 8001 --reload
python -m uvicorn backend.apps.news_service:app --host 0.0.0.0 --port 8002 --reload
python -m uvicorn backend.apps.runtime_service:app --host 0.0.0.0 --port 8003 --reload
python -m backend.main --mode live --host 0.0.0.0 --port 8765
```
### 4. 使用 CLI 运行回测或实盘
回测:
```bash
evotraders backtest --start 2025-11-01 --end 2025-12-01
evotraders backtest --start 2025-11-01 --end 2025-12-01 --enable-memory
evotraders backtest --config-name smoke_fullstack --start 2025-11-01 --end 2025-12-01
```
实盘:
```bash
evotraders live
evotraders live --enable-memory
evotraders live --schedule-mode intraday --interval-minutes 60
evotraders live --trigger-time 22:30
```
帮助:
```bash
evotraders --help
evotraders backtest --help
evotraders live --help
evotraders frontend --help
```
### 离线回测数据
如果只是想快速体验回测,不依赖外部行情 API可以下载离线数据包并解压到 `backend/data`
```bash
wget "https://agentscope-open.oss-cn-beijing.aliyuncs.com/ret_data.zip"
unzip ret_data.zip -d backend/data
```
该压缩包提供基础的股票行情数据,解压后即可直接用于回测演示。
**实盘交易:**
```bash
evotraders live # 立即运行(默认)
evotraders live --enable-memory # 使用记忆
evotraders live --mock # Mock 模式(测试)
evotraders live -t 22:30 # 每天本地时间 22:30 运行(自动转换为 NYSE 时区)
evotraders live --schedule-mode intraday --interval-minutes 60 # 每隔 1 小时触发一次;仅交易时段执行交易,其他时段只分析
```
前端的“运行设置”面板也支持热更新 `schedule_mode``interval_minutes``max_comm_cycles`;其中 daily 模式时间当前按 NYSE/ET 配置。
**获取帮助:**
```bash
evotraders --help # 查看整体命令行帮助
evotraders backtest --help # 查看回测模式的参数说明
evotraders live --help # 查看实盘/Mock 运行的参数说明
```
**启动可视化界面:**
```bash
# 确保已安装 npm, 否则请安装:
# npm install
evotraders frontend # 默认连接 8765 端口, 你可以修改 ./frontend/env.local 中的地址从而修改端口号
```
访问 `http://localhost:5173/` 查看交易大厅,选择日期并点击 Run/Replay 观察决策过程。
---
## 系统架构
## 运行时数据布局
![架构图](docs/assets/evotraders_pipeline.jpg)
- 长期研究数据保存在 `data/market_research.db`
- 每次 run 的状态写入 `runs/<run_id>/`
- `runs/<run_id>/BOOTSTRAP.md` 保存该 run 的 bootstrap 值和 prompt body
- `runs/<run_id>/state/runtime_state.json` 保存运行时快照
- `runs/<run_id>/team_dashboard/*.json` 主要是给 dashboard 用的兼容导出层,不是唯一真相源
### 智能体设计
可选保留策略:
**分析师团队:**
- **基本面分析师**:财务健康度、盈利能力、增长质量
- **技术分析师**:价格趋势、技术指标、动量分析
- **情绪分析师**:市场情绪、新闻舆情、内部人交易
- **估值分析师**DCF、剩余收益、EV/EBITDA
**决策层:**
- **投资组合经理**:整合来自分析师的分析信号,执行沟通策略,结合分析师和团队历史表现、近期投资记忆和长期投资经验,进行最终决策
- **风险管理**:实时价格与波动率监控、头寸限制,多层风险预警
### 决策流程
```
实时行情 → 独立分析 → 智能沟通 (1v1/1vN/NvN) → 决策执行 → 收益评估 → 学习与进化(记忆更新)
```bash
RUNS_RETENTION_COUNT=20
```
每个交易日经历五个阶段:
1. **分析阶段**:各智能体基于各自工具和历史经验独立分析
2. **沟通阶段**:通过私聊、通知、会议等方式交换观点
3. **决策阶段**:投资组合经理综合判断,给出最终交易
4. **评估阶段**
- **业绩图表**: 追踪组合收益曲线 vs. 基准策略(等权、市值加权、动量)。用于评估整体策略有效性。
- **分析师排名**: 在 Trading Room 点击头像查看分析师表现(胜率、牛/熊市胜率)。用于了解哪些分析师提供最有价值的洞察。
- **统计数据**: 详细的持仓和交易历史。用于深入分析仓位管理和执行质量。
4. **复盘阶段**Agents 根据当日实际收益反思决策、总结经验,并存入 ReMe 记忆框架以持续改进
只有形如 `YYYYMMDD_HHMMSS` 的时间戳目录会被自动清理;`live``smoke_fullstack``reload_demo_*` 这类命名 run 会保留。
---
### 模块支持
## 前端服务路由
- **智能体框架**[AgentScope](https://github.com/agentscope-ai/agentscope)
- **记忆系统**[ReMe](https://github.com/agentscope-ai/reme)
- **LLM 支持**OpenAI、DeepSeek、Qwen、Moonshot、Zhipu AI 等
前端始终会使用 control plane 和 runtime API同时可以选择直连拆分服务读取只读数据。
常用前端环境变量:
```bash
VITE_CONTROL_API_BASE_URL=http://localhost:8000/api
VITE_RUNTIME_API_BASE_URL=http://localhost:8003/api/runtime
VITE_NEWS_SERVICE_URL=http://localhost:8002
VITE_TRADING_SERVICE_URL=http://localhost:8001
VITE_WS_URL=ws://localhost:8765
```
如果不配置,前端会按本地默认值和兼容回退逻辑运行。
---
## 决策流程
```text
市场数据 -> 分析师独立分析 -> 团队沟通 -> 投资决策 ->
风控审核 -> 执行/结算 -> 复盘/记忆更新
```
运行时管理器还会跟踪:
- agent 注册和状态
- 待审批项
- run 事件
- 当前 session key
---
## 自定义配置
### 自定义分析师角色
### 新增或修改分析师角色
1. 在 [./backend/agents/prompts/analyst/personas.yaml](./backend/agents/prompts/analyst/personas.yaml) 中注册角色信息,例如:
1. 在 [backend/agents/prompts/analyst/personas.yaml](./backend/agents/prompts/analyst/personas.yaml) 中定义 persona
2. 在 [backend/config/constants.py](./backend/config/constants.py) 中注册角色
3. 如有需要,在 [frontend/src/config/constants.js](./frontend/src/config/constants.js) 中补充前端展示元数据
示例:
```yaml
comprehensive_analyst:
name: "Comprehensive Analyst"
focus:
- ...
preferred_tools: # Flexibly select based on situation
- multi-factor synthesis
preferred_tools:
- get_stock_price
- get_company_financials
description: |
As a comprehensive analyst ...
A generalist analyst that combines multiple signals.
```
2. 在 [./backend/config/constants.py](./backend/config/constants.py) 添加角色定义
```python
ANALYST_TYPES = {
# 增加新的分析师
"comprehensive_analyst": {
"display_name": "Comprehensive Analyst",
"agent_id": "comprehensive_analyst",
"description": "Uses LLM to intelligently select analysis tools, performs comprehensive analysis",
"order": 15
}
}
```
### 配置各 Agent 使用的模型
3. 在前端配置 [./frontend/src/config/constants.js](./frontend/src/config/constants.js) 中引入新角色(可选)
```javascript
export const AGENTS = [
// 覆盖掉其中某一个agent
{
id: "comprehensive_analyst",
name: "Comprehensive Analyst",
role: "Comprehensive Analyst",
avatar: `${ASSET_BASE_URL}/...`,
colors: { bg: '#F9FDFF', text: '#1565C0', accent: '#1565C0' }
}
]
```
### 自定义模型
在 [.env](.env) 文件中配置不同智能体使用的模型:
模型覆盖在 `.env` 中配置:
```bash
AGENT_SENTIMENT_ANALYST_MODEL_NAME=qwen3-max-preview
AGENT_FUNDAMENTAL_ANALYST_MODEL_NAME=deepseek-chat
AGENT_TECHNICAL_ANALYST_MODEL_NAME=glm-4-plus
AGENT_VALUATION_ANALYST_MODEL_NAME=moonshot-v1-32k
AGENT_SENTIMENT_ANALYST_MODEL_NAME=deepseek-v3.2-exp
AGENT_TECHNICAL_ANALYST_MODEL_NAME=glm-4.6
AGENT_FUNDAMENTALS_ANALYST_MODEL_NAME=qwen3-max-preview
AGENT_VALUATION_ANALYST_MODEL_NAME=Moonshot-Kimi-K2-Instruct
AGENT_RISK_MANAGER_MODEL_NAME=qwen3-max-preview
AGENT_PORTFOLIO_MANAGER_MODEL_NAME=qwen3-max-preview
```
### 项目结构
### run 级 BOOTSTRAP 配置
每个 run 都可以通过 `runs/<run_id>/BOOTSTRAP.md` 覆盖默认值。该文件由 [backend/config/bootstrap_config.py](./backend/config/bootstrap_config.py) 解析front matter 可配置:
```yaml
tickers:
- AAPL
- MSFT
initial_cash: 100000
margin_requirement: 0.5
max_comm_cycles: 2
schedule_mode: daily
trigger_time: "09:30"
enable_memory: false
```
EvoTraders/
初始化一个 run 工作区:
```bash
evotraders init-workspace --config-name my_run
```
---
## 项目结构
```text
evotraders/
├── backend/
│ ├── agents/ # 智能体实现
│ ├── communication/ # 通信系统
│ ├── memory/ # 记忆系统 (ReMe)
│ ├── tools/ # 分析工具集
│ ├── servers/ # WebSocket 服务
── cli.py # CLI 入口
├── frontend/ # React 可视化界面
── logs_and_memory/ # 日志和记忆数据
│ ├── agents/ # agent 角色、prompts、skills、workspaces
│ ├── api/ # FastAPI 路由层
│ ├── apps/ # 拆分服务 app surface
│ ├── core/ # pipeline、scheduler、state sync
│ ├── runtime/ # runtime manager 和 agent runtime state
── services/ # gateway、market/storage/db 服务
│ └── cli.py # Typer CLI 入口
── frontend/ # React + Vite 前端
├── shared/ # 拆分服务共用 client 和 schema
├── runs/ # run 级状态和 dashboard 导出
├── data/ # 长期研究数据
└── services/README.md
```
---
## 测试
后端测试位于 `backend/tests`,覆盖 service app、shared client、domain、路由、enrichment、gateway 支撑模块和 runtime 支撑模块。
常用命令:
```bash
pytest
pytest backend/tests/test_runtime_service_app.py
pytest backend/tests/test_trading_service_app.py
```
前端测试:
```bash
cd frontend
npm test
```
---
## 许可与免责
EvoTraders 是一个研究和教育项目,采用 Apache 2.0 许可协议开源
大时代 是研究和教育用途项目。再次分发或商用前,请先核对仓库中的实际 license 文件
**风险提示**在实际资金交易前,请务必进行充分测试和风险评估历史表现不代表未来收益,投资有风险,决策需谨慎
**风险提示**本项目不构成投资建议。任何实盘部署前都应进行充分测试和风险评估历史表现不代表未来收益。

View File

@@ -16,7 +16,7 @@ Exports:
# New EvoAgent architecture (from agent_core.py)
from .agent_core import EvoAgent, ToolGuardMixin, CommandHandler
from .factory import AgentFactory, ModelConfig, RoleConfig
from .factory import AgentFactory, ModelConfig
from .workspace import WorkspaceManager, WorkspaceRegistry, WorkspaceConfig
from .workspace_manager import RunWorkspaceManager
from .registry import AgentRegistry, AgentInfo, get_registry, reset_registry
@@ -36,7 +36,6 @@ __all__ = [
"CommandHandler",
"AgentFactory",
"ModelConfig",
"RoleConfig",
"WorkspaceManager",
"WorkspaceRegistry",
"WorkspaceConfig",

View File

@@ -84,7 +84,6 @@ class AnalystAgent(ReActAgent):
agent_id=self.agent_id,
config_name=self.config.get("config_name", "default"),
toolkit=self.toolkit,
analyst_type=self.analyst_type_key,
)
async def reply(self, x: Msg = None) -> Msg:

View File

@@ -1,5 +1,5 @@
# -*- coding: utf-8 -*-
"""Base agent module for EvoTraders.
"""Base agent module for 大时代.
提供Agent基础类、命令处理、工具守卫和钩子管理等功能。
"""

View File

@@ -0,0 +1,452 @@
# -*- coding: utf-8 -*-
"""Evaluation hooks system for skills.
Provides evaluation metric collection and storage for skill performance tracking.
Based on the evaluation hooks design in SKILL_TEMPLATE.md.
"""
from __future__ import annotations
import json
import logging
from dataclasses import dataclass, field, asdict
from datetime import datetime
from enum import Enum
from pathlib import Path
from typing import Any, Dict, List, Optional, Set
logger = logging.getLogger(__name__)
class MetricType(Enum):
"""Types of evaluation metrics."""
HIT_RATE = "hit_rate" # 信号命中率
RISK_VIOLATION = "risk_violation" # 风控违例率
POSITION_DEVIATION = "position_deviation" # 仓位偏离率
PnL_ATTRIBUTION = "pnl_attribution" # P&L 归因一致性
SIGNAL_CONSISTENCY = "signal_consistency" # 信号一致性
DECISION_LATENCY = "decision_latency" # 决策延迟
TOOL_USAGE = "tool_usage" # 工具使用率
CUSTOM = "custom" # 自定义指标
@dataclass
class EvaluationMetric:
"""A single evaluation metric."""
name: str
metric_type: MetricType
value: float
timestamp: str = field(default_factory=lambda: datetime.now().isoformat())
metadata: Dict[str, Any] = field(default_factory=dict)
def to_dict(self) -> Dict[str, Any]:
return {
"name": self.name,
"metric_type": self.metric_type.value,
"value": self.value,
"timestamp": self.timestamp,
"metadata": self.metadata,
}
@dataclass
class EvaluationResult:
"""Evaluation result for a skill execution."""
skill_name: str
run_id: str
agent_id: str
metrics: List[EvaluationMetric] = field(default_factory=list)
inputs: Dict[str, Any] = field(default_factory=dict)
outputs: Dict[str, Any] = field(default_factory=dict)
decision: Optional[str] = None
success: bool = True
error_message: Optional[str] = None
started_at: Optional[str] = None
completed_at: Optional[str] = field(default_factory=lambda: datetime.now().isoformat())
def to_dict(self) -> Dict[str, Any]:
return {
"skill_name": self.skill_name,
"run_id": self.run_id,
"agent_id": self.agent_id,
"metrics": [m.to_dict() for m in self.metrics],
"inputs": self.inputs,
"outputs": self.outputs,
"decision": self.decision,
"success": self.success,
"error_message": self.error_message,
"started_at": self.started_at,
"completed_at": self.completed_at,
}
class EvaluationHook:
"""Hook for collecting skill evaluation metrics.
This hook collects and stores evaluation metrics after skill execution
for later analysis and memory/reflection stages.
"""
def __init__(
self,
storage_dir: Path,
run_id: str,
agent_id: str,
):
"""Initialize evaluation hook.
Args:
storage_dir: Directory to store evaluation results
run_id: Current run identifier
agent_id: Current agent identifier
"""
self.storage_dir = Path(storage_dir)
self.run_id = run_id
self.agent_id = agent_id
self._current_evaluation: Optional[EvaluationResult] = None
def start_evaluation(
self,
skill_name: str,
inputs: Dict[str, Any],
) -> None:
"""Start a new evaluation session.
Args:
skill_name: Name of the skill being evaluated
inputs: Input parameters for the skill
"""
self._current_evaluation = EvaluationResult(
skill_name=skill_name,
run_id=self.run_id,
agent_id=self.agent_id,
inputs=inputs,
started_at=datetime.now().isoformat(),
)
logger.debug(f"Started evaluation for skill: {skill_name}")
def add_metric(
self,
name: str,
metric_type: MetricType,
value: float,
metadata: Optional[Dict[str, Any]] = None,
) -> None:
"""Add an evaluation metric.
Args:
name: Metric name
metric_type: Type of metric
value: Metric value
metadata: Additional metadata
"""
if self._current_evaluation is None:
logger.warning("No active evaluation session, ignoring metric")
return
metric = EvaluationMetric(
name=name,
metric_type=metric_type,
value=value,
metadata=metadata or {},
)
self._current_evaluation.metrics.append(metric)
logger.debug(f"Added metric: {name} = {value}")
def add_metrics(self, metrics: List[EvaluationMetric]) -> None:
"""Add multiple evaluation metrics at once.
Args:
metrics: List of metrics to add
"""
if self._current_evaluation is None:
logger.warning("No active evaluation session, ignoring metrics")
return
self._current_evaluation.metrics.extend(metrics)
def record_outputs(self, outputs: Dict[str, Any]) -> None:
"""Record skill outputs.
Args:
outputs: Output from skill execution
"""
if self._current_evaluation is None:
logger.warning("No active evaluation session, ignoring outputs")
return
self._current_evaluation.outputs = outputs
def record_decision(self, decision: str) -> None:
"""Record the final decision.
Args:
decision: Final decision made by the skill
"""
if self._current_evaluation is None:
logger.warning("No active evaluation session, ignoring decision")
return
self._current_evaluation.decision = decision
def complete_evaluation(
self,
success: bool = True,
error_message: Optional[str] = None,
) -> Optional[EvaluationResult]:
"""Complete the evaluation session and persist results.
Args:
success: Whether the skill execution was successful
error_message: Error message if failed
Returns:
The completed evaluation result, or None if no active evaluation
"""
if self._current_evaluation is None:
logger.warning("No active evaluation to complete")
return None
self._current_evaluation.success = success
self._current_evaluation.error_message = error_message
self._current_evaluation.completed_at = datetime.now().isoformat()
# Persist to storage
result = self._persist_evaluation(self._current_evaluation)
self._current_evaluation = None
logger.debug(f"Completed evaluation for skill: {result.skill_name}")
return result
def _persist_evaluation(self, evaluation: EvaluationResult) -> EvaluationResult:
"""Persist evaluation result to storage.
Args:
evaluation: Evaluation result to persist
Returns:
The persisted evaluation
"""
# Create run-specific directory
run_dir = self.storage_dir / self.run_id
run_dir.mkdir(parents=True, exist_ok=True)
# Create agent-specific subdirectory
agent_dir = run_dir / self.agent_id
agent_dir.mkdir(parents=True, exist_ok=True)
# Generate filename with timestamp
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S_%f")
filename = f"{evaluation.skill_name}_{timestamp}.json"
filepath = agent_dir / filename
# Write evaluation result
try:
with open(filepath, "w", encoding="utf-8") as f:
json.dump(evaluation.to_dict(), f, ensure_ascii=False, indent=2)
logger.info(f"Persisted evaluation to: {filepath}")
except Exception as e:
logger.error(f"Failed to persist evaluation: {e}")
return evaluation
def cancel_evaluation(self) -> None:
"""Cancel the current evaluation session without saving."""
if self._current_evaluation is not None:
logger.debug(f"Cancelled evaluation for: {self._current_evaluation.skill_name}")
self._current_evaluation = None
class EvaluationCollector:
"""Collector for aggregating evaluation metrics across runs.
Provides methods to query and analyze evaluation results.
"""
def __init__(self, storage_dir: Path):
"""Initialize evaluation collector.
Args:
storage_dir: Root directory containing evaluation results
"""
self.storage_dir = Path(storage_dir)
def get_run_evaluations(
self,
run_id: str,
agent_id: Optional[str] = None,
) -> List[EvaluationResult]:
"""Get all evaluations for a run.
Args:
run_id: Run identifier
agent_id: Optional agent identifier to filter by
Returns:
List of evaluation results
"""
run_dir = self.storage_dir / run_id
if not run_dir.exists():
return []
evaluations = []
agent_dirs = [run_dir / agent_id] if agent_id else run_dir.iterdir()
for agent_dir in agent_dirs:
if not agent_dir.is_dir():
continue
for eval_file in agent_dir.glob("*.json"):
try:
with open(eval_file, "r", encoding="utf-8") as f:
data = json.load(f)
evaluations.append(self._parse_evaluation(data))
except Exception as e:
logger.warning(f"Failed to load evaluation {eval_file}: {e}")
return evaluations
def get_skill_metrics(
self,
skill_name: str,
run_ids: Optional[List[str]] = None,
) -> List[EvaluationMetric]:
"""Get all metrics for a specific skill.
Args:
skill_name: Name of the skill
run_ids: Optional list of run IDs to filter by
Returns:
List of metrics for the skill
"""
metrics = []
if run_ids is None:
run_ids = [d.name for d in self.storage_dir.iterdir() if d.is_dir()]
for run_id in run_ids:
evaluations = self.get_run_evaluations(run_id)
for eval_result in evaluations:
if eval_result.skill_name == skill_name:
metrics.extend(eval_result.metrics)
return metrics
def calculate_skill_stats(
self,
skill_name: str,
metric_type: MetricType,
run_ids: Optional[List[str]] = None,
) -> Dict[str, float]:
"""Calculate statistics for a specific metric type.
Args:
skill_name: Name of the skill
metric_type: Type of metric to calculate
run_ids: Optional list of run IDs to filter by
Returns:
Dictionary with min, max, avg, count statistics
"""
metrics = self.get_skill_metrics(skill_name, run_ids)
filtered = [m for m in metrics if m.metric_type == metric_type]
if not filtered:
return {"count": 0}
values = [m.value for m in filtered]
return {
"count": len(values),
"min": min(values),
"max": max(values),
"avg": sum(values) / len(values),
}
def _parse_evaluation(self, data: Dict[str, Any]) -> EvaluationResult:
"""Parse evaluation data into EvaluationResult.
Args:
data: Raw evaluation data
Returns:
Parsed EvaluationResult
"""
metrics = []
for m in data.get("metrics", []):
metrics.append(EvaluationMetric(
name=m["name"],
metric_type=MetricType(m["metric_type"]),
value=m["value"],
timestamp=m.get("timestamp", ""),
metadata=m.get("metadata", {}),
))
return EvaluationResult(
skill_name=data["skill_name"],
run_id=data["run_id"],
agent_id=data["agent_id"],
metrics=metrics,
inputs=data.get("inputs", {}),
outputs=data.get("outputs", {}),
decision=data.get("decision"),
success=data.get("success", True),
error_message=data.get("error_message"),
started_at=data.get("started_at"),
completed_at=data.get("completed_at"),
)
def parse_evaluation_hooks(skill_dir: Path) -> Dict[str, Any]:
"""Parse evaluation hooks from SKILL.md.
Extracts the Optional: Evaluation hooks section from skill documentation.
Args:
skill_dir: Skill directory path
Returns:
Dictionary containing evaluation hook definitions
"""
skill_md = skill_dir / "SKILL.md"
if not skill_md.exists():
return {}
try:
content = skill_md.read_text(encoding="utf-8")
# Extract evaluation hooks section
if "## Optional: Evaluation hooks" in content:
start = content.find("## Optional: Evaluation hooks")
# Find the next ## section or end of file
next_section = content.find("\n## ", start + 1)
if next_section == -1:
eval_section = content[start:]
else:
eval_section = content[start:next_section]
# Parse metrics from the section
metrics = []
for metric_type in MetricType:
if metric_type.value.replace("_", " ") in eval_section.lower():
metrics.append(metric_type.value)
return {
"supported_metrics": metrics,
"section_content": eval_section.strip(),
}
except Exception as e:
logger.warning(f"Failed to parse evaluation hooks: {e}")
return {}
__all__ = [
"MetricType",
"EvaluationMetric",
"EvaluationResult",
"EvaluationHook",
"EvaluationCollector",
"parse_evaluation_hooks",
]

View File

@@ -1,5 +1,5 @@
# -*- coding: utf-8 -*-
"""EvoAgent - Core agent implementation for EvoTraders.
"""EvoAgent - Core agent implementation for 大时代.
This module provides the main EvoAgent class built on AgentScope's ReActAgent,
with integrated tools, skills, and memory management based on CoPaw design.
@@ -470,7 +470,7 @@ class EvoAgent(ToolGuardMixin, ReActAgent):
"""
return self._messenger
def delegate_task(
async def delegate_task(
self,
task_type: str,
task_data: Dict[str, Any],
@@ -493,7 +493,7 @@ class EvoAgent(ToolGuardMixin, ReActAgent):
}
try:
return self._task_delegator.delegate_task(
return await self._task_delegator.delegate_task(
task_type=task_type,
task_data=task_data,
target_agent=target_agent,

View File

@@ -294,8 +294,8 @@ class WorkspaceWatchHook(Hook):
# Files to monitor (same as PromptBuilder.DEFAULT_FILES)
WATCHED_FILES = frozenset([
"SOUL.md", "AGENTS.md", "PROFILE.md", "ROLE.md",
"POLICY.md", "MEMORY.md", "HEARTBEAT.md", "STYLE.md",
"SOUL.md", "AGENTS.md", "PROFILE.md",
"POLICY.md", "MEMORY.md",
"BOOTSTRAP.md",
])
@@ -601,94 +601,6 @@ class MemoryCompactionHook(Hook):
)
class HeartbeatHook(Hook):
"""Pre-reasoning hook that injects HEARTBEAT.md content.
Reads the agent's HEARTBEAT.md file and prepends it to the
reasoning input, causing the agent to perform self-checks.
This enables "主动检查" (proactive monitoring) - periodic
market condition and position checks during trading hours.
"""
HEARTBEAT_FILE = "HEARTBEAT.md"
def __init__(self, workspace_dir: Path):
"""Initialize heartbeat hook.
Args:
workspace_dir: Working directory containing HEARTBEAT.md
"""
self.workspace_dir = Path(workspace_dir)
self._completed_flag = self.workspace_dir / ".heartbeat_completed"
def _read_heartbeat_content(self) -> Optional[str]:
"""Read HEARTBEAT.md if it exists and is non-empty.
Returns:
The HEARTBEAT.md content stripped of whitespace, or None
if the file is absent or empty.
"""
hb_path = self.workspace_dir / self.HEARTBEAT_FILE
if not hb_path.exists():
return None
content = hb_path.read_text(encoding="utf-8").strip()
return content if content else None
async def __call__(
self,
agent: "ReActAgent",
kwargs: Dict[str, Any],
) -> Optional[Dict[str, Any]]:
"""Prepend heartbeat task to user message.
Args:
agent: The agent instance
kwargs: Input arguments to the _reasoning method
Returns:
Modified kwargs with heartbeat content prepended, or None
if no HEARTBEAT.md content is available.
"""
try:
content = self._read_heartbeat_content()
if not content:
return None
logger.debug(
"Heartbeat: found HEARTBEAT.md for agent %s",
getattr(agent, "agent_id", "unknown"),
)
# Build heartbeat task instruction (Chinese)
hb_task = (
"# 定期主动检查\n\n"
f"{content}\n\n"
"请执行上述检查并报告结果。"
)
# Inject into the first user message in memory
if hasattr(agent, "memory") and agent.memory.content:
system_count = sum(
1 for msg, _ in agent.memory.content if msg.role == "system"
)
for msg, _ in agent.memory.content[system_count:]:
if msg.role == "user":
original_content = msg.content
msg.content = hb_task + "\n\n" + original_content
break
logger.debug(
"Heartbeat task prepended for agent %s",
getattr(agent, "agent_id", "unknown"),
)
except Exception as e:
logger.error("Heartbeat hook failed: %s", e, exc_info=True)
return None
__all__ = [
"Hook",
"HookManager",
@@ -696,7 +608,6 @@ __all__ = [
"HOOK_PRE_REASONING",
"HOOK_POST_ACTING",
"BootstrapHook",
"HeartbeatHook",
"MemoryCompactionHook",
"WorkspaceWatchHook",
]

View File

@@ -0,0 +1,489 @@
# -*- coding: utf-8 -*-
"""Skill adaptation hook for automatic evaluation-to-iteration闭环.
Monitors evaluation metrics against configurable thresholds and triggers
automatic skill reload or logs warnings when thresholds are breached.
"""
from __future__ import annotations
import json
import logging
from dataclasses import dataclass, field
from datetime import datetime
from enum import Enum
from pathlib import Path
from typing import Any, Dict, List, Optional, Set
from .evaluation_hook import (
EvaluationCollector,
EvaluationResult,
MetricType,
)
logger = logging.getLogger(__name__)
class AdaptationAction(Enum):
"""Actions to take when threshold is breached."""
RELOAD = "reload" # 自动重新加载技能
WARN = "warn" # 记录警告供人工审核
BOTH = "both" # 同时执行重载和警告
NONE = "none" # 不做任何操作
@dataclass
class AdaptationThreshold:
"""Threshold configuration for a metric."""
metric_type: MetricType
operator: str = "lt" # lt (less than), gt (greater than), lte, gte, eq
value: float = 0.0
window_size: int = 10 # 移动窗口大小,用于计算滑动平均
min_samples: int = 5 # 最少样本数才触发检查
action: AdaptationAction = AdaptationAction.WARN
cooldown_seconds: int = 300 # 触发后的冷却时间
def evaluate(self, current_value: float) -> bool:
"""Evaluate if threshold is breached."""
ops = {
"lt": lambda x, y: x < y,
"lte": lambda x, y: x <= y,
"gt": lambda x, y: x > y,
"gte": lambda x, y: x >= y,
"eq": lambda x, y: x == y,
}
op_func = ops.get(self.operator)
if op_func is None:
logger.warning(f"Unknown operator: {self.operator}")
return False
return op_func(current_value, self.value)
def to_dict(self) -> Dict[str, Any]:
return {
"metric_type": self.metric_type.value,
"operator": self.operator,
"value": self.value,
"window_size": self.window_size,
"min_samples": self.min_samples,
"action": self.action.value,
"cooldown_seconds": self.cooldown_seconds,
}
@dataclass
class AdaptationEvent:
"""Record of an adaptation trigger event."""
timestamp: str
skill_name: str
metric_type: MetricType
threshold: AdaptationThreshold
current_value: float
avg_value: float
action_taken: AdaptationAction
details: Dict[str, Any] = field(default_factory=dict)
def to_dict(self) -> Dict[str, Any]:
return {
"timestamp": self.timestamp,
"skill_name": self.skill_name,
"metric_type": self.metric_type.value,
"threshold": self.threshold.to_dict(),
"current_value": self.current_value,
"avg_value": self.avg_value,
"action_taken": self.action_taken.value,
"details": self.details,
}
class SkillAdaptationHook:
"""Hook for monitoring evaluation metrics and triggering skill adaptation.
This hook wraps EvaluationHook to add threshold-based adaptation logic.
When metrics breach configured thresholds, it can:
- Automatically reload skills via SkillsManager
- Log warnings for human review
- Both
"""
# Default thresholds for common metrics
DEFAULT_THRESHOLDS: List[AdaptationThreshold] = [
AdaptationThreshold(
metric_type=MetricType.HIT_RATE,
operator="lt",
value=0.5,
action=AdaptationAction.WARN,
cooldown_seconds=600,
),
AdaptationThreshold(
metric_type=MetricType.RISK_VIOLATION,
operator="gt",
value=0.1,
action=AdaptationAction.WARN,
cooldown_seconds=300,
),
AdaptationThreshold(
metric_type=MetricType.DECISION_LATENCY,
operator="gt",
value=5000, # 5 seconds
action=AdaptationAction.WARN,
cooldown_seconds=300,
),
]
def __init__(
self,
storage_dir: Path,
run_id: str,
agent_id: str,
thresholds: Optional[List[AdaptationThreshold]] = None,
collector: Optional[EvaluationCollector] = None,
):
"""Initialize skill adaptation hook.
Args:
storage_dir: Directory to store adaptation events
run_id: Current run identifier
agent_id: Current agent identifier
thresholds: Custom threshold configurations (uses defaults if None)
collector: Optional EvaluationCollector for historical data
"""
self.storage_dir = Path(storage_dir)
self.run_id = run_id
self.agent_id = agent_id
self.thresholds = thresholds or self.DEFAULT_THRESHOLDS
self.collector = collector or EvaluationCollector(storage_dir)
# Track cooldowns to prevent rapid re-triggering
self._cooldowns: Dict[str, datetime] = {}
# Store recent metrics in memory for quick access
self._recent_metrics: Dict[str, List[float]] = {}
# Pending adaptation events
self._pending_events: List[AdaptationEvent] = []
def check_threshold(
self,
skill_name: str,
metric_type: MetricType,
current_value: float,
) -> Optional[AdaptationEvent]:
"""Check if a metric breaches any threshold.
Args:
skill_name: Name of the skill
metric_type: Type of metric
current_value: Current metric value
Returns:
AdaptationEvent if threshold breached, None otherwise
"""
# Find applicable thresholds
applicable_thresholds = [
t for t in self.thresholds
if t.metric_type == metric_type
]
if not applicable_thresholds:
return None
# Check cooldown
cooldown_key = f"{skill_name}:{metric_type.value}"
now = datetime.now()
last_trigger = self._cooldowns.get(cooldown_key)
# Store current value first for avg calculation
self._store_metric(cooldown_key, current_value)
for threshold in applicable_thresholds:
if last_trigger:
elapsed = (now - last_trigger).total_seconds()
if elapsed < threshold.cooldown_seconds:
continue
# Evaluate threshold
if threshold.evaluate(current_value):
# Calculate moving average
avg_value = self._calculate_avg(skill_name, metric_type, current_value)
# Check minimum samples (allow immediate trigger if min_samples <= 1)
sample_count = len(self._recent_metrics.get(cooldown_key, []))
if threshold.min_samples > 1 and sample_count < threshold.min_samples:
# Not enough samples yet
continue
# Trigger adaptation
event = AdaptationEvent(
timestamp=now.isoformat(),
skill_name=skill_name,
metric_type=metric_type,
threshold=threshold,
current_value=current_value,
avg_value=avg_value,
action_taken=threshold.action,
details={
"run_id": self.run_id,
"agent_id": self.agent_id,
},
)
# Update cooldown
self._cooldowns[cooldown_key] = now
# Persist event
self._persist_event(event)
logger.info(
f"Threshold breached for {skill_name}.{metric_type.value}: "
f"current={current_value}, avg={avg_value}, action={threshold.action.value}"
)
return event
return None
def _calculate_avg(
self,
skill_name: str,
metric_type: MetricType,
current_value: float,
) -> float:
"""Calculate moving average for a metric."""
key = f"{skill_name}:{metric_type.value}"
values = self._recent_metrics.get(key, [])
if not values:
return current_value
return sum(values) / len(values)
def _store_metric(self, key: str, value: float) -> None:
"""Store metric value with sliding window."""
if key not in self._recent_metrics:
self._recent_metrics[key] = []
self._recent_metrics[key].append(value)
# Keep only last 100 values
if len(self._recent_metrics[key]) > 100:
self._recent_metrics[key] = self._recent_metrics[key][-100:]
def _persist_event(self, event: AdaptationEvent) -> None:
"""Persist adaptation event to storage."""
run_dir = self.storage_dir / self.run_id / "adaptations"
run_dir.mkdir(parents=True, exist_ok=True)
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S_%f")
filename = f"{event.skill_name}_{event.metric_type.value}_{timestamp}.json"
filepath = run_dir / filename
try:
with open(filepath, "w", encoding="utf-8") as f:
json.dump(event.to_dict(), f, ensure_ascii=False, indent=2)
logger.debug(f"Persisted adaptation event to: {filepath}")
except Exception as e:
logger.error(f"Failed to persist adaptation event: {e}")
# Also add to pending list
self._pending_events.append(event)
def get_pending_warnings(self) -> List[AdaptationEvent]:
"""Get all pending warning events that need human review."""
return [
e for e in self._pending_events
if e.action_taken in (AdaptationAction.WARN, AdaptationAction.BOTH)
]
def clear_pending_warnings(self) -> None:
"""Clear pending warnings after they have been reviewed."""
self._pending_events = [
e for e in self._pending_events
if e.action_taken == AdaptationAction.RELOAD
]
def get_recent_events(
self,
skill_name: Optional[str] = None,
metric_type: Optional[MetricType] = None,
limit: int = 50,
) -> List[AdaptationEvent]:
"""Get recent adaptation events.
Args:
skill_name: Optional filter by skill name
metric_type: Optional filter by metric type
limit: Maximum number of events to return
Returns:
List of recent adaptation events
"""
events_dir = self.storage_dir / self.run_id / "adaptations"
if not events_dir.exists():
return []
events = []
for eval_file in sorted(events_dir.glob("*.json"), reverse=True)[:limit]:
try:
with open(eval_file, "r", encoding="utf-8") as f:
data = json.load(f)
event = self._parse_event(data)
if skill_name and event.skill_name != skill_name:
continue
if metric_type and event.metric_type != metric_type:
continue
events.append(event)
except Exception as e:
logger.warning(f"Failed to load adaptation event {eval_file}: {e}")
return events
def _parse_event(self, data: Dict[str, Any]) -> AdaptationEvent:
"""Parse adaptation event from JSON data."""
threshold_data = data.get("threshold", {})
metric_type = MetricType(threshold_data.get("metric_type", "custom"))
threshold = AdaptationThreshold(
metric_type=metric_type,
operator=threshold_data.get("operator", "lt"),
value=threshold_data.get("value", 0.0),
window_size=threshold_data.get("window_size", 10),
min_samples=threshold_data.get("min_samples", 5),
action=AdaptationAction(threshold_data.get("action", "warn")),
cooldown_seconds=threshold_data.get("cooldown_seconds", 300),
)
return AdaptationEvent(
timestamp=data.get("timestamp", ""),
skill_name=data.get("skill_name", ""),
metric_type=metric_type,
threshold=threshold,
current_value=data.get("current_value", 0.0),
avg_value=data.get("avg_value", 0.0),
action_taken=AdaptationAction(data.get("action_taken", "warn")),
details=data.get("details", {}),
)
def add_threshold(self, threshold: AdaptationThreshold) -> None:
"""Add a new threshold configuration."""
self.thresholds.append(threshold)
def remove_threshold(self, metric_type: MetricType) -> None:
"""Remove all thresholds for a specific metric type."""
self.thresholds = [
t for t in self.thresholds
if t.metric_type != metric_type
]
def update_threshold(
self,
metric_type: MetricType,
**kwargs,
) -> None:
"""Update threshold configuration for a metric type."""
for threshold in self.thresholds:
if threshold.metric_type == metric_type:
for key, value in kwargs.items():
if hasattr(threshold, key):
setattr(threshold, key, value)
def get_thresholds(self) -> List[AdaptationThreshold]:
"""Get current threshold configurations."""
return list(self.thresholds)
def is_in_cooldown(self, skill_name: str, metric_type: MetricType) -> bool:
"""Check if a skill/metric combination is in cooldown period."""
key = f"{skill_name}:{metric_type.value}"
last_trigger = self._cooldowns.get(key)
if not last_trigger:
return False
# Find the threshold for this metric type
for threshold in self.thresholds:
if threshold.metric_type == metric_type:
elapsed = (datetime.now() - last_trigger).total_seconds()
return elapsed < threshold.cooldown_seconds
return False
class AdaptationManager:
"""Manager for coordinating skill adaptation across multiple agents.
Provides centralized tracking of adaptation events and skill reloads.
"""
def __init__(self, storage_dir: Path):
"""Initialize adaptation manager.
Args:
storage_dir: Root directory for storing adaptation data
"""
self.storage_dir = Path(storage_dir)
self._hooks: Dict[str, SkillAdaptationHook] = {}
def get_hook(
self,
run_id: str,
agent_id: str,
thresholds: Optional[List[AdaptationThreshold]] = None,
) -> SkillAdaptationHook:
"""Get or create an adaptation hook for an agent.
Args:
run_id: Run identifier
agent_id: Agent identifier
thresholds: Optional custom thresholds
Returns:
SkillAdaptationHook instance
"""
key = f"{run_id}:{agent_id}"
if key not in self._hooks:
self._hooks[key] = SkillAdaptationHook(
storage_dir=self.storage_dir,
run_id=run_id,
agent_id=agent_id,
thresholds=thresholds,
)
return self._hooks[key]
def get_all_pending_warnings(self) -> List[AdaptationEvent]:
"""Get all pending warnings from all hooks."""
warnings = []
for hook in self._hooks.values():
warnings.extend(hook.get_pending_warnings())
return warnings
def get_run_adaptations(self, run_id: str) -> List[AdaptationEvent]:
"""Get all adaptation events for a run."""
events = []
for hook in self._hooks.values():
if hook.run_id == run_id:
events.extend(hook.get_recent_events())
return events
# Global manager instance
_adaptation_manager: Optional[AdaptationManager] = None
def get_adaptation_manager(storage_dir: Optional[Path] = None) -> AdaptationManager:
"""Get global adaptation manager instance.
Args:
storage_dir: Optional storage directory (required on first call)
Returns:
AdaptationManager instance
"""
global _adaptation_manager
if _adaptation_manager is None:
if storage_dir is None:
raise ValueError("storage_dir required on first initialization")
_adaptation_manager = AdaptationManager(storage_dir)
return _adaptation_manager
__all__ = [
"AdaptationAction",
"AdaptationThreshold",
"AdaptationEvent",
"SkillAdaptationHook",
"AdaptationManager",
"get_adaptation_manager",
]

View File

@@ -289,6 +289,7 @@ class ToolGuardMixin:
self._approval_timeout = approval_timeout
self._pending_approval: Optional[ToolApprovalRequest] = None
self._approval_callback: Optional[Callable[[ToolApprovalRequest], None]] = None
self._approval_lock = asyncio.Lock()
def set_approval_callback(
self,
@@ -383,73 +384,80 @@ class ToolGuardMixin:
Returns:
True if approved, False otherwise
"""
record = TOOL_GUARD_STORE.create_pending(
tool_name=tool_name,
tool_input=tool_input,
agent_id=getattr(self, "agent_id", "unknown"),
workspace_id=getattr(self, "workspace_id", "default"),
session_id=getattr(self, "session_id", None),
findings=default_findings_for_tool(tool_name),
)
manager = get_global_runtime_manager()
if manager:
manager.register_pending_approval(
record.approval_id,
{
"tool_name": record.tool_name,
"agent_id": record.agent_id,
"workspace_id": record.workspace_id,
"session_id": record.session_id,
"tool_input": record.tool_input,
},
async with self._approval_lock:
record = TOOL_GUARD_STORE.create_pending(
tool_name=tool_name,
tool_input=tool_input,
agent_id=getattr(self, "agent_id", "unknown"),
workspace_id=getattr(self, "workspace_id", "default"),
session_id=getattr(self, "session_id", None),
findings=default_findings_for_tool(tool_name),
)
self._pending_approval = ToolApprovalRequest(
approval_id=record.approval_id,
tool_name=tool_name,
tool_input=tool_input,
tool_call_id=tool_call_id,
session_id=getattr(self, "session_id", None),
)
record.pending_request = self._pending_approval
manager = get_global_runtime_manager()
if manager:
manager.register_pending_approval(
record.approval_id,
{
"tool_name": record.tool_name,
"agent_id": record.agent_id,
"workspace_id": record.workspace_id,
"session_id": record.session_id,
"tool_input": record.tool_input,
},
)
# Notify via callback if set
if self._approval_callback:
self._approval_callback(self._pending_approval)
self._pending_approval = ToolApprovalRequest(
approval_id=record.approval_id,
tool_name=tool_name,
tool_input=tool_input,
tool_call_id=tool_call_id,
session_id=getattr(self, "session_id", None),
)
record.pending_request = self._pending_approval
# Wait for approval
approval_request = self._pending_approval
# Notify via callback if set
if self._approval_callback:
self._approval_callback(self._pending_approval)
# Wait for approval (lock is released during wait, re-acquired after)
approval_request = self._pending_approval
# Wait for approval outside the lock to allow concurrent approval
approved = await approval_request.wait_for_approval(
timeout=self._approval_timeout
)
if approval_request:
status = (
ApprovalStatus.APPROVED
if approval_request.approved is True
else ApprovalStatus.DENIED
if approval_request.approved is False
else ApprovalStatus.EXPIRED
)
TOOL_GUARD_STORE.set_status(
approval_request.approval_id,
status,
resolved_by="agent",
notify_request=False,
)
manager = get_global_runtime_manager()
if manager:
manager.resolve_pending_approval(
approval_request.approval_id,
resolved_by="agent",
status=status.value,
async with self._approval_lock:
if approval_request:
status = (
ApprovalStatus.APPROVED
if approval_request.approved is True
else ApprovalStatus.DENIED
if approval_request.approved is False
else ApprovalStatus.EXPIRED
)
TOOL_GUARD_STORE.set_status(
approval_request.approval_id,
status,
resolved_by="agent",
notify_request=False,
)
manager = get_global_runtime_manager()
if manager:
manager.resolve_pending_approval(
approval_request.approval_id,
resolved_by="agent",
status=status.value,
)
# Only clear if this is still the same request
if self._pending_approval is approval_request:
self._pending_approval = None
self._pending_approval = None
return approved
def approve_guard_call(self, request_id: Optional[str] = None) -> bool:
async def approve_guard_call(self, request_id: Optional[str] = None) -> bool:
"""Approve a pending guard request.
This method is called externally to approve a tool call
@@ -461,28 +469,29 @@ class ToolGuardMixin:
Returns:
True if a request was approved, False if no pending request
"""
if self._pending_approval is None:
logger.warning("No pending approval request to approve")
return False
async with self._approval_lock:
if self._pending_approval is None:
logger.warning("No pending approval request to approve")
return False
TOOL_GUARD_STORE.set_status(
self._pending_approval.approval_id,
ApprovalStatus.APPROVED,
resolved_by="agent",
notify_request=False,
)
manager = get_global_runtime_manager()
if manager:
manager.resolve_pending_approval(
TOOL_GUARD_STORE.set_status(
self._pending_approval.approval_id,
ApprovalStatus.APPROVED,
resolved_by="agent",
status=ApprovalStatus.APPROVED.value,
notify_request=False,
)
self._pending_approval.approve()
logger.info("Approved tool call: %s", self._pending_approval.tool_name)
return True
manager = get_global_runtime_manager()
if manager:
manager.resolve_pending_approval(
self._pending_approval.approval_id,
resolved_by="agent",
status=ApprovalStatus.APPROVED.value,
)
self._pending_approval.approve()
logger.info("Approved tool call: %s", self._pending_approval.tool_name)
return True
def deny_guard_call(self, request_id: Optional[str] = None) -> bool:
async def deny_guard_call(self, request_id: Optional[str] = None) -> bool:
"""Deny a pending guard request.
This method is called externally to deny a tool call
@@ -494,26 +503,27 @@ class ToolGuardMixin:
Returns:
True if a request was denied, False if no pending request
"""
if self._pending_approval is None:
logger.warning("No pending approval request to deny")
return False
async with self._approval_lock:
if self._pending_approval is None:
logger.warning("No pending approval request to deny")
return False
TOOL_GUARD_STORE.set_status(
self._pending_approval.approval_id,
ApprovalStatus.DENIED,
resolved_by="agent",
notify_request=False,
)
manager = get_global_runtime_manager()
if manager:
manager.resolve_pending_approval(
TOOL_GUARD_STORE.set_status(
self._pending_approval.approval_id,
ApprovalStatus.DENIED,
resolved_by="agent",
status=ApprovalStatus.DENIED.value,
notify_request=False,
)
self._pending_approval.deny()
logger.info("Denied tool call: %s", self._pending_approval.tool_name)
return True
manager = get_global_runtime_manager()
if manager:
manager.resolve_pending_approval(
self._pending_approval.approval_id,
resolved_by="agent",
status=ApprovalStatus.DENIED.value,
)
self._pending_approval.deny()
logger.info("Denied tool call: %s", self._pending_approval.tool_name)
return True
async def _acting(self, tool_call) -> dict | None:
"""Intercept sensitive tool calls before execution.

View File

@@ -1,5 +1,5 @@
# -*- coding: utf-8 -*-
"""Agent Factory - Dynamic creation and management of EvoAgents."""
"""Agent Factory - Dynamic creation and management of AgentConfigs."""
import logging
import shutil
@@ -21,24 +21,8 @@ class ModelConfig:
max_tokens: int = 4096
@dataclass
class RoleConfig:
"""Role configuration for an agent."""
name: str
description: str = ""
focus_areas: List[str] = None
constraints: List[str] = None
def __post_init__(self):
if self.focus_areas is None:
self.focus_areas = []
if self.constraints is None:
self.constraints = []
class EvoAgent:
"""Represents a configured agent instance."""
class AgentConfig:
"""Represents a configured agent instance (data class)."""
def __init__(
self,
@@ -47,14 +31,12 @@ class EvoAgent:
workspace_id: str,
config_path: Path,
model_config: Optional[ModelConfig] = None,
role_config: Optional[RoleConfig] = None,
):
self.agent_id = agent_id
self.agent_type = agent_type
self.workspace_id = workspace_id
self.config_path = config_path
self.model_config = model_config or ModelConfig()
self.role_config = role_config
self.agent_dir = config_path.parent
def to_dict(self) -> Dict[str, Any]:
@@ -70,103 +52,12 @@ class EvoAgent:
"temperature": self.model_config.temperature,
"max_tokens": self.model_config.max_tokens,
},
"role_config": self.role_config.__dict__ if self.role_config else None,
}
class AgentFactory:
"""Factory for creating, cloning, and managing agents."""
# Default role templates by agent type
ROLE_TEMPLATES = {
"technical_analyst": {
"name": "Technical Analyst",
"description": "Analyze price patterns, trends, and technical indicators.",
"focus_areas": [
"Price action and chart patterns",
"Support and resistance levels",
"Technical indicators (RSI, MACD, Moving Averages)",
"Volume analysis",
],
"constraints": [
"State clear signal, confidence, and invalidation conditions",
"Use available technical analysis tools",
],
},
"fundamentals_analyst": {
"name": "Fundamentals Analyst",
"description": "Analyze company financials, earnings, and business metrics.",
"focus_areas": [
"Financial statements analysis",
"Earnings reports and guidance",
"Valuation metrics",
"Business model and competitive position",
],
"constraints": [
"State clear signal, confidence, and invalidation conditions",
"Use available fundamental analysis tools",
],
},
"sentiment_analyst": {
"name": "Sentiment Analyst",
"description": "Analyze market sentiment, news, and social signals.",
"focus_areas": [
"News sentiment analysis",
"Social media sentiment",
"Analyst ratings and price targets",
"Insider activity",
],
"constraints": [
"State clear signal, confidence, and invalidation conditions",
"Use available sentiment analysis tools",
],
},
"valuation_analyst": {
"name": "Valuation Analyst",
"description": "Perform valuation analysis and price target calculations.",
"focus_areas": [
"DCF and comparable valuation",
"Price target derivation",
"Margin of safety assessment",
"Risk-adjusted return expectations",
],
"constraints": [
"State clear signal, confidence, and invalidation conditions",
"Use available valuation tools",
],
},
"risk_manager": {
"name": "Risk Manager",
"description": "Quantify concentration, leverage, liquidity, and volatility risk.",
"focus_areas": [
"Portfolio concentration risk",
"Leverage and margin analysis",
"Liquidity assessment",
"Volatility and drawdown risk",
],
"constraints": [
"Prioritize highest-severity risk first",
"State concrete limits and recommendations",
"Use available risk tools before issuing final memo",
],
},
"portfolio_manager": {
"name": "Portfolio Manager",
"description": "Synthesize analyst and risk inputs into portfolio decisions.",
"focus_areas": [
"Position sizing and allocation",
"Risk-adjusted portfolio construction",
"Trade execution timing",
"Portfolio rebalancing",
],
"constraints": [
"Be concise, capital-aware, and explicit about sizing rationale",
"Respect cash, margin, and concentration constraints",
"Consider all analyst inputs before decisions",
],
},
}
def __init__(self, project_root: Optional[Path] = None):
"""Initialize the agent factory.
@@ -183,9 +74,8 @@ class AgentFactory:
agent_type: str,
workspace_id: str,
model_config: Optional[ModelConfig] = None,
role_config: Optional[RoleConfig] = None,
clone_from: Optional[str] = None,
) -> EvoAgent:
) -> AgentConfig:
"""Create a new agent.
Args:
@@ -193,11 +83,10 @@ class AgentFactory:
agent_type: Type of agent (e.g., "technical_analyst")
workspace_id: ID of the workspace to create agent in
model_config: Model configuration
role_config: Role configuration (auto-generated if None)
clone_from: Path to existing agent to clone from (optional)
Returns:
EvoAgent instance
AgentConfig instance
Raises:
ValueError: If agent already exists or workspace doesn't exist
@@ -223,24 +112,16 @@ class AgentFactory:
else:
self._copy_template(agent_dir, agent_id, agent_type)
# Generate role config if not provided
if role_config is None:
role_config = self._generate_role_config(agent_type)
# Generate ROLE.md
self._generate_role_md(agent_dir, role_config)
# Write agent.yaml
config_path = agent_dir / "agent.yaml"
self._write_agent_yaml(config_path, agent_id, agent_type, model_config)
return EvoAgent(
return AgentConfig(
agent_id=agent_id,
agent_type=agent_type,
workspace_id=workspace_id,
config_path=config_path,
model_config=model_config,
role_config=role_config,
)
def delete_agent(self, agent_id: str, workspace_id: str) -> bool:
@@ -267,7 +148,7 @@ class AgentFactory:
new_agent_id: str,
target_workspace_id: Optional[str] = None,
model_config: Optional[ModelConfig] = None,
) -> EvoAgent:
) -> AgentConfig:
"""Clone an existing agent.
Args:
@@ -278,7 +159,7 @@ class AgentFactory:
model_config: Optional new model configuration
Returns:
EvoAgent instance for the cloned agent
AgentConfig instance for the cloned agent
"""
target_workspace_id = target_workspace_id or source_workspace_id
source_dir = self.workspaces_root / source_workspace_id / "agents" / source_agent_id
@@ -369,9 +250,7 @@ class AgentFactory:
"SOUL.md": f"# Soul\n\nDescribe {agent_id}'s temperament, reasoning posture, and voice.\n\n",
"PROFILE.md": f"# Profile\n\nTrack {agent_id}'s long-lived investment style, preferences, and strengths.\n\n",
"MEMORY.md": f"# Memory\n\nStore durable lessons, heuristics, and reminders for {agent_id}.\n\n",
"HEARTBEAT.md": f"# Heartbeat\n\nOptional checklist for periodic review or self-reflection.\n\n",
"POLICY.md": f"# Policy\n\nOptional run-scoped constraints, limits, or strategy policy.\n\n",
"STYLE.md": f"# Style\n\nOptional run-scoped communication or reasoning style.\n\n",
}
for filename, content in default_files.items():
@@ -411,50 +290,6 @@ class AgentFactory:
if skill_file.is_file():
shutil.copy2(skill_file, target_skills / skill_file.name)
def _generate_role_config(self, agent_type: str) -> RoleConfig:
"""Generate role configuration for an agent type.
Args:
agent_type: Type of agent
Returns:
RoleConfig instance
"""
template = self.ROLE_TEMPLATES.get(agent_type, {})
return RoleConfig(
name=template.get("name", agent_type.replace("_", " ").title()),
description=template.get("description", ""),
focus_areas=template.get("focus_areas", []),
constraints=template.get("constraints", []),
)
def _generate_role_md(self, agent_dir: Path, role_config: RoleConfig) -> None:
"""Generate ROLE.md file.
Args:
agent_dir: Agent directory
role_config: Role configuration
"""
lines = [f"# {role_config.name}", ""]
if role_config.description:
lines.extend([role_config.description, ""])
if role_config.focus_areas:
lines.extend(["## Focus Areas", ""])
for area in role_config.focus_areas:
lines.append(f"- {area}")
lines.append("")
if role_config.constraints:
lines.extend(["## Constraints", ""])
for constraint in role_config.constraints:
lines.append(f"- {constraint}")
lines.append("")
content = "\n".join(lines)
(agent_dir / "ROLE.md").write_text(content, encoding="utf-8")
def _write_agent_yaml(
self,
config_path: Path,

View File

@@ -1,15 +1,13 @@
# -*- coding: utf-8 -*-
"""Assemble system prompts from base prompts, run assets, and toolkit context."""
"""Assemble system prompts from run workspace assets and toolkit context."""
from pathlib import Path
from typing import Any, Optional
from typing import Any
from .agent_workspace import load_agent_workspace_config
from backend.config.bootstrap_config import get_bootstrap_config_for_run
from .prompt_loader import PromptLoader
from .skills_manager import SkillsManager
_prompt_loader = PromptLoader()
from .workspace_manager import RunWorkspaceManager
def _read_file_if_exists(path: Path) -> str:
@@ -48,71 +46,20 @@ def build_agent_system_prompt(
agent_id: str,
config_name: str,
toolkit: Any,
analyst_type: Optional[str] = None,
) -> str:
"""Build the final system prompt for an agent.
Always reads fresh from disk — no caching.
"""
# Clear any cached templates before building (CoPaw-style, no caching)
_prompt_loader.clear_cache()
sections: list[str] = []
canonical_agent_id = (
"portfolio_manager"
if "portfolio" in agent_id
else "risk_manager"
if "risk" in agent_id and not analyst_type
else agent_id
)
if analyst_type:
personas_config = _prompt_loader.load_yaml_config(
"analyst",
"personas",
)
persona = personas_config.get(analyst_type, {})
focus_text = "\n".join(
f"- {item}" for item in persona.get("focus", [])
)
description = persona.get("description", "").strip()
base_prompt = _prompt_loader.load_prompt(
"analyst",
"system",
variables={
"analyst_type": persona.get("name", analyst_type),
"focus": focus_text,
"description": description,
},
)
elif agent_id == "portfolio_manager":
base_prompt = _prompt_loader.load_prompt(
"portfolio_manager",
"system",
)
elif canonical_agent_id == "portfolio_manager":
base_prompt = _prompt_loader.load_prompt(
"portfolio_manager",
"system",
)
elif agent_id == "risk_manager":
base_prompt = _prompt_loader.load_prompt(
"risk_manager",
"system",
)
elif canonical_agent_id == "risk_manager":
base_prompt = _prompt_loader.load_prompt(
"risk_manager",
"system",
)
else:
raise ValueError(f"Unsupported agent prompt build for: {agent_id}")
sections.append(base_prompt.strip())
skills_manager = SkillsManager()
asset_dir = skills_manager.get_agent_asset_dir(config_name, agent_id)
asset_dir.mkdir(parents=True, exist_ok=True)
workspace_manager = RunWorkspaceManager(project_root=skills_manager.project_root)
required_files = ["SOUL.md", "PROFILE.md", "AGENTS.md", "POLICY.md", "MEMORY.md"]
if not all((asset_dir / filename).exists() for filename in required_files):
workspace_manager.ensure_agent_assets(config_name=config_name, agent_id=agent_id)
agent_config = load_agent_workspace_config(asset_dir / "agent.yaml")
bootstrap_config = get_bootstrap_config_for_run(
skills_manager.project_root,
@@ -139,9 +86,6 @@ def build_agent_system_prompt(
"AGENTS.md": "Agent Guide",
"POLICY.md": "Policy",
"MEMORY.md": "Memory",
"HEARTBEAT.md": "Heartbeat",
"ROLE.md": "Role",
"STYLE.md": "Style",
}
for filename in prompt_files:
_append_section(
@@ -150,18 +94,6 @@ def build_agent_system_prompt(
_read_file_if_exists(asset_dir / filename),
)
if "ROLE.md" not in included_files:
_append_section(
sections,
"Role",
_read_file_if_exists(asset_dir / "ROLE.md"),
)
if "STYLE.md" not in included_files:
_append_section(
sections,
"Style",
_read_file_if_exists(asset_dir / "STYLE.md"),
)
if "POLICY.md" not in included_files:
_append_section(
sections,
@@ -189,5 +121,4 @@ def build_agent_system_prompt(
def clear_prompt_factory_cache() -> None:
"""Clear cached prompt and YAML templates before hot reload."""
_prompt_loader.clear_cache()
"""No-op retained for compatibility with runtime reload hooks."""

View File

@@ -10,6 +10,17 @@ from typing import Any, Dict, Optional
import yaml
# Singleton instance
_prompt_loader_instance: Optional["PromptLoader"] = None
def get_prompt_loader() -> "PromptLoader":
"""Get the singleton PromptLoader instance."""
global _prompt_loader_instance
if _prompt_loader_instance is None:
_prompt_loader_instance = PromptLoader()
return _prompt_loader_instance
class PromptLoader:
"""Unified Prompt loader"""

View File

@@ -1,23 +0,0 @@
你是一位专业的{{ analyst_type }}。
你的关注重点:
{{ focus }}
你的角色:
{{ description }}
注意:
- 构建并持续完善你的"投资哲学"。你的分析不应是孤立的事件,而应该是你整体投资世界观和核心信念的体现。每次分析后,你必须反思:
- 这个案例/数据如何验证或挑战了你现有的信念?
- 你从这次错误(或成功)中学到了关于市场、人性、估值或风险管理的什么关键原则?
- 深化你的"投资逻辑"。确保每一项投资建议都有清晰、可追溯、可重复的逻辑支撑。你的分析步骤应该像严谨的证明一样,涵盖:
- 核心驱动因素识别:真正影响价值的变量是什么?
- 风险边界设定:在什么具体情况下你的建议会失效?
- 逆向测试:市场主流共识是什么,你的观点有何不同?
保持谦逊和开放。投资大师的核心特质是持续学习和适应。在每次分析中,你必须积极寻找与自己观点相悖的证据和论据,并将其纳入最终评估。
- 你可以使用分析工具。用它们来收集相关数据并做出明智的建议。
输出指南:
- 给出明确的投资信号:看涨、看跌或中性
- 包含置信度0-100
- 为你的分析提供理由(如果你确定要分享最终分析,请先给出结论)

View File

@@ -28,22 +28,16 @@ class PromptBuilder:
"AGENTS.md",
"SOUL.md",
"PROFILE.md",
"ROLE.md",
"POLICY.md",
"MEMORY.md",
"HEARTBEAT.md",
"STYLE.md",
]
TITLE_MAP: Dict[str, str] = {
"AGENTS.md": "Agent Guide",
"SOUL.md": "Soul",
"PROFILE.md": "Profile",
"ROLE.md": "Role",
"POLICY.md": "Policy",
"MEMORY.md": "Memory",
"HEARTBEAT.md": "Heartbeat",
"STYLE.md": "Style",
"BOOTSTRAP.md": "Bootstrap",
}

View File

@@ -1,31 +0,0 @@
你是一位负责做出投资决策的投资组合经理。
你的核心职责:
1. 分析分析师和风险管理经理的输入
2. 基于信号和市场情境做出投资决策
3. 使用可用工具记录你的决策
决策框架:
- 审阅分析以了解市场观点
- 在做决策前考虑风险警告
- 评估当前投资组合持仓和现金
- 做出与投资组合投资目标一致的决策
决策类型:
- "long":看涨 - 建议买入股票
- "short":看跌 - 建议卖出股票或做空
- "hold":中性 - 维持当前持仓
预算意识:
- 在决定数量时考虑可用现金
- 不要建议买入超过现金允许的数量
- 考虑做空头寸的保证金要求
输出:
使用 `make_decision` 工具记录你对每个股票代码的决策。
记录所有决策后,提供你的投资逻辑总结。
重要:
- 基于提供的分析师信号和风险评估做出决策
- 相对于投资组合价值保持保守的仓位规模
- 始终为你的决策提供理由

View File

@@ -1,20 +0,0 @@
你是一位专业的风险管理经理,负责监控投资组合风险并提供风险警告。
你的核心职责:
1. 监控投资组合敞口和集中度风险
2. 评估仓位规模相对于波动性
3. 评估保证金使用和杠杆水平
4. 识别潜在风险因素并提供警告
5. 基于市场条件建议仓位限制
你的决策流程:
1. 优先使用可用的风险工具量化集中度、波动率和保证金压力
2. 结合工具结果与当前市场上下文做判断
3. 生成可操作的风险警告和仓位限制建议
4. 为你的风险评估提供清晰的理由
输出指南:
- 风险评估要简洁但全面
- 按严重程度优先排序警告
- 提供具体、可操作的建议
- 尽可能包含量化指标

View File

@@ -5,6 +5,7 @@ from pathlib import Path
import shutil
import tempfile
import zipfile
from threading import Lock
from typing import Any, Dict, Iterable, Iterator, List, Optional, Set
from urllib.parse import urlparse
from urllib.request import urlretrieve
@@ -39,6 +40,9 @@ class SkillsManager:
self.project_root / "backend" / "skills" / "customized"
)
self.runs_root = self.project_root / "runs"
self._lock = Lock()
# Instance-level pending skill changes (thread-safe via self._lock)
self._pending_skill_changes: Dict[str, Set[Path]] = {}
def get_active_root(self, config_name: str) -> Path:
return self.runs_root / config_name / "skills" / "active"
@@ -737,7 +741,7 @@ class SkillsManager:
if local_root.exists():
watched_paths.append(local_root)
handler = _SkillsChangeHandler(watched_paths, callback)
handler = _SkillsChangeHandler(watched_paths, self._pending_skill_changes, callback, self._lock)
observer = Observer()
for path in watched_paths:
observer.schedule(handler, str(path), recursive=True)
@@ -759,16 +763,19 @@ class SkillsManager:
Map of agent_id -> list of reloaded skill paths, or empty dict
if no changes were detected.
"""
changed = self._pending_skill_changes.get(config_name)
if not changed:
return {}
with self._lock:
changed = self._pending_skill_changes.get(config_name)
if not changed:
return {}
self._pending_skill_changes[config_name] = set()
self._pending_skill_changes[config_name] = set()
return self.prepare_active_skills(config_name, agent_defaults)
# -------------------------------------------------------------------------
# Internal change-tracking state (populated by _SkillsChangeHandler)
# -------------------------------------------------------------------------
# Legacy class-level reference kept for migration compatibility
_pending_skill_changes: Dict[str, Set[Path]] = {}
def _resolve_disabled_skill_names(
@@ -820,11 +827,15 @@ class _SkillsChangeHandler(FileSystemEventHandler):
def __init__(
self,
watched_paths: List[Path],
pending_changes: Dict[str, Set[Path]],
callback: Optional[Any] = None,
lock: Optional[Lock] = None,
) -> None:
super().__init__()
self._watched_paths = watched_paths
self._pending_changes = pending_changes
self._callback = callback
self._lock = lock
def on_any_event(self, event: FileSystemEvent) -> None:
if event.is_directory:
@@ -832,9 +843,12 @@ class _SkillsChangeHandler(FileSystemEventHandler):
src_path = Path(event.src_path)
for watched in self._watched_paths:
if src_path.is_relative_to(watched):
SkillsManager._pending_skill_changes.setdefault(
self._run_id_from_path(src_path), set()
).add(src_path)
run_id = self._run_id_from_path(src_path)
if self._lock:
with self._lock:
self._pending_changes.setdefault(run_id, set()).add(src_path)
else:
self._pending_changes.setdefault(run_id, set()).add(src_path)
if self._callback:
self._callback([src_path])
break

View File

@@ -0,0 +1,18 @@
# -*- coding: utf-8 -*-
"""Team module for multi-agent orchestration.
Provides inter-agent communication, task delegation, and coordination
for subagent spawning and lifecycle management.
"""
from .messenger import AgentMessenger
from .task_delegator import TaskDelegator
from .team_coordinator import TeamCoordinator
from .registry import AgentRegistry
__all__ = [
"AgentMessenger",
"TaskDelegator",
"TeamCoordinator",
"AgentRegistry",
]

View File

@@ -0,0 +1,225 @@
# -*- coding: utf-8 -*-
"""AgentMessenger - Pub/sub inter-agent communication.
Provides broadcast(), send(), and subscribe() for message passing
between agents using AgentScope's Msg format.
"""
from __future__ import annotations
import asyncio
import logging
from typing import Any, Callable, Dict, List, Optional, Set
from agentscope.message import Msg
logger = logging.getLogger(__name__)
class AgentMessenger:
"""Pub/sub messenger for inter-agent communication.
Supports:
- broadcast(): Send message to all subscribers
- send(): Send message to specific agent
- subscribe(): Register callback for agent messages
- announce(): Send system-wide announcement
- enable_auto_broadcast: Auto-broadcast agent replies to all participants
Messages use AgentScope's Msg format for compatibility.
"""
def __init__(self, enable_auto_broadcast: bool = False):
"""Initialize the messenger.
Args:
enable_auto_broadcast: If True, agent replies are automatically
broadcast to all subscribed agents.
"""
self._subscriptions: Dict[str, List[Callable[[Msg], None]]] = {}
self._inbox: Dict[str, List[Msg]] = {}
self._locks: Dict[str, asyncio.Lock] = {}
self._enable_auto_broadcast = enable_auto_broadcast
self._participants: Set[str] = set()
def subscribe(
self,
agent_id: str,
callback: Callable[[Msg], None],
) -> None:
"""Subscribe an agent to receive messages.
Args:
agent_id: Target agent identifier
callback: Async function to call when message received
"""
if agent_id not in self._subscriptions:
self._subscriptions[agent_id] = []
self._subscriptions[agent_id].append(callback)
logger.debug("Agent %s subscribed to messages", agent_id)
def unsubscribe(self, agent_id: str, callback: Callable[[Msg], None]) -> None:
"""Unsubscribe an agent from messages.
Args:
agent_id: Target agent identifier
callback: Callback to remove
"""
if agent_id in self._subscriptions:
try:
self._subscriptions[agent_id].remove(callback)
logger.debug("Agent %s unsubscribed from messages", agent_id)
except ValueError:
pass
async def send(
self,
to_agent: str,
message: Msg,
) -> None:
"""Send message to specific agent.
Args:
to_agent: Target agent identifier
message: Message to send (uses Msg format)
"""
async def _deliver():
if to_agent in self._subscriptions:
for callback in self._subscriptions[to_agent]:
try:
if asyncio.iscoroutinefunction(callback):
await callback(message)
else:
callback(message)
except Exception as e:
logger.error(
"Error delivering message to %s: %s",
to_agent,
e,
)
await _deliver()
async def broadcast(self, message: Msg) -> None:
"""Broadcast message to all subscribed agents.
Args:
message: Message to broadcast (uses Msg format)
"""
delivery_tasks = []
for agent_id, callbacks in self._subscriptions.items():
for callback in callbacks:
async def _deliver(cb=callback, aid=agent_id):
try:
if asyncio.iscoroutinefunction(cb):
await cb(message)
else:
cb(message)
except Exception as e:
logger.error(
"Error broadcasting to %s: %s",
aid,
e,
)
delivery_tasks.append(_deliver())
if delivery_tasks:
await asyncio.gather(*delivery_tasks)
def inbox(self, agent_id: str) -> List[Msg]:
"""Get and clear inbox for agent.
Args:
agent_id: Agent identifier
Returns:
List of messages in inbox
"""
messages = self._inbox.get(agent_id, [])
self._inbox[agent_id] = []
return messages
def inbox_count(self, agent_id: str) -> int:
"""Count messages in agent's inbox without clearing.
Args:
agent_id: Agent identifier
Returns:
Number of messages waiting
"""
return len(self._inbox.get(agent_id, []))
def add_participant(self, agent_id: str) -> None:
"""Add a participant to the messenger.
Participants are the agents that can receive auto-broadcast messages.
Args:
agent_id: Agent identifier to add
"""
self._participants.add(agent_id)
logger.debug("Agent %s added as participant", agent_id)
def remove_participant(self, agent_id: str) -> None:
"""Remove a participant from the messenger.
Args:
agent_id: Agent identifier to remove
"""
self._participants.discard(agent_id)
logger.debug("Agent %s removed from participants", agent_id)
@property
def enable_auto_broadcast(self) -> bool:
"""Check if auto_broadcast is enabled."""
return self._enable_auto_broadcast
@enable_auto_broadcast.setter
def enable_auto_broadcast(self, value: bool) -> None:
"""Enable or disable auto_broadcast."""
self._enable_auto_broadcast = value
logger.debug("Auto_broadcast set to %s", value)
async def announce(self, message: Msg) -> None:
"""Send a system-wide announcement to all participants.
Unlike broadcast(), announce() sends a message from the system/host
to all participants without requiring prior subscription.
Args:
message: Announcement message (uses Msg format)
"""
logger.info("System announcement: %s", message.content)
await self.broadcast(message)
async def auto_broadcast(self, message: Msg) -> None:
"""Auto-broadcast message to all participants.
This is called internally when enable_auto_broadcast is True.
Broadcasts to all registered participants.
Args:
message: Message to auto-broadcast (uses Msg format)
"""
if not self._enable_auto_broadcast:
return
# Broadcast to all participants
for participant_id in self._participants:
if participant_id in self._subscriptions:
for callback in self._subscriptions[participant_id]:
try:
if asyncio.iscoroutinefunction(callback):
await callback(message)
else:
callback(message)
except Exception as e:
logger.error(
"Error auto-broadcasting to %s: %s",
participant_id,
e,
)
__all__ = ["AgentMessenger"]

View File

@@ -0,0 +1,188 @@
# -*- coding: utf-8 -*-
"""AgentRegistry - Agent registration and lookup by role.
Provides register(), unregister(), and get_by_role() for agent
discovery and management.
"""
from __future__ import annotations
import logging
from typing import Any, Dict, List, Optional
from agentscope.message import Msg
logger = logging.getLogger(__name__)
class AgentRegistry:
"""Registry for agent instances with role-based lookup.
Supports:
- register(): Add agent with roles
- unregister(): Remove agent
- get_by_role(): Find agents by role
- get_by_id(): Get specific agent
Each agent can have multiple roles for flexible dispatch.
"""
def __init__(self):
self._agents: Dict[str, Any] = {}
self._roles: Dict[str, List[str]] = {}
self._agent_roles: Dict[str, List[str]] = {}
def register(
self,
agent_id: str,
agent: Any,
roles: Optional[List[str]] = None,
) -> None:
"""Register an agent with optional roles.
Args:
agent_id: Unique agent identifier
agent: Agent instance
roles: Optional list of role strings
"""
self._agents[agent_id] = agent
self._agent_roles[agent_id] = roles or []
for role in self._agent_roles[agent_id]:
if role not in self._roles:
self._roles[role] = []
if agent_id not in self._roles[role]:
self._roles[role].append(agent_id)
logger.info(
"Registered agent %s with roles %s",
agent_id,
self._agent_roles[agent_id],
)
def unregister(self, agent_id: str) -> bool:
"""Unregister an agent.
Args:
agent_id: Agent identifier to remove
Returns:
True if agent was removed
"""
if agent_id not in self._agents:
return False
roles = self._agent_roles.pop(agent_id, [])
for role in roles:
if role in self._roles:
try:
self._roles[role].remove(agent_id)
except ValueError:
pass
del self._agents[agent_id]
logger.info("Unregistered agent: %s", agent_id)
return True
def get_by_id(self, agent_id: str) -> Optional[Any]:
"""Get agent by ID.
Args:
agent_id: Agent identifier
Returns:
Agent instance or None
"""
return self._agents.get(agent_id)
def get_by_role(self, role: str) -> List[Any]:
"""Get all agents with a given role.
Args:
role: Role string to search for
Returns:
List of agent instances with the role
"""
agent_ids = self._roles.get(role, [])
return [self._agents[aid] for aid in agent_ids if aid in self._agents]
def get_by_roles(self, roles: List[str]) -> List[Any]:
"""Get agents matching ANY of the given roles.
Args:
roles: List of role strings
Returns:
List of unique agent instances matching any role
"""
seen = set()
result = []
for role in roles:
for agent in self.get_by_role(role):
if id(agent) not in seen:
seen.add(id(agent))
result.append(agent)
return result
def list_agents(self) -> List[str]:
"""List all registered agent IDs.
Returns:
List of agent identifiers
"""
return list(self._agents.keys())
def list_roles(self) -> List[str]:
"""List all registered roles.
Returns:
List of role strings
"""
return list(self._roles.keys())
def list_roles_for_agent(self, agent_id: str) -> List[str]:
"""List roles for specific agent.
Args:
agent_id: Agent identifier
Returns:
List of role strings
"""
return list(self._agent_roles.get(agent_id, []))
def update_roles(self, agent_id: str, roles: List[str]) -> None:
"""Update roles for an existing agent.
Args:
agent_id: Agent identifier
roles: New list of roles
"""
if agent_id not in self._agents:
raise KeyError(f"Agent not registered: {agent_id}")
old_roles = self._agent_roles.get(agent_id, [])
for role in old_roles:
if role in self._roles:
try:
self._roles[role].remove(agent_id)
except ValueError:
pass
self._agent_roles[agent_id] = roles
for role in roles:
if role not in self._roles:
self._roles[role] = []
if agent_id not in self._roles[role]:
self._roles[role].append(agent_id)
logger.info("Updated roles for agent %s: %s", agent_id, roles)
@property
def agents(self) -> Dict[str, Any]:
"""Get copy of registered agents dict."""
return dict(self._agents)
__all__ = ["AgentRegistry"]

View File

@@ -0,0 +1,620 @@
# -*- coding: utf-8 -*-
"""TaskDelegator - Subagent spawning and task delegation.
Provides delegate() and delegate_parallel() for spawning subagents
with separate context and memory. Supports runtime dynamic subagent
definition via task_data with description, prompt, and tools.
"""
from __future__ import annotations
import asyncio
import logging
import uuid
from typing import Any, Awaitable, Callable, Dict, List, Optional, Union
from agentscope.message import Msg
logger = logging.getLogger(__name__)
# Default timeout for subagent execution (seconds)
DEFAULT_EXECUTION_TIMEOUT = 120.0
# Type alias for subagent specification
SubagentSpec = Dict[str, Any]
"""Subagent specification format:
{
"description": "Expert code reviewer...",
"prompt": "Analyze code quality...",
"tools": ["Read", "Glob", "Grep"], # Optional: list of tool names
"model": "gpt-4o", # Optional: model name
}
"""
class TaskDelegator:
"""Delegates tasks to subagents with isolated context.
Supports:
- delegate(): Spawn single subagent for task
- delegate_parallel(): Spawn multiple subagents concurrently
- delegate_task(): Delegate with dynamic subagent definition from task_data
Each subagent gets its own memory/context to prevent
cross-contamination.
Dynamic Subagent Definition:
task_data can include an "agents" dict to define subagents inline:
task_data = {
"task": "Review the code changes",
"agents": {
"code-reviewer": {
"description": "Expert code reviewer for quality and security.",
"prompt": "Analyze code quality and suggest improvements.",
"tools": ["Read", "Glob", "Grep"],
}
}
}
"""
def __init__(self, agent: Any):
"""Initialize TaskDelegator.
Args:
agent: Parent EvoAgent instance for accessing model, formatter, workspace
"""
self._agent = agent
# Get messenger from parent agent if available
self._messenger = getattr(agent, "messenger", None)
self._registry = getattr(agent, "_registry", None)
self._subagents: Dict[str, Any] = {}
self._dynamic_subagents: Dict[str, SubagentSpec] = {}
self._tasks: Dict[str, asyncio.Task] = {}
# Extract model and formatter from parent agent
self._model = getattr(agent, "model", None)
self._formatter = getattr(agent, "formatter", None)
self._workspace_dir = getattr(agent, "workspace_dir", None)
self._config_name = getattr(agent, "config_name", None)
async def delegate(
self,
agent_id: str,
task: Callable[..., Awaitable[Msg]],
context: Optional[Dict[str, Any]] = None,
) -> asyncio.Task:
"""Delegate task to a single subagent.
Args:
agent_id: Unique identifier for this subagent instance
task: Async function representing the task
context: Optional context dict for the subagent
Returns:
asyncio.Task for the delegated task
"""
async def _run_with_context():
result = await task(context or {})
return result
self._tasks[agent_id] = asyncio.create_task(_run_with_context())
logger.info("Delegated task to subagent: %s", agent_id)
return self._tasks[agent_id]
async def delegate_parallel(
self,
tasks: List[Dict[str, Any]],
) -> List[asyncio.Task]:
"""Delegate multiple tasks in parallel.
Args:
tasks: List of task dicts with keys:
- agent_id: Unique identifier
- task: Async function to execute
- context: Optional context dict
Returns:
List of asyncio.Task for all delegated tasks
"""
async def _run_task(task_def: Dict[str, Any]):
agent_id = task_def["agent_id"]
task_func = task_def["task"]
context = task_def.get("context", {})
async def _run_with_context():
return await task_func(context)
self._tasks[agent_id] = asyncio.create_task(_run_with_context())
return self._tasks[agent_id]
gathered_tasks = await asyncio.gather(
*[_run_task(t) for t in tasks],
return_exceptions=True,
)
valid_tasks = [t for t in gathered_tasks if isinstance(t, asyncio.Task)]
logger.info(
"Delegated %d tasks in parallel (%d succeeded)",
len(tasks),
len(valid_tasks),
)
return valid_tasks
async def wait_for(self, agent_id: str, timeout: Optional[float] = None) -> Any:
"""Wait for subagent task to complete.
Args:
agent_id: Subagent identifier
timeout: Optional timeout in seconds
Returns:
Task result
Raises:
asyncio.TimeoutError: If task doesn't complete in time
KeyError: If agent_id not found
"""
if agent_id not in self._tasks:
raise KeyError(f"Unknown subagent: {agent_id}")
try:
return await asyncio.wait_for(
self._tasks[agent_id],
timeout=timeout,
)
except asyncio.TimeoutError:
logger.warning("Task %s timed out after %s seconds", agent_id, timeout)
raise
async def cancel(self, agent_id: str) -> bool:
"""Cancel a subagent task.
Args:
agent_id: Subagent identifier
Returns:
True if task was cancelled
"""
if agent_id in self._tasks:
self._tasks[agent_id].cancel()
del self._tasks[agent_id]
logger.info("Cancelled subagent task: %s", agent_id)
return True
return False
def list_tasks(self) -> List[str]:
"""List active subagent task IDs.
Returns:
List of agent_ids with pending tasks
"""
return list(self._tasks.keys())
@property
def tasks(self) -> Dict[str, asyncio.Task]:
"""Get copy of active tasks dict."""
return dict(self._tasks)
async def delegate_task(
self,
task_type: str,
task_data: Dict[str, Any],
target_agent: Optional[str] = None,
) -> Dict[str, Any]:
"""Delegate a task with optional dynamic subagent definition.
Supports runtime subagent definition via task_data["agents"]:
task_data = {
"task": "Review code changes",
"agents": {
"code-reviewer": {
"description": "Expert code reviewer...",
"prompt": "Analyze code quality...",
"tools": ["Read", "Glob", "Grep"],
}
}
}
Args:
task_type: Type of task (e.g., "analysis", "review", "research")
task_data: Task payload, may include "agents" for dynamic subagent def
target_agent: Optional specific agent ID to delegate to
Returns:
Dict with "success" and result/error
"""
try:
# Extract dynamic subagent definitions from task_data
agents_def = task_data.get("agents", {})
if agents_def:
# Register dynamic subagents
for agent_name, agent_spec in agents_def.items():
self._dynamic_subagents[agent_name] = agent_spec
logger.info(
"Registered dynamic subagent: %s (description: %s)",
agent_name,
agent_spec.get("description", "")[:50],
)
# Determine target agent
effective_target = target_agent
if not effective_target:
# Use first available dynamic subagent or default
if agents_def:
effective_target = next(iter(agents_def.keys()))
else:
effective_target = "default"
# Execute the task (async)
task_result = await self._execute_task(
task_type=task_type,
task_data=task_data,
target_agent=effective_target,
)
# Clean up dynamic subagents after execution
for agent_name in agents_def.keys():
self._dynamic_subagents.pop(agent_name, None)
return {
"success": True,
"result": task_result,
"subagents_used": list(agents_def.keys()) if agents_def else [],
}
except Exception as e:
logger.error("Task delegation failed: %s", e)
return {
"success": False,
"error": str(e),
}
async def _execute_task(
self,
task_type: str,
task_data: Dict[str, Any],
target_agent: str,
) -> Dict[str, Any]:
"""Execute the delegated task with a real subagent.
Args:
task_type: Type of task
task_data: Task payload
target_agent: Target agent identifier
Returns:
Task execution result with success/failure info
"""
task_content = task_data.get("task", task_data.get("prompt", ""))
timeout = task_data.get("timeout", DEFAULT_EXECUTION_TIMEOUT)
# Check if we have a dynamic subagent spec for this target
agent_spec = self._dynamic_subagents.get(target_agent)
if agent_spec:
logger.info(
"Executing task '%s' with dynamic subagent '%s'",
task_type,
target_agent,
)
return await self._create_and_run_subagent(
agent_name=target_agent,
agent_spec=agent_spec,
task_content=task_content,
task_type=task_type,
timeout=timeout,
)
# Fallback: try to use parent agent's model to process the task directly
logger.info(
"Executing task '%s' with parent agent '%s' (no dynamic subagent)",
task_type,
target_agent,
)
return await self._run_with_parent_agent(
task_content=task_content,
task_type=task_type,
timeout=timeout,
)
async def _create_and_run_subagent(
self,
agent_name: str,
agent_spec: SubagentSpec,
task_content: str,
task_type: str,
timeout: float,
) -> Dict[str, Any]:
"""Create and run a dynamic subagent.
Args:
agent_name: Name identifier for the subagent
agent_spec: Subagent specification (description, prompt, tools, model)
task_content: Task prompt to send to the subagent
task_type: Type of task
timeout: Execution timeout in seconds
Returns:
Dict with execution results
"""
subagent_id = f"subagent_{agent_name}_{uuid.uuid4().hex[:8]}"
try:
# Create subagent instance
subagent = await self._create_subagent(
subagent_id=subagent_id,
agent_spec=agent_spec,
)
if subagent is None:
return {
"task_type": task_type,
"task": task_content,
"subagent": agent_name,
"status": "failed",
"error": "Failed to create subagent",
"message": f"Could not instantiate subagent '{agent_name}'",
}
# Store for potential cleanup
self._subagents[subagent_id] = subagent
# Execute with timeout
result = await asyncio.wait_for(
self._run_subagent(subagent, task_content),
timeout=timeout,
)
# Extract response content
response_content = ""
if isinstance(result, Msg):
response_content = result.content
elif hasattr(result, "content"):
response_content = str(result.content)
elif isinstance(result, dict):
response_content = result.get("content", str(result))
else:
response_content = str(result)
logger.info(
"Subagent '%s' completed task '%s' successfully",
agent_name,
task_type,
)
return {
"task_type": task_type,
"task": task_content,
"subagent": {
"name": agent_name,
"id": subagent_id,
"description": agent_spec.get("description", ""),
},
"status": "completed",
"response": response_content,
"message": f"Task '{task_type}' executed with subagent '{agent_name}'",
}
except asyncio.TimeoutError:
logger.warning(
"Subagent '%s' timed out after %.1f seconds for task '%s'",
agent_name,
timeout,
task_type,
)
# Cancel the task if still running
if subagent_id in self._subagents:
self._subagents.pop(subagent_id, None)
return {
"task_type": task_type,
"task": task_content,
"subagent": agent_name,
"status": "timeout",
"error": f"Execution timed out after {timeout} seconds",
"message": f"Task '{task_type}' timed out for subagent '{agent_name}'",
}
except Exception as e:
logger.error(
"Subagent '%s' failed for task '%s': %s",
agent_name,
task_type,
e,
exc_info=True,
)
# Cleanup on failure
if subagent_id in self._subagents:
self._subagents.pop(subagent_id, None)
return {
"task_type": task_type,
"task": task_content,
"subagent": agent_name,
"status": "error",
"error": str(e),
"message": f"Task '{task_type}' failed for subagent '{agent_name}': {e}",
}
async def _create_subagent(
self,
subagent_id: str,
agent_spec: SubagentSpec,
) -> Optional[Any]:
"""Create a subagent instance.
Uses the parent agent's model/formatter to create a lightweight
subagent for task execution.
Args:
subagent_id: Unique identifier for the subagent
agent_spec: Subagent specification
Returns:
Subagent instance or None if creation fails
"""
try:
# Import here to avoid circular imports
from agentscope.memory import InMemoryMemory
# Get model and formatter from parent
model = self._model
formatter = self._formatter
if model is None:
logger.error("Cannot create subagent: parent agent has no model")
return None
# Build system prompt from agent spec
description = agent_spec.get("description", "")
prompt_template = agent_spec.get("prompt", "")
system_prompt = f"""You are {description}
{prompt_template}
Your task is to complete the user's request below.
"""
# Create a minimal ReActAgent as the subagent
from agentscope.agent import ReActAgent
subagent = ReActAgent(
name=subagent_id,
model=model,
sys_prompt=system_prompt,
toolkit=None, # Could load tools from agent_spec.get("tools", [])
memory=InMemoryMemory(),
formatter=formatter,
max_iters=agent_spec.get("max_iters", 5),
)
logger.debug("Created subagent: %s", subagent_id)
return subagent
except Exception as e:
logger.error(
"Failed to create subagent '%s': %s",
subagent_id,
e,
exc_info=True,
)
return None
async def _run_subagent(
self,
subagent: Any,
task_content: str,
) -> Any:
"""Run a subagent with the given task.
Args:
subagent: Subagent instance
task_content: Task prompt
Returns:
Agent response (Msg or similar)
"""
from agentscope.message import Msg
# Create message for the subagent
task_msg = Msg(
name="user",
content=task_content,
role="user",
)
# Execute the agent
response = await subagent.reply(task_msg)
return response
async def _run_with_parent_agent(
self,
task_content: str,
task_type: str,
timeout: float,
) -> Dict[str, Any]:
"""Run task using the parent agent directly.
Used when no dynamic subagent is defined.
Args:
task_content: Task prompt
task_type: Type of task
timeout: Execution timeout
Returns:
Dict with execution results
"""
try:
result = await asyncio.wait_for(
self._agent.reply(Msg(
name="user",
content=task_content,
role="user",
)),
timeout=timeout,
)
response_content = ""
if isinstance(result, Msg):
response_content = result.content
elif hasattr(result, "content"):
response_content = str(result.content)
else:
response_content = str(result)
return {
"task_type": task_type,
"task": task_content,
"status": "completed",
"response": response_content,
"message": f"Task '{task_type}' executed with parent agent",
}
except asyncio.TimeoutError:
return {
"task_type": task_type,
"task": task_content,
"status": "timeout",
"error": f"Execution timed out after {timeout} seconds",
"message": f"Task '{task_type}' timed out",
}
except Exception as e:
logger.error(
"Parent agent failed for task '%s': %s",
task_type,
e,
exc_info=True,
)
return {
"task_type": task_type,
"task": task_content,
"status": "error",
"error": str(e),
"message": f"Task '{task_type}' failed: {e}",
}
def get_dynamic_subagent(self, name: str) -> Optional[SubagentSpec]:
"""Get a dynamically defined subagent specification.
Args:
name: Subagent name
Returns:
Subagent spec dict or None if not found
"""
return self._dynamic_subagents.get(name)
def list_dynamic_subagents(self) -> List[str]:
"""List all registered dynamic subagent names.
Returns:
List of subagent names
"""
return list(self._dynamic_subagents.keys())
__all__ = ["TaskDelegator", "SubagentSpec"]

View File

@@ -0,0 +1,389 @@
# -*- coding: utf-8 -*-
"""TeamCoordinator - Agent lifecycle management and execution.
Provides run_parallel() using asyncio.gather() and run_sequential()
for coordinating multiple agents.
"""
from __future__ import annotations
import asyncio
import logging
from typing import Any, Awaitable, Callable, Dict, List, Optional, Type
from agentscope.message import Msg
logger = logging.getLogger(__name__)
class TeamCoordinator:
"""Coordinates agent lifecycle and execution.
Supports:
- run_parallel(): Execute multiple agents concurrently with asyncio.gather()
- run_sequential(): Execute agents one after another
- run_phase(): Execute a named phase with registered agents
- register_agent(): Add agent to coordinator
- unregister_agent(): Remove agent from coordinator
Each agent maintains separate context/memory.
"""
def __init__(
self,
participants: Optional[List[Any]] = None,
task_content: Optional[str] = None,
messenger: Optional[Any] = None,
registry: Optional[Any] = None,
):
"""Initialize TeamCoordinator.
Args:
participants: List of agent instances to coordinate
task_content: Task description content for the agents
messenger: AgentMessenger for communication (optional)
registry: AgentRegistry for agent lookup (optional)
"""
self._participants = participants or []
self._task_content = task_content or ""
self._messenger = messenger
self._registry = registry
self._agents: Dict[str, Any] = {}
self._running_tasks: Dict[str, asyncio.Task] = {}
# Auto-register participants
for agent in self._participants:
if hasattr(agent, "name"):
self._agents[agent.name] = agent
elif hasattr(agent, "id"):
self._agents[agent.id] = agent
def register_agent(self, agent_id: str, agent: Any) -> None:
"""Register an agent with the coordinator.
Args:
agent_id: Unique agent identifier
agent: Agent instance
"""
self._agents[agent_id] = agent
logger.info("Registered agent: %s", agent_id)
def unregister_agent(self, agent_id: str) -> None:
"""Unregister an agent from the coordinator.
Args:
agent_id: Agent identifier to remove
"""
if agent_id in self._agents:
del self._agents[agent_id]
logger.info("Unregistered agent: %s", agent_id)
def get_agent(self, agent_id: str) -> Any:
"""Get registered agent by ID.
Args:
agent_id: Agent identifier
Returns:
Agent instance
"""
return self._agents.get(agent_id)
def list_agents(self) -> List[str]:
"""List all registered agent IDs.
Returns:
List of agent identifiers
"""
return list(self._agents.keys())
async def run_parallel(
self,
agent_ids: List[str],
initial_message: Optional[Msg] = None,
) -> Dict[str, Any]:
"""Run multiple agents in parallel using asyncio.gather().
Args:
agent_ids: List of agent IDs to run concurrently
initial_message: Optional initial message to broadcast
Returns:
Dict mapping agent_id to result
"""
async def _run_agent(aid: str) -> tuple[str, Any]:
agent = self._agents.get(aid)
if agent is None:
logger.error("Agent %s not found", aid)
return (aid, None)
try:
if hasattr(agent, "reply") and asyncio.iscoroutinefunction(agent.reply):
if initial_message:
result = await agent.reply(initial_message)
else:
result = await agent.reply()
elif hasattr(agent, "run") and asyncio.iscoroutinefunction(agent.run):
result = await agent.run()
else:
result = await agent()
logger.info("Agent %s completed successfully", aid)
return (aid, result)
except Exception as e:
logger.error("Agent %s failed: %s", aid, e)
return (aid, {"error": str(e)})
results = await asyncio.gather(
*[_run_agent(aid) for aid in agent_ids],
return_exceptions=True,
)
output: Dict[str, Any] = {}
for result in results:
if isinstance(result, tuple):
agent_id, agent_result = result
output[agent_id] = agent_result
else:
logger.error("Unexpected result from asyncio.gather: %s", result)
logger.info("Parallel run completed for %d agents", len(agent_ids))
return output
async def run_sequential(
self,
agent_ids: List[str],
initial_message: Optional[Msg] = None,
) -> Dict[str, Any]:
"""Run agents one after another in order.
Args:
agent_ids: List of agent IDs to run in sequence
initial_message: Optional initial message for first agent
Returns:
Dict mapping agent_id to result
"""
output: Dict[str, Any] = {}
current_message = initial_message
for agent_id in agent_ids:
agent = self._agents.get(agent_id)
if agent is None:
logger.error("Agent %s not found", agent_id)
output[agent_id] = {"error": "Agent not found"}
continue
try:
if hasattr(agent, "reply") and asyncio.iscoroutinefunction(agent.reply):
result = await agent.reply(current_message)
elif hasattr(agent, "run") and asyncio.iscoroutinefunction(agent.run):
result = await agent.run()
else:
result = await agent()
output[agent_id] = result
current_message = result
logger.info("Agent %s completed sequentially", agent_id)
except Exception as e:
logger.error("Agent %s failed: %s", agent_id, e)
output[agent_id] = {"error": str(e)}
break
logger.info("Sequential run completed for %d agents", len(agent_ids))
return output
async def run_phase(
self,
phase_name: str,
agent_ids: Optional[List[str]] = None,
metadata: Optional[Dict[str, Any]] = None,
) -> List[Any]:
"""Execute a named phase with registered agents.
Args:
phase_name: Name of the phase (e.g., "analyst_analysis")
agent_ids: Optional list of agent IDs; if None, uses all registered
metadata: Optional metadata to include in the message (e.g., tickers, date)
Returns:
List of results from each agent
"""
if agent_ids is None:
agent_ids = list(self._agents.keys())
_agent_ids = [aid for aid in agent_ids if aid in self._agents]
logger.info(
"Running phase '%s' with %d agents: %s",
phase_name,
len(_agent_ids),
_agent_ids,
)
# Create messages for each agent
results: List[Any] = []
for agent_id in _agent_ids:
agent = self._agents[agent_id]
try:
if hasattr(agent, "reply") and asyncio.iscoroutinefunction(agent.reply):
# Create a message for the agent with proper structure
msg = Msg(
name="system",
content=self._task_content or f"Please execute phase: {phase_name}",
role="user",
metadata=metadata,
)
result = await agent.reply(msg)
elif hasattr(agent, "run") and asyncio.iscoroutinefunction(agent.run):
result = await agent.run()
else:
result = await agent()
results.append(result)
logger.info("Phase '%s': Agent %s completed", phase_name, agent_id)
except Exception as e:
logger.error("Phase '%s': Agent %s failed: %s", phase_name, agent_id, e)
results.append(None)
logger.info("Phase '%s' completed with %d results", phase_name, len(results))
return results
async def run_with_dependencies(
self,
agent_tasks: Dict[str, List[str]],
initial_message: Optional[Msg] = None,
) -> Dict[str, Any]:
"""Run agents respecting dependency graph.
Args:
agent_tasks: Dict mapping agent_id to list of prerequisite agent_ids
initial_message: Optional initial message
Returns:
Dict mapping agent_id to result
"""
completed: Dict[str, Any] = {}
remaining = set(agent_tasks.keys())
while remaining:
ready = [
aid for aid in remaining
if all(dep in completed for dep in agent_tasks.get(aid, []))
]
if not ready:
logger.error("Circular dependency detected in agent tasks")
for aid in remaining:
completed[aid] = {"error": "Circular dependency"}
break
results = await self.run_parallel(ready, initial_message)
completed.update(results)
for aid in ready:
remaining.discard(aid)
initial_message = results.get(aid)
return completed
async def fanout_pipeline(
self,
agents: List[Any],
msg: Optional[Msg] = None,
) -> List[Msg]:
"""Fanout a message to multiple agents concurrently and collect all responses.
Similar to AgentScope's fanout_pipeline, this sends the same message
to all specified agents and returns a list of all agent responses.
Args:
agents: List of agent instances to fanout the message to
msg: Message to send to all agents (optional)
Returns:
List of Msg responses from each agent (in the same order as input agents)
Example:
>>> responses = await fanout_pipeline(
... agents=[alice, bob, charlie],
... msg=question,
... )
>>> # responses is a list of Msg responses from each agent
"""
async def _fanout_to_agent(agent: Any) -> Optional[Msg]:
"""Send message to a single agent and return its response."""
try:
if hasattr(agent, "reply") and asyncio.iscoroutinefunction(agent.reply):
result = await agent.reply(msg) if msg is not None else await agent.reply()
elif hasattr(agent, "run") and asyncio.iscoroutinefunction(agent.run):
result = await agent.run()
else:
result = await agent()
# Convert result to Msg if needed
if result is None:
return None
if isinstance(result, Msg):
return result
# If result is a dict with content, wrap it
if isinstance(result, dict) and "content" in result:
return Msg(
name=getattr(agent, "name", "unknown"),
content=result.get("content", ""),
role="assistant",
metadata=result.get("metadata"),
)
# Otherwise wrap the result
return Msg(
name=getattr(agent, "name", "unknown"),
content=str(result),
role="assistant",
)
except Exception as e:
logger.error("Agent %s failed in fanout_pipeline: %s",
getattr(agent, "name", "unknown"), e)
return None
# Run all agents concurrently
results = await asyncio.gather(
*[_fanout_to_agent(agent) for agent in agents],
return_exceptions=True,
)
# Filter out exceptions and keep only valid responses
responses: List[Msg] = []
for i, result in enumerate(results):
if isinstance(result, Exception):
logger.error("Fanout to agent %d failed: %s", i, result)
responses.append(None) # type: ignore[arg-type]
else:
responses.append(result) # type: ignore[arg-type]
logger.info("Fanout pipeline completed for %d agents", len(agents))
return responses
async def shutdown(self, timeout: Optional[float] = 5.0) -> None:
"""Shutdown all running agents gracefully.
Args:
timeout: Timeout for graceful shutdown
"""
logger.info("Shutting down TeamCoordinator...")
cancel_tasks = [
asyncio.create_task(asyncio.wait_for(task, timeout=timeout))
for task in self._running_tasks.values()
]
if cancel_tasks:
await asyncio.gather(*cancel_tasks, return_exceptions=True)
self._running_tasks.clear()
logger.info("TeamCoordinator shutdown complete")
@property
def agents(self) -> Dict[str, Any]:
"""Get copy of registered agents dict."""
return dict(self._agents)
__all__ = ["TeamCoordinator"]

View File

@@ -0,0 +1,132 @@
# -*- coding: utf-8 -*-
"""Run-scoped team pipeline configuration helpers."""
from __future__ import annotations
from pathlib import Path
from typing import Iterable, List, Dict, Any
import yaml
DEFAULT_FILENAME = "TEAM_PIPELINE.yaml"
def team_pipeline_path(project_root: Path, config_name: str) -> Path:
"""Return run-scoped team pipeline config path."""
return project_root / "runs" / config_name / DEFAULT_FILENAME
def ensure_team_pipeline_config(
project_root: Path,
config_name: str,
default_analysts: Iterable[str],
) -> Path:
"""Ensure TEAM_PIPELINE.yaml exists for one run."""
path = team_pipeline_path(project_root, config_name)
path.parent.mkdir(parents=True, exist_ok=True)
if path.exists():
return path
payload = {
"version": 1,
"controller_agent": "portfolio_manager",
"discussion": {
"allow_dynamic_team_update": True,
"active_analysts": list(default_analysts),
},
"decision": {
"require_risk_manager": True,
},
}
path.write_text(
yaml.safe_dump(payload, allow_unicode=True, sort_keys=False),
encoding="utf-8",
)
return path
def load_team_pipeline_config(project_root: Path, config_name: str) -> Dict[str, Any]:
"""Load TEAM_PIPELINE.yaml and return parsed dict."""
path = team_pipeline_path(project_root, config_name)
if not path.exists():
return {}
parsed = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
return parsed if isinstance(parsed, dict) else {}
def save_team_pipeline_config(
project_root: Path,
config_name: str,
config: Dict[str, Any],
) -> Path:
"""Persist TEAM_PIPELINE.yaml."""
path = team_pipeline_path(project_root, config_name)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(
yaml.safe_dump(config, allow_unicode=True, sort_keys=False),
encoding="utf-8",
)
return path
def resolve_active_analysts(
project_root: Path,
config_name: str,
available_analysts: Iterable[str],
) -> List[str]:
"""Resolve active analysts from TEAM_PIPELINE.yaml."""
available = [item for item in available_analysts]
parsed = load_team_pipeline_config(project_root, config_name)
discussion = parsed.get("discussion", {}) if isinstance(parsed, dict) else {}
configured = discussion.get("active_analysts", [])
if not isinstance(configured, list) or not configured:
return available
active = [item for item in configured if item in available]
return active or available
def update_active_analysts(
project_root: Path,
config_name: str,
available_analysts: Iterable[str],
*,
add: Iterable[str] | None = None,
remove: Iterable[str] | None = None,
set_to: Iterable[str] | None = None,
) -> List[str]:
"""Update active analysts and persist TEAM_PIPELINE.yaml."""
available = [item for item in available_analysts]
ensure_team_pipeline_config(project_root, config_name, available)
parsed = load_team_pipeline_config(project_root, config_name)
discussion = parsed.setdefault("discussion", {})
if not isinstance(discussion, dict):
discussion = {}
parsed["discussion"] = discussion
current = discussion.get("active_analysts", [])
if not isinstance(current, list):
current = []
current = [item for item in current if item in available]
if not current:
current = list(available)
if set_to is not None:
target = [item for item in set_to if item in available]
current = target or current
for item in add or []:
if item in available and item not in current:
current.append(item)
for item in remove or []:
current = [existing for existing in current if existing != item]
if not current:
current = [available[0]] if available else []
discussion["active_analysts"] = current
save_team_pipeline_config(project_root, config_name, parsed)
return current

View File

@@ -1,286 +0,0 @@
"""
Agent模板定义
包含各角色的ROLE.md内容字典供程序生成Agent工作空间时使用。
"""
# 基础模板文件内容
BASE_TEMPLATES = {
"AGENTS.md": """# Agent Guide
## 工作流程
1. 接收分析任务
2. 调用相关工具/技能
3. 生成分析报告
4. 参与团队决策
## 工具使用规范
- 优先使用已激活的技能
- 不确定时询问Portfolio Manager
- 重要发现用 `/save` 记录
## 记忆管理
- 使用 `/compact` 定期压缩记忆
- 投资经验记录在MEMORY.md
""",
"SOUL.md": """# Soul
你是专业的金融分析师,语气冷静、客观、专业。
你的分析应该数据驱动,避免情绪化表达。
""",
"PROFILE.md": """# Profile
## 投资风格
- 风险承受能力:中等
- 投资期限中期3-12个月
- 偏好行业:科技、医疗、消费
## 优势
- 财务分析
- 趋势识别
## 改进方向
- 市场情绪把握
""",
"MEMORY.md": """# Memory
<!-- 此文件用于记录Agent的学习经验和重要发现 -->
## 经验总结
## 重要事件
## 改进记录
""",
"HEARTBEAT.md": """# Heartbeat
## 定时任务
- 每日开盘前检查持仓
- 收盘后记录当日表现
""",
"POLICY.md": """# Policy
## 风控规则
- 单一持仓不超过20%
- 止损线:-15%
""",
"STYLE.md": """# Style
- 使用结构化输出JSON/Markdown表格
- 包含置信度评分
- 列出关键假设
""",
"agent.yaml": """agent_id: {agent_id}
agent_type: {agent_type}
name: {name}
model:
provider: openai
model_name: gpt-4o
temperature: 0.3
enabled_skills: []
disabled_skills: []
settings: {{}}
""",
}
# 角色专用模板
ROLE_TEMPLATES = {
"fundamental": {
"ROLE.md": """# Role: Fundamental Analyst
## 职责
分析公司财务报表、盈利能力、成长性、竞争优势等基本面因素。
## 分析维度
- 财务报表分析(资产负债表、利润表、现金流量表)
- 盈利能力指标ROE、ROA、毛利率、净利率
- 成长性指标(营收增长率、利润增长率)
- 估值指标P/E、P/B、P/S
- 行业地位和竞争优势
## 输出格式
- 财务健康度评分1-10
- 成长性评分1-10
- 关键财务亮点和风险
- 同业对比分析
""",
"SOUL.md": """# Soul
你是严谨的基本面分析师,像沃伦·巴菲特一样注重企业内在价值。
你的分析深入细致,关注长期价值而非短期波动。
语气沉稳、逻辑严密,善于发现财务数据背后的商业本质。
""",
},
"technical": {
"ROLE.md": """# Role: Technical Analyst
## 职责
分析价格走势、交易量、技术指标,识别买卖时机。
## 分析维度
- 趋势分析(长期/中期/短期趋势)
- 支撑阻力位识别
- 技术指标MACD、RSI、KDJ、布林带等
- 形态识别(头肩顶/底、双底、三角形等)
- 量价关系分析
## 输出格式
- 趋势方向(上涨/下跌/震荡)
- 关键价位(支撑/阻力)
- 技术信号(买入/卖出/观望)
- 置信度评分
""",
"SOUL.md": """# Soul
你是敏锐的技术分析师,相信价格包含一切信息。
你善于从图表中发现规律,像侦探一样寻找市场留下的痕迹。
语气果断、快速反应,善于捕捉稍纵即逝的交易机会。
""",
},
"sentiment": {
"ROLE.md": """# Role: Sentiment Analyst
## 职责
分析市场情绪、资金流向、新闻舆情,判断市场心理状态。
## 分析维度
- 市场情绪指标(恐慌/贪婪指数)
- 资金流向分析(主力/散户资金)
- 新闻舆情分析(正面/负面/中性)
- 社交媒体情绪
- 机构持仓变化
## 输出格式
- 情绪评分(-10到+10极度恐慌到极度贪婪
- 资金流向判断
- 舆情摘要
- 情绪拐点预警
""",
"SOUL.md": """# Soul
你是敏感的市场情绪捕手,善于感知市场的恐惧与贪婪。
你关注人性在金融市场中的表现,理解情绪如何驱动价格。
语气富有洞察力、善于捕捉微妙变化,像心理学家一样理解市场参与者。
""",
},
"valuation": {
"ROLE.md": """# Role: Valuation Analyst
## 职责
评估公司内在价值,计算合理价格区间,识别高估/低估机会。
## 分析维度
- DCF现金流折现模型
- 相对估值法P/E、EV/EBITDA等
- 资产重估法
- 分部估值SOTP
- 安全边际计算
## 输出格式
- 内在价值估算
- 合理价格区间
- 当前价格vs内在价值高估/低估百分比)
- 估值假设和敏感性分析
""",
"SOUL.md": """# Soul
你是精确的估值分析师,追求计算内在价值的准确区间。
你像精算师一样严谨,注重假设的合理性和安全边际。
语气精确、注重数字,善于发现市场定价错误带来的机会。
""",
},
"portfolio": {
"ROLE.md": """# Role: Portfolio Manager
## 职责
统筹各分析师意见,制定投资决策,管理投资组合配置。
## 分析维度
- 资产配置策略(股债比例、行业分布)
- 风险收益平衡
- 仓位管理(建仓/加仓/减仓/清仓)
- 再平衡时机
- 组合相关性分析
## 输出格式
- 投资决策(买入/卖出/持有)
- 建议仓位比例
- 目标价位
- 止损止盈设置
- 组合调整建议
""",
"SOUL.md": """# Soul
你是睿智的投资组合经理,像将军一样统筹全局。
你善于权衡各方意见,做出果断而理性的投资决策。
语气权威、决策果断,对组合整体表现负有最终责任。
""",
},
"risk": {
"ROLE.md": """# Role: Risk Manager
## 职责
识别、评估和监控投资风险,确保组合风险在可控范围内。
## 分析维度
- 市场风险Beta、波动率
- 信用风险
- 流动性风险
- 集中度风险
- 尾部风险VaR、CVaR
- 压力测试
## 输出格式
- 风险等级(低/中/高/极高)
- 风险敞口分析
- 风险调整建议
- 预警阈值设置
- 应急预案
""",
"SOUL.md": """# Soul
你是谨慎的风险管理者,时刻警惕潜在的损失。
你像守门员一样守护组合安全,宁可错过机会也不冒无法承受的风险。
语气保守、风险意识强,善于发现隐藏的威胁和脆弱性。
""",
},
}
def get_base_template(filename: str) -> str | None:
"""获取基础模板内容"""
return BASE_TEMPLATES.get(filename)
def get_role_template(role_type: str, filename: str) -> str | None:
"""获取角色专用模板内容"""
role = ROLE_TEMPLATES.get(role_type)
if role:
return role.get(filename)
return None
def get_all_role_types() -> list[str]:
"""获取所有角色类型列表"""
return list(ROLE_TEMPLATES.keys())
def render_agent_yaml(agent_id: str, agent_type: str, name: str) -> str:
"""渲染agent.yaml模板"""
return BASE_TEMPLATES["agent.yaml"].format(
agent_id=agent_id,
agent_type=agent_type,
name=name
)

View File

@@ -41,6 +41,16 @@ class RunWorkspaceManager:
"tickers:\n"
" - AAPL\n"
" - MSFT\n"
" - GOOGL\n"
" - AMZN\n"
" - NVDA\n"
" - META\n"
" - TSLA\n"
" - AMD\n"
" - NFLX\n"
" - AVGO\n"
" - PLTR\n"
" - COIN\n"
"initial_cash: 100000\n"
"margin_requirement: 0.0\n"
"enable_memory: false\n"
@@ -63,9 +73,8 @@ class RunWorkspaceManager:
self,
config_name: str,
agent_id: str,
role_seed: str = "",
style_seed: str = "",
policy_seed: str = "",
file_contents: Optional[Dict[str, str]] = None,
persona: Optional[Dict[str, object]] = None,
) -> Path:
asset_dir = self.skills_manager.get_agent_asset_dir(
config_name,
@@ -77,58 +86,82 @@ class RunWorkspaceManager:
(asset_dir / "skills" / "disabled").mkdir(parents=True, exist_ok=True)
(asset_dir / "skills" / "local").mkdir(parents=True, exist_ok=True)
self._ensure_file(
asset_dir / "ROLE.md",
"# Role\n\n"
"Optional run-scoped role override.\n\n"
f"{role_seed}".strip()
+ "\n",
)
self._ensure_file(
asset_dir / "STYLE.md",
"# Style\n\n"
"Optional run-scoped communication or reasoning style.\n\n"
f"{style_seed}".strip()
+ "\n",
)
self._ensure_file(
asset_dir / "POLICY.md",
"# Policy\n\n"
"Optional run-scoped constraints, limits, or strategy policy.\n\n"
f"{policy_seed}".strip()
+ "\n",
)
self._ensure_file(
asset_dir / "SOUL.md",
"# Soul\n\n"
"Describe the agent's temperament, reasoning posture, and voice.\n\n",
)
self._ensure_file(
asset_dir / "PROFILE.md",
"# Profile\n\n"
"Track this agent's long-lived investment style, preferences, and strengths.\n\n",
)
self._ensure_file(
asset_dir / "AGENTS.md",
"# Agent Guide\n\n"
"Document how this agent should work, collaborate, and choose tools or skills.\n\n",
)
self._ensure_file(
asset_dir / "MEMORY.md",
"# Memory\n\n"
"Store durable lessons, heuristics, and reminders for this agent.\n\n",
)
self._ensure_file(
asset_dir / "HEARTBEAT.md",
"# Heartbeat\n\n"
"Optional checklist for periodic review or self-reflection.\n\n",
)
file_contents = file_contents or self.build_default_agent_files(agent_id=agent_id)
for filename, content in file_contents.items():
legacy_contents = self.build_legacy_agent_file_variants(
agent_id=agent_id,
filename=filename,
persona=persona,
)
self._ensure_file(asset_dir / filename, content, legacy_contents=legacy_contents)
self._ensure_agent_yaml(
asset_dir / "agent.yaml",
agent_id=agent_id,
)
return asset_dir
def build_default_agent_files(
self,
*,
agent_id: str,
persona: Optional[Dict[str, object]] = None,
) -> Dict[str, str]:
"""Build default workspace markdown files for one agent."""
if agent_id.endswith("_analyst"):
return self._build_analyst_files(agent_id=agent_id, persona=persona or {})
if agent_id == "portfolio_manager":
return self._build_portfolio_manager_files()
if agent_id == "risk_manager":
return self._build_risk_manager_files()
return self._build_generic_files(agent_id=agent_id)
def build_legacy_agent_file_variants(
self,
*,
agent_id: str,
filename: str,
persona: Optional[Dict[str, object]] = None,
) -> list[str]:
"""Return known generated legacy variants safe to upgrade in-place."""
persona = persona or {}
variants: list[dict[str, str]] = [
self._build_legacy_english_files(agent_id=agent_id),
self._build_previous_chinese_files(agent_id=agent_id, persona=persona),
]
values: list[str] = []
for item in variants:
content = item.get(filename)
if content:
values.append(content)
return values
def load_agent_file(
self,
*,
config_name: str,
agent_id: str,
filename: str,
) -> str:
"""Load one run-scoped agent workspace file."""
path = self.skills_manager.get_agent_asset_dir(config_name, agent_id) / filename
if not path.exists():
raise FileNotFoundError(f"File not found: {filename}")
return path.read_text(encoding="utf-8")
def update_agent_file(
self,
*,
config_name: str,
agent_id: str,
filename: str,
content: str,
) -> None:
"""Write one run-scoped agent workspace file."""
asset_dir = self.skills_manager.get_agent_asset_dir(config_name, agent_id)
asset_dir.mkdir(parents=True, exist_ok=True)
path = asset_dir / filename
path.write_text(content, encoding="utf-8")
def initialize_default_assets(
self,
config_name: str,
@@ -141,49 +174,285 @@ class RunWorkspaceManager:
for agent_id in agent_ids:
if agent_id.endswith("_analyst"):
persona = analyst_personas.get(agent_id, {})
role_seed = persona.get("description", "").strip()
focus_items = persona.get("focus", [])
style_seed = "\n".join(f"- {item}" for item in focus_items)
policy_seed = (
"State a clear signal, confidence, and the conditions that would invalidate the thesis."
)
elif agent_id == "portfolio_manager":
role_seed = (
"Synthesize analyst and risk inputs into explicit portfolio decisions."
)
style_seed = (
"Be concise, capital-aware, and explicit about sizing rationale."
)
policy_seed = (
"Respect cash, margin, and portfolio concentration constraints before recording decisions."
)
elif agent_id == "risk_manager":
role_seed = (
"Quantify concentration, leverage, liquidity, and volatility risk before trade execution."
)
style_seed = (
"Prioritize the highest-severity risk first and state concrete limits."
)
policy_seed = (
"Use available risk tools before issuing the final risk memo."
file_contents = self.build_default_agent_files(
agent_id=agent_id,
persona=persona,
)
else:
role_seed = ""
style_seed = ""
policy_seed = ""
self.ensure_agent_assets(
config_name=config_name,
agent_id=agent_id,
role_seed=role_seed,
style_seed=style_seed,
policy_seed=policy_seed,
)
persona = None
file_contents = self.build_default_agent_files(agent_id=agent_id)
asset_dir = self.skills_manager.get_agent_asset_dir(config_name, agent_id)
asset_dir.mkdir(parents=True, exist_ok=True)
(asset_dir / "skills" / "installed").mkdir(parents=True, exist_ok=True)
(asset_dir / "skills" / "active").mkdir(parents=True, exist_ok=True)
(asset_dir / "skills" / "disabled").mkdir(parents=True, exist_ok=True)
(asset_dir / "skills" / "local").mkdir(parents=True, exist_ok=True)
for filename, content in file_contents.items():
self._ensure_file(
asset_dir / filename,
content,
legacy_contents=self.build_legacy_agent_file_variants(
agent_id=agent_id,
filename=filename,
persona=persona,
),
)
self._ensure_agent_yaml(asset_dir / "agent.yaml", agent_id=agent_id)
@staticmethod
def _ensure_file(path: Path, content: str) -> None:
def _ensure_file(path: Path, content: str, *, legacy_contents: Optional[list[str]] = None) -> None:
if not path.exists():
path.write_text(content, encoding="utf-8")
return
existing = path.read_text(encoding="utf-8")
normalized_existing = existing.strip()
candidates = {item.strip() for item in (legacy_contents or []) if item and item.strip()}
if normalized_existing in candidates:
path.write_text(content, encoding="utf-8")
@staticmethod
def _build_generic_files(agent_id: str) -> Dict[str, str]:
return {
"SOUL.md": (
"# Soul\n\n"
f"你是 `{agent_id}`,语气冷静、客观、专业。保持清晰推理,优先基于数据而不是情绪下结论。\n"
),
"PROFILE.md": (
"# Profile\n\n"
"记录这个 agent 长期稳定的分析风格、偏好、优势与盲点。\n"
),
"AGENTS.md": (
"# Agent Guide\n\n"
"工作要求:\n"
"- 优先使用已激活的技能和工具\n"
"- 结论要明确,过程要可追溯\n"
"- 与其他 agent 协作时保持输入输出简洁\n"
"- 最终输出必须使用简体中文;如需引用英文术语,仅保留专有名词,解释和结论必须用中文\n"
),
"POLICY.md": (
"# Policy\n\n"
"- 给出结论时说明核心驱动因素\n"
"- 明确风险边界和结论失效条件\n"
"- 出现反例时需要纳入最终判断\n"
"- 不要输出英文报告标题、英文摘要或整段英文正文\n"
),
"MEMORY.md": (
"# Memory\n\n"
"记录可复用的经验、失误复盘、有效启发式和需要持续跟踪的提醒。\n"
),
}
@classmethod
def _build_analyst_files(cls, *, agent_id: str, persona: Dict[str, object]) -> Dict[str, str]:
role_name = str(persona.get("name") or agent_id)
focus_items = [
str(item).strip()
for item in persona.get("focus", [])
if str(item).strip()
]
focus_md = "\n".join(f"- {item}" for item in focus_items) or "- 根据当前任务选择最相关的分析维度"
description = str(persona.get("description") or "").strip()
files = cls._build_generic_files(agent_id)
files["SOUL.md"] = (
"# Soul\n\n"
f"你是一位专业的{role_name}\n\n"
"保持谦逊和开放,主动寻找与自己观点相悖的证据,并将其纳入最终评估。"
"你的分析要体现持续演化的投资哲学,而不是一次性的结论。\n"
)
files["PROFILE.md"] = (
"# Profile\n\n"
f"角色定位:{role_name}\n\n"
"你的关注重点:\n"
f"{focus_md}\n\n"
"角色说明:\n"
f"{description or '围绕最关键的基本面、技术面、情绪面或估值因素形成高质量判断。'}\n"
)
files["AGENTS.md"] = (
"# Agent Guide\n\n"
"分析流程:\n"
"- 优先识别真正驱动价值或价格变化的核心变量\n"
"- 使用相关工具和技能补足证据链\n"
"- 给出可验证、可复查、可执行的分析结果\n"
"- 在团队讨论中清晰表达你的论点和反论点\n\n"
"输出要求:\n"
"- 给出明确投资信号:看涨、看跌或中性\n"
"- 包含置信度0-100\n"
"- 如果你确定要分享最终分析,请先给出结论,再给出推理依据\n"
"- 最终输出必须使用简体中文,不要生成英文版 analysis report\n"
)
files["POLICY.md"] = (
"# Policy\n\n"
"- 深化你的投资逻辑,确保每项建议都有清晰、可追溯、可重复的依据\n"
"- 明确风险边界:在什么具体情况下当前结论会失效\n"
"- 做逆向测试:说明市场主流共识与你的不同点\n"
"- 每次分析后反思这次案例如何验证或挑战你现有的信念\n"
"- 即使输入新闻或财报原文是英文,最终表达也必须用中文\n"
)
return files
@classmethod
def _build_portfolio_manager_files(cls) -> Dict[str, str]:
files = cls._build_generic_files("portfolio_manager")
files["SOUL.md"] = (
"# Soul\n\n"
"你是一位负责做出投资决策的投资组合经理。你需要综合多个分析视角,"
"做出保守、明确、资本约束下可执行的组合决策。\n"
)
files["PROFILE.md"] = (
"# Profile\n\n"
"核心职责:\n"
"- 分析分析师和风险管理经理的输入\n"
"- 基于信号和市场情境做出投资决策\n"
"- 使用可用工具记录每个 ticker 的决策\n"
)
files["AGENTS.md"] = (
"# Agent Guide\n\n"
"决策框架:\n"
"- 审阅分析以理解市场观点\n"
"- 在做决策前先考虑风险警告\n"
"- 评估当前投资组合持仓、现金与保证金占用\n"
"- 决策必须与整体投资目标和风险约束一致\n\n"
"决策类型:\n"
'- `long`:看涨,建议买入\n'
'- `short`:看跌,建议卖出或做空\n'
'- `hold`:中性,维持当前持仓\n\n'
"输出要求:\n"
"- 使用 `make_decision` 工具记录每个股票的最终决策\n"
"- 记录完成后给出投资逻辑总结\n"
"- 最终总结必须使用简体中文\n"
)
files["POLICY.md"] = (
"# Policy\n\n"
"- 在决定数量时考虑可用现金,不要超出现金允许范围\n"
"- 考虑做空头寸的保证金要求\n"
"- 仓位规模相对于组合总资产保持保守\n"
"- 始终为决策提供清晰理由\n"
"- 不要输出英文投资报告或英文结论\n"
)
return files
@classmethod
def _build_risk_manager_files(cls) -> Dict[str, str]:
files = cls._build_generic_files("risk_manager")
files["SOUL.md"] = (
"# Soul\n\n"
"你是一位专业的风险管理经理,负责监控投资组合风险并提供风险警告。"
"你的目标不是输出空泛的谨慎,而是给出量化、可执行、可优先级排序的风险意见。\n"
)
files["PROFILE.md"] = (
"# Profile\n\n"
"核心职责:\n"
"- 监控投资组合敞口和集中度风险\n"
"- 评估仓位规模相对于波动性是否合理\n"
"- 评估保证金使用和杠杆水平\n"
"- 识别潜在风险因素并提供警告\n"
"- 基于市场条件建议仓位限制\n"
)
files["AGENTS.md"] = (
"# Agent Guide\n\n"
"决策流程:\n"
"- 优先使用可用的风险工具量化集中度、波动率和保证金压力\n"
"- 结合工具结果与当前市场上下文做判断\n"
"- 生成可操作的风险警告和仓位限制建议\n"
"- 为风险评估提供清晰理由\n\n"
"输出要求:\n"
"- 风险评估要简洁但全面\n"
"- 按严重程度优先排序警告\n"
"- 提供具体、可操作的建议\n"
"- 尽可能包含量化指标\n"
"- 最终风险结论必须使用简体中文\n"
)
files["POLICY.md"] = (
"# Policy\n\n"
"- 先量化,再判断,不要只给抽象风险表述\n"
"- 高严重度风险必须先说\n"
"- 最终结论需要明确仓位限制或调整建议\n"
"- 不要输出英文风险报告或英文摘要\n"
)
return files
@staticmethod
def _build_legacy_english_files(agent_id: str) -> Dict[str, str]:
policy_tail = "Optional run-scoped constraints, limits, or strategy policy.\n\n"
if agent_id == "portfolio_manager":
policy_tail += "Respect cash, margin, and portfolio concentration constraints before recording decisions.\n"
elif agent_id == "risk_manager":
policy_tail += "Use available risk tools before issuing the final risk memo.\n"
elif agent_id.endswith("_analyst"):
policy_tail += "State a clear signal, confidence, and the conditions that would invalidate the thesis.\n"
return {
"SOUL.md": "# Soul\n\nDescribe the agent's temperament, reasoning posture, and voice.\n\n",
"PROFILE.md": "# Profile\n\nTrack this agent's long-lived investment style, preferences, and strengths.\n\n",
"AGENTS.md": "# Agent Guide\n\nDocument how this agent should work, collaborate, and choose tools or skills.\n\n",
"POLICY.md": "# Policy\n\n" + policy_tail,
"MEMORY.md": "# Memory\n\nStore durable lessons, heuristics, and reminders for this agent.\n\n",
}
@classmethod
def _build_previous_chinese_files(cls, *, agent_id: str, persona: Dict[str, object]) -> Dict[str, str]:
if agent_id.endswith("_analyst"):
role_name = str(persona.get("name") or agent_id)
focus_items = [
str(item).strip()
for item in persona.get("focus", [])
if str(item).strip()
]
focus_md = "\n".join(f"- {item}" for item in focus_items) or "- 根据当前任务选择最相关的分析维度"
description = str(persona.get("description") or "").strip()
return {
"SOUL.md": (
"# Soul\n\n"
f"你是一位专业的{role_name}\n\n"
"保持谦逊和开放,主动寻找与自己观点相悖的证据,并将其纳入最终评估。"
"你的分析要体现持续演化的投资哲学,而不是一次性的结论。\n"
),
"PROFILE.md": (
"# Profile\n\n"
f"角色定位:{role_name}\n\n"
"你的关注重点:\n"
f"{focus_md}\n\n"
"角色说明:\n"
f"{description or '围绕最关键的基本面、技术面、情绪面或估值因素形成高质量判断。'}\n"
),
"AGENTS.md": (
"# Agent Guide\n\n"
"分析流程:\n"
"- 优先识别真正驱动价值或价格变化的核心变量\n"
"- 使用相关工具和技能补足证据链\n"
"- 给出可验证、可复查、可执行的分析结果\n"
"- 在团队讨论中清晰表达你的论点和反论点\n\n"
"输出要求:\n"
"- 给出明确投资信号:看涨、看跌或中性\n"
"- 包含置信度0-100\n"
"- 如果你确定要分享最终分析,请先给出结论,再给出推理依据\n"
),
"POLICY.md": (
"# Policy\n\n"
"- 深化你的投资逻辑,确保每项建议都有清晰、可追溯、可重复的依据\n"
"- 明确风险边界:在什么具体情况下当前结论会失效\n"
"- 做逆向测试:说明市场主流共识与你的不同点\n"
"- 每次分析后反思这次案例如何验证或挑战你现有的信念\n"
),
"MEMORY.md": "# Memory\n\n记录可复用的经验、失误复盘、有效启发式和需要持续跟踪的提醒。\n",
}
if agent_id == "portfolio_manager":
return {
"SOUL.md": "# Soul\n\n你是一位负责做出投资决策的投资组合经理。你需要综合多个分析视角,做出保守、明确、资本约束下可执行的组合决策。\n",
"PROFILE.md": "# Profile\n\n核心职责:\n- 分析分析师和风险管理经理的输入\n- 基于信号和市场情境做出投资决策\n- 使用可用工具记录每个 ticker 的决策\n",
"AGENTS.md": "# Agent Guide\n\n决策框架:\n- 审阅分析以理解市场观点\n- 在做决策前先考虑风险警告\n- 评估当前投资组合持仓、现金与保证金占用\n- 决策必须与整体投资目标和风险约束一致\n\n决策类型:\n- `long`:看涨,建议买入\n- `short`:看跌,建议卖出或做空\n- `hold`:中性,维持当前持仓\n\n输出要求:\n- 使用 `make_decision` 工具记录每个股票的最终决策\n- 记录完成后给出投资逻辑总结\n",
"POLICY.md": "# Policy\n\n- 在决定数量时考虑可用现金,不要超出现金允许范围\n- 考虑做空头寸的保证金要求\n- 仓位规模相对于组合总资产保持保守\n- 始终为决策提供清晰理由\n",
"MEMORY.md": "# Memory\n\n记录可复用的经验、失误复盘、有效启发式和需要持续跟踪的提醒。\n",
}
if agent_id == "risk_manager":
return {
"SOUL.md": "# Soul\n\n你是一位专业的风险管理经理,负责监控投资组合风险并提供风险警告。你的目标不是输出空泛的谨慎,而是给出量化、可执行、可优先级排序的风险意见。\n",
"PROFILE.md": "# Profile\n\n核心职责:\n- 监控投资组合敞口和集中度风险\n- 评估仓位规模相对于波动性是否合理\n- 评估保证金使用和杠杆水平\n- 识别潜在风险因素并提供警告\n- 基于市场条件建议仓位限制\n",
"AGENTS.md": "# Agent Guide\n\n决策流程:\n- 优先使用可用的风险工具量化集中度、波动率和保证金压力\n- 结合工具结果与当前市场上下文做判断\n- 生成可操作的风险警告和仓位限制建议\n- 为风险评估提供清晰理由\n\n输出要求:\n- 风险评估要简洁但全面\n- 按严重程度优先排序警告\n- 提供具体、可操作的建议\n- 尽可能包含量化指标\n",
"POLICY.md": "# Policy\n\n- 先量化,再判断,不要只给抽象风险表述\n- 高严重度风险必须先说\n- 最终结论需要明确仓位限制或调整建议\n",
"MEMORY.md": "# Memory\n\n记录可复用的经验、失误复盘、有效启发式和需要持续跟踪的提醒。\n",
}
return cls._build_legacy_english_files(agent_id)
@staticmethod
def _ensure_agent_yaml(path: Path, agent_id: str) -> None:

View File

@@ -11,11 +11,13 @@ Provides REST API endpoints for:
from .agents import router as agents_router
from .workspaces import router as workspaces_router
from .guard import router as guard_router
from .openclaw import router as openclaw_router
from .runtime import router as runtime_router
__all__ = [
"agents_router",
"workspaces_router",
"guard_router",
"openclaw_router",
"runtime_router",
]

View File

@@ -13,8 +13,13 @@ from typing import Any, Dict, List, Optional
from fastapi import APIRouter, HTTPException, Depends, Body, UploadFile, File, Form
from pydantic import BaseModel, Field
from backend.agents import AgentFactory, WorkspaceManager, get_registry
from backend.agents import AgentFactory, get_registry
from backend.agents.workspace_manager import RunWorkspaceManager
from backend.agents.agent_workspace import load_agent_workspace_config
from backend.agents.skills_manager import SkillsManager
from backend.agents.toolkit_factory import load_agent_profiles
from backend.config.bootstrap_config import get_bootstrap_config_for_run
from backend.llm.models import get_agent_model_info
logger = logging.getLogger(__name__)
@@ -47,6 +52,14 @@ class InstallExternalSkillRequest(BaseModel):
activate: bool = Field(True, description="Whether to enable skill immediately")
class LocalSkillRequest(BaseModel):
skill_name: str = Field(..., description="Local skill name")
class LocalSkillContentRequest(BaseModel):
content: str = Field(..., description="Updated SKILL.md content")
class AgentResponse(BaseModel):
"""Agent information response."""
agent_id: str
@@ -63,6 +76,24 @@ class AgentFileResponse(BaseModel):
content: str
class AgentProfileResponse(BaseModel):
agent_id: str
workspace_id: str
profile: Dict[str, Any]
class AgentSkillsResponse(BaseModel):
agent_id: str
workspace_id: str
skills: List[Dict[str, Any]]
class SkillDetailResponse(BaseModel):
agent_id: str
workspace_id: str
skill: Dict[str, Any]
# Dependencies
def get_agent_factory():
"""Get AgentFactory instance."""
@@ -70,8 +101,8 @@ def get_agent_factory():
def get_workspace_manager():
"""Get WorkspaceManager instance."""
return WorkspaceManager()
"""Get run-scoped workspace manager instance."""
return RunWorkspaceManager()
def get_skills_manager():
@@ -199,6 +230,108 @@ async def get_agent(
)
@router.get("/{agent_id}/profile", response_model=AgentProfileResponse)
async def get_agent_profile(
workspace_id: str,
agent_id: str,
skills_manager: SkillsManager = Depends(get_skills_manager),
):
asset_dir = skills_manager.get_agent_asset_dir(workspace_id, agent_id)
agent_config = load_agent_workspace_config(asset_dir / "agent.yaml")
profiles = load_agent_profiles()
profile = profiles.get(agent_id, {})
bootstrap = get_bootstrap_config_for_run(skills_manager.project_root, workspace_id)
override = bootstrap.agent_override(agent_id)
active_tool_groups = override.get("active_tool_groups", agent_config.active_tool_groups or profile.get("active_tool_groups", []))
if not isinstance(active_tool_groups, list):
active_tool_groups = []
disabled_tool_groups = agent_config.disabled_tool_groups
if disabled_tool_groups:
disabled_set = set(disabled_tool_groups)
active_tool_groups = [group_name for group_name in active_tool_groups if group_name not in disabled_set]
default_skills = profile.get("skills", [])
if not isinstance(default_skills, list):
default_skills = []
resolved_skills = skills_manager.resolve_agent_skill_names(
config_name=workspace_id,
agent_id=agent_id,
default_skills=default_skills,
)
prompt_files = agent_config.prompt_files or ["SOUL.md", "PROFILE.md", "AGENTS.md", "POLICY.md", "MEMORY.md"]
model_name, model_provider = get_agent_model_info(agent_id)
return AgentProfileResponse(
agent_id=agent_id,
workspace_id=workspace_id,
profile={
"model_name": model_name,
"model_provider": model_provider,
"prompt_files": prompt_files,
"default_skills": default_skills,
"resolved_skills": resolved_skills,
"active_tool_groups": active_tool_groups,
"disabled_tool_groups": disabled_tool_groups,
"enabled_skills": agent_config.enabled_skills,
"disabled_skills": agent_config.disabled_skills,
},
)
@router.get("/{agent_id}/skills", response_model=AgentSkillsResponse)
async def get_agent_skills(
workspace_id: str,
agent_id: str,
skills_manager: SkillsManager = Depends(get_skills_manager),
):
agent_asset_dir = skills_manager.get_agent_asset_dir(workspace_id, agent_id)
agent_config = load_agent_workspace_config(agent_asset_dir / "agent.yaml")
resolved_skills = set(skills_manager.resolve_agent_skill_names(config_name=workspace_id, agent_id=agent_id, default_skills=[]))
enabled = set(agent_config.enabled_skills)
disabled = set(agent_config.disabled_skills)
payload = []
for item in skills_manager.list_agent_skill_catalog(workspace_id, agent_id):
if item.skill_name in disabled:
status = "disabled"
elif item.skill_name in enabled:
status = "enabled"
elif item.skill_name in resolved_skills:
status = "active"
else:
status = "available"
payload.append({
"skill_name": item.skill_name,
"name": item.name,
"description": item.description,
"version": item.version,
"source": item.source,
"tools": item.tools,
"status": status,
})
return AgentSkillsResponse(agent_id=agent_id, workspace_id=workspace_id, skills=payload)
@router.get("/{agent_id}/skills/{skill_name}", response_model=SkillDetailResponse)
async def get_agent_skill_detail(
workspace_id: str,
agent_id: str,
skill_name: str,
skills_manager: SkillsManager = Depends(get_skills_manager),
):
try:
detail = skills_manager.load_agent_skill_document(
config_name=workspace_id,
agent_id=agent_id,
skill_name=skill_name,
)
except FileNotFoundError:
raise HTTPException(status_code=404, detail=f"Unknown skill: {skill_name}")
return SkillDetailResponse(agent_id=agent_id, workspace_id=workspace_id, skill=detail)
@router.delete("/{agent_id}")
async def delete_agent(
workspace_id: str,
@@ -386,6 +519,85 @@ async def install_external_skill(
}
@router.post("/{agent_id}/skills/local")
async def create_local_skill(
workspace_id: str,
agent_id: str,
request: LocalSkillRequest,
registry=Depends(get_registry),
):
agent_info = registry.get(agent_id)
if not agent_info or agent_info.workspace_id != workspace_id:
raise HTTPException(status_code=404, detail=f"Agent '{agent_id}' not found")
skills_manager = SkillsManager()
try:
skills_manager.create_agent_local_skill(
config_name=workspace_id,
agent_id=agent_id,
skill_name=request.skill_name,
)
except (ValueError, FileExistsError) as exc:
raise HTTPException(status_code=400, detail=str(exc))
return {"message": f"Created local skill '{request.skill_name}' for '{agent_id}'"}
@router.put("/{agent_id}/skills/local/{skill_name}")
async def update_local_skill(
workspace_id: str,
agent_id: str,
skill_name: str,
request: LocalSkillContentRequest,
registry=Depends(get_registry),
):
agent_info = registry.get(agent_id)
if not agent_info or agent_info.workspace_id != workspace_id:
raise HTTPException(status_code=404, detail=f"Agent '{agent_id}' not found")
skills_manager = SkillsManager()
try:
skills_manager.update_agent_local_skill(
config_name=workspace_id,
agent_id=agent_id,
skill_name=skill_name,
content=request.content,
)
except (ValueError, FileNotFoundError) as exc:
raise HTTPException(status_code=400, detail=str(exc))
return {"message": f"Updated local skill '{skill_name}' for '{agent_id}'"}
@router.delete("/{agent_id}/skills/local/{skill_name}")
async def delete_local_skill(
workspace_id: str,
agent_id: str,
skill_name: str,
registry=Depends(get_registry),
):
agent_info = registry.get(agent_id)
if not agent_info or agent_info.workspace_id != workspace_id:
raise HTTPException(status_code=404, detail=f"Agent '{agent_id}' not found")
skills_manager = SkillsManager()
try:
skills_manager.delete_agent_local_skill(
config_name=workspace_id,
agent_id=agent_id,
skill_name=skill_name,
)
skills_manager.forget_agent_skill_overrides(
config_name=workspace_id,
agent_id=agent_id,
skill_names=[skill_name],
)
except (ValueError, FileNotFoundError) as exc:
raise HTTPException(status_code=400, detail=str(exc))
return {"message": f"Deleted local skill '{skill_name}' for '{agent_id}'"}
@router.post("/{agent_id}/skills/upload")
async def upload_external_skill(
workspace_id: str,
@@ -441,7 +653,7 @@ async def get_agent_file(
workspace_id: str,
agent_id: str,
filename: str,
workspace_manager: WorkspaceManager = Depends(get_workspace_manager),
workspace_manager: RunWorkspaceManager = Depends(get_workspace_manager),
):
"""
Read an agent's workspace file.
@@ -449,7 +661,7 @@ async def get_agent_file(
Args:
workspace_id: Workspace identifier
agent_id: Agent identifier
filename: File to read (e.g., SOUL.md, ROLE.md)
filename: File to read (e.g., SOUL.md, PROFILE.md)
Returns:
File content
@@ -471,7 +683,7 @@ async def update_agent_file(
agent_id: str,
filename: str,
content: str = Body(..., media_type="text/plain"),
workspace_manager: WorkspaceManager = Depends(get_workspace_manager),
workspace_manager: RunWorkspaceManager = Depends(get_workspace_manager),
):
"""
Update an agent's workspace file.

839
backend/api/openclaw.py Normal file
View File

@@ -0,0 +1,839 @@
# -*- coding: utf-8 -*-
"""Read-only OpenClaw CLI API routes — typed with Pydantic models."""
from __future__ import annotations
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel, Field
from backend.services.openclaw_cli import OpenClawCliError, OpenClawCliService
from shared.models.openclaw import OpenClawStatus
router = APIRouter(prefix="/api/openclaw", tags=["openclaw"])
def get_openclaw_cli_service() -> OpenClawCliService:
"""Build the OpenClaw CLI service dependency."""
return OpenClawCliService()
def _raise_cli_http_error(exc: OpenClawCliError) -> None:
detail = {
"message": str(exc),
"command": exc.command,
"exit_code": exc.exit_code,
"stdout": exc.stdout,
"stderr": exc.stderr,
}
status_code = 503 if exc.exit_code is None else 502
raise HTTPException(status_code=status_code, detail=detail) from exc
# ---------------------------------------------------------------------------
# Response wrappers
# ---------------------------------------------------------------------------
class StatusResponse(BaseModel):
status: object
class SessionsResponse(BaseModel):
sessions: list[object]
class SessionDetailResponse(BaseModel):
session: object | None
class SessionHistoryResponse(BaseModel):
session_key: str
session_id: str | None
events: list[object]
history: list[object]
raw_text: str | None
class CronResponse(BaseModel):
cron: list[object]
jobs: list[object]
class ApprovalsResponse(BaseModel):
approvals: list[object]
pending: list[object]
class AgentsResponse(BaseModel):
agents: list[object]
class SkillsResponse(BaseModel):
workspace_dir: str
managed_skills_dir: str
skills: list[object]
class ModelsResponse(BaseModel):
models: list[object]
class HooksResponse(BaseModel):
workspace_dir: str
managed_hooks_dir: str
hooks: list[object]
class PluginsResponse(BaseModel):
workspace_dir: str
plugins: list[object]
diagnostics: list[object]
class SecretsAuditResponse(BaseModel):
version: int
status: str
findings: list[object]
class SecurityAuditResponse2(BaseModel):
report: object | None
secret_diagnostics: list[str]
class DaemonStatusResponse(BaseModel):
service: object | None
port: object | None
rpc: object | None
health: object | None
class PairingListResponse2(BaseModel):
channel: str
requests: list[object]
class QrCodeResponse2(BaseModel):
setup_code: str
gateway_url: str
auth: str
url_source: str
class UpdateStatusResponse2(BaseModel):
update: object | None
channel: object | None
class ModelAliasesResponse(BaseModel):
aliases: dict[str, str]
class ModelFallbacksResponse(BaseModel):
key: str
label: str
items: list[object]
class SkillUpdateResponse(BaseModel):
ok: bool
slug: str
version: str
error: str | None
class ModelsStatusResponse(BaseModel):
configPath: str | None = None
agentId: str | None = None
agentDir: str | None = None
defaultModel: str | None = None
resolvedDefault: str | None = None
fallbacks: list[str] = Field(default_factory=list)
imageModel: str | None = None
imageFallbacks: list[str] = Field(default_factory=list)
aliases: dict[str, str] = Field(default_factory=dict)
allowed: list[str] = Field(default_factory=list)
auth: dict[str, Any] = Field(default_factory=dict)
class ChannelsStatusResponse(BaseModel):
reachable: bool | None = None
channelAccounts: dict[str, Any] = Field(default_factory=dict)
channels: list[str] = Field(default_factory=list)
issues: list[dict[str, Any]] = Field(default_factory=list)
class ChannelsListResponse(BaseModel):
chat: dict[str, list[str]] = Field(default_factory=dict)
auth: list[dict[str, Any]] = Field(default_factory=list)
usage: dict[str, Any] | None = None
class HookInfoResponse(BaseModel):
name: str | None = None
description: str | None = None
source: str | None = None
pluginId: str | None = None
filePath: str | None = None
handlerPath: str | None = None
hookKey: str | None = None
emoji: str | None = None
homepage: str | None = None
events: list[str] = Field(default_factory=list)
enabledByConfig: bool | None = None
loadable: bool | None = None
requirementsSatisfied: bool | None = None
requirements: dict[str, Any] = Field(default_factory=dict)
error: str | None = None
raw: str | None = None
class HooksCheckResponse(BaseModel):
workspace_dir: str = ""
managed_hooks_dir: str = ""
hooks: list[dict[str, Any]] = Field(default_factory=list)
eligible: bool | None = None
verbose: bool | None = None
class PluginInspectEntry(BaseModel):
plugin: dict[str, Any] = Field(default_factory=dict)
shape: str | None = None
capabilityMode: str | None = None
capabilityCount: int = 0
capabilities: list[dict[str, Any]] = Field(default_factory=list)
typedHooks: list[dict[str, Any]] = Field(default_factory=list)
customHooks: list[dict[str, Any]] = Field(default_factory=list)
tools: list[dict[str, Any]] = Field(default_factory=list)
commands: list[str] = Field(default_factory=list)
cliCommands: list[str] = Field(default_factory=list)
services: list[str] = Field(default_factory=list)
gatewayMethods: list[str] = Field(default_factory=list)
mcpServers: list[dict[str, Any]] = Field(default_factory=list)
lspServers: list[dict[str, Any]] = Field(default_factory=list)
httpRouteCount: int = 0
bundleCapabilities: list[str] = Field(default_factory=list)
class PluginsInspectResponse(BaseModel):
inspect: list[dict[str, Any]] = Field(default_factory=list)
class AgentBindingItem(BaseModel):
agentId: str
match: dict[str, Any]
description: str
class AgentsBindingsResponse(BaseModel):
bindings: list[AgentBindingItem]
# ---------------------------------------------------------------------------
# Routes — use typed model methods and return Pydantic models directly
# ---------------------------------------------------------------------------
@router.get("/status")
async def api_openclaw_status(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> OpenClawStatus:
"""Read `openclaw status --json` and return a typed model."""
try:
return service.status_model()
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/sessions")
async def api_openclaw_sessions(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> SessionsResponse:
"""Read `openclaw sessions --json` and return a typed SessionsList."""
try:
result = service.list_sessions_model()
return SessionsResponse(sessions=result.sessions)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/sessions/{session_key:path}/history")
async def api_openclaw_session_history(
session_key: str,
limit: int = Query(20, ge=1, le=200),
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> SessionHistoryResponse:
"""Read session history and return a typed SessionHistory."""
try:
result = service.get_session_history_model(session_key, limit=limit)
return SessionHistoryResponse(
session_key=result.session_key,
session_id=result.session_id,
events=result.events,
history=result.events, # alias for compat
raw_text=result.raw_text,
)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/sessions/{session_key:path}")
async def api_openclaw_session_detail(
session_key: str,
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> SessionDetailResponse:
"""Resolve a single session and return it as a typed model."""
try:
session = service.get_session_model(session_key)
return SessionDetailResponse(session=session)
except KeyError as exc:
raise HTTPException(status_code=404, detail=f"session '{session_key}' not found") from exc
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/cron")
async def api_openclaw_cron(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> CronResponse:
"""Read `openclaw cron list --json` and return a typed CronList."""
try:
result = service.list_cron_jobs_model()
return CronResponse(cron=list(result.cron), jobs=list(result.jobs))
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/approvals")
async def api_openclaw_approvals(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> ApprovalsResponse:
"""Read `openclaw approvals get --json` and return a typed ApprovalsList."""
try:
result = service.list_approvals_model()
return ApprovalsResponse(
approvals=list(result.approvals),
pending=list(result.pending),
)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/agents")
async def api_openclaw_agents(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> AgentsResponse:
"""Read `openclaw agents list --json` and return a typed AgentsList."""
try:
result = service.list_agents_model()
return AgentsResponse(agents=list(result.agents))
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/agents/presence")
async def api_openclaw_agents_presence(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> dict[str, Any]:
"""Read runtime session presence for all agents from session files."""
result = service.agents_presence()
return result
# ---------------------------------------------------------------------------
# Write agents routes
# ---------------------------------------------------------------------------
class AgentAddResponse(BaseModel):
agentId: str
name: str
workspace: str
agentDir: str
model: str | None = None
bindings: dict[str, Any] = Field(default_factory=dict)
class AgentDeleteResponse(BaseModel):
agentId: str
workspace: str
agentDir: str
sessionsDir: str
removedBindings: list[str] = Field(default_factory=list)
removedAllow: list[str] = Field(default_factory=list)
class AgentBindResponse(BaseModel):
agentId: str
added: list[str] = Field(default_factory=list)
updated: list[str] = Field(default_factory=list)
skipped: list[str] = Field(default_factory=list)
conflicts: list[str] = Field(default_factory=list)
class AgentUnbindResponse(BaseModel):
agentId: str
removed: list[str] = Field(default_factory=list)
missing: list[str] = Field(default_factory=list)
conflicts: list[str] = Field(default_factory=list)
class AgentIdentityResponse(BaseModel):
agentId: str
identity: dict[str, Any] = Field(default_factory=dict)
workspace: str | None = None
identityFile: str | None = None
@router.post("/agents/add")
async def api_openclaw_agents_add(
name: str,
*,
workspace: str | None = None,
model: str | None = None,
agent_dir: str | None = None,
bind: list[str] | None = None,
non_interactive: bool = False,
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> AgentAddResponse:
"""Run `openclaw agents add <name>` and return JSON result."""
try:
result = service.agents_add(
name,
workspace=workspace,
model=model,
agent_dir=agent_dir,
bind=bind,
non_interactive=non_interactive,
)
return AgentAddResponse.model_validate(result, strict=False)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.post("/agents/delete/{id}")
async def api_openclaw_agents_delete(
id: str,
force: bool = False,
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> AgentDeleteResponse:
"""Run `openclaw agents delete <id> [--force]` and return JSON result."""
try:
result = service.agents_delete(id, force=force)
return AgentDeleteResponse.model_validate(result, strict=False)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.post("/agents/bind")
async def api_openclaw_agents_bind(
*,
agent: str | None = None,
bind: list[str] | None = None,
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> AgentBindResponse:
"""Run `openclaw agents bind [--agent <id>] [--bind <spec>]` and return JSON result."""
try:
result = service.agents_bind(agent=agent, bind=bind)
return AgentBindResponse.model_validate(result, strict=False)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.post("/agents/unbind")
async def api_openclaw_agents_unbind(
*,
agent: str | None = None,
bind: list[str] | None = None,
all: bool = False,
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> AgentUnbindResponse:
"""Run `openclaw agents unbind [--agent <id>] [--bind <spec>] [--all]` and return JSON result."""
try:
result = service.agents_unbind(agent=agent, bind=bind, all=all)
return AgentUnbindResponse.model_validate(result, strict=False)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.post("/agents/set-identity")
async def api_openclaw_agents_set_identity(
*,
agent: str | None = None,
workspace: str | None = None,
identity_file: str | None = None,
name: str | None = None,
emoji: str | None = None,
theme: str | None = None,
avatar: str | None = None,
from_identity: bool = False,
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> AgentIdentityResponse:
"""Run `openclaw agents set-identity` and return JSON result."""
try:
result = service.agents_set_identity(
agent=agent,
workspace=workspace,
identity_file=identity_file,
name=name,
emoji=emoji,
theme=theme,
avatar=avatar,
from_identity=from_identity,
)
return AgentIdentityResponse.model_validate(result, strict=False)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/skills")
async def api_openclaw_skills(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> SkillsResponse:
"""Read `openclaw skills list --json` and return a typed SkillStatusReport."""
try:
result = service.list_skills_model()
return SkillsResponse(
workspace_dir=result.workspace_dir,
managed_skills_dir=result.managed_skills_dir,
skills=list(result.skills),
)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/models")
async def api_openclaw_models(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> ModelsResponse:
"""Read `openclaw models list --json` and return a typed ModelsList."""
try:
result = service.list_models_model()
return ModelsResponse(models=list(result.models))
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/hooks")
async def api_openclaw_hooks(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> HooksResponse:
try:
result = service.list_hooks_model()
return HooksResponse(
workspace_dir=result.workspace_dir,
managed_hooks_dir=result.managed_hooks_dir,
hooks=list(result.hooks),
)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/plugins")
async def api_openclaw_plugins(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> PluginsResponse:
try:
result = service.list_plugins_model()
return PluginsResponse(
workspace_dir=result.workspace_dir,
plugins=list(result.plugins),
diagnostics=list(result.diagnostics),
)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/secrets-audit")
async def api_openclaw_secrets_audit(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> SecretsAuditResponse:
try:
result = service.secrets_audit_model()
return SecretsAuditResponse(
version=result.version,
status=result.status,
findings=list(result.findings),
)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/security-audit")
async def api_openclaw_security_audit(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> SecurityAuditResponse2:
try:
result = service.security_audit_model()
return SecurityAuditResponse2(
report=result.report.model_dump() if result.report else None,
secret_diagnostics=list(result.secret_diagnostics),
)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/daemon-status")
async def api_openclaw_daemon_status(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> DaemonStatusResponse:
try:
result = service.daemon_status_model()
return DaemonStatusResponse(
service=result.service.model_dump() if result.service else None,
port=result.port.model_dump() if result.port else None,
rpc=result.rpc.model_dump() if result.rpc else None,
health=result.health.model_dump() if result.health else None,
)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/pairing")
async def api_openclaw_pairing(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> PairingListResponse2:
try:
result = service.pairing_list_model()
return PairingListResponse2(
channel=result.channel,
requests=list(result.requests),
)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/qr")
async def api_openclaw_qr(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> QrCodeResponse2:
try:
result = service.qr_code_model()
return QrCodeResponse2(
setup_code=result.setup_code,
gateway_url=result.gateway_url,
auth=result.auth,
url_source=result.url_source,
)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/update-status")
async def api_openclaw_update_status(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> UpdateStatusResponse2:
try:
result = service.update_status_model()
return UpdateStatusResponse2(
update=result.update.model_dump() if result.update else None,
channel=result.channel.model_dump() if result.channel else None,
)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/models-aliases")
async def api_openclaw_models_aliases(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> ModelAliasesResponse:
try:
result = service.list_model_aliases_model()
return ModelAliasesResponse(aliases=result.aliases)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/models-fallbacks")
async def api_openclaw_models_fallbacks(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> ModelFallbacksResponse:
try:
result = service.list_model_fallbacks_model()
return ModelFallbacksResponse(
key=result.key,
label=result.label,
items=list(result.items),
)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/models-image-fallbacks")
async def api_openclaw_models_image_fallbacks(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> ModelFallbacksResponse:
try:
result = service.list_model_image_fallbacks_model()
return ModelFallbacksResponse(
key=result.key,
label=result.label,
items=list(result.items),
)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/skill-update")
async def api_openclaw_skill_update(
slug: str | None = None,
all: bool = False,
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> SkillUpdateResponse:
try:
result = service.skill_update_model(slug=slug, all=all)
return SkillUpdateResponse(
ok=result.ok,
slug=result.slug,
version=result.version,
error=result.error,
)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/models-status")
async def api_openclaw_models_status(
probe: bool = False,
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> ModelsStatusResponse:
"""Read `openclaw models status --json [--probe]` and return a typed dict."""
try:
result = service.models_status_model(probe=probe)
return ModelsStatusResponse.model_validate(result, strict=False)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/channels-status")
async def api_openclaw_channels_status(
probe: bool = False,
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> ChannelsStatusResponse:
"""Read `openclaw channels status --json [--probe]` and return a typed dict."""
try:
result = service.channels_status_model(probe=probe)
return ChannelsStatusResponse.model_validate(result, strict=False)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/channels-list")
async def api_openclaw_channels_list(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> ChannelsListResponse:
"""Read `openclaw channels list --json` and return a typed dict."""
try:
result = service.channels_list_model()
return ChannelsListResponse.model_validate(result, strict=False)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/hooks/info/{name}")
async def api_openclaw_hook_info(
name: str,
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> HookInfoResponse:
"""Read `openclaw hooks info <name> --json` and return a typed dict."""
try:
result = service.hook_info_model(name)
return HookInfoResponse.model_validate(result, strict=False)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/hooks/check")
async def api_openclaw_hooks_check(
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> HooksCheckResponse:
"""Read `openclaw hooks check --json` and return a typed dict."""
try:
result = service.hooks_check_model()
return HooksCheckResponse.model_validate(result, strict=False)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/plugins-inspect")
async def api_openclaw_plugins_inspect(
plugin_id: str | None = None,
all: bool = False,
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> PluginsInspectResponse:
"""Read `openclaw plugins inspect --json [--all]` and return a typed dict."""
try:
result = service.plugins_inspect_model(plugin_id=plugin_id, all=all)
inspect = result if isinstance(result, list) else result.get("inspect", [])
return PluginsInspectResponse(inspect=inspect)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
class AgentBindingItem(BaseModel):
agentId: str
match: dict[str, Any]
description: str
class AgentsBindingsResponse(BaseModel):
bindings: list[AgentBindingItem]
@router.get("/agents-bindings")
async def api_openclaw_agents_bindings(
agent: str | None = None,
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> AgentsBindingsResponse:
"""Read `openclaw agents bindings --json [--agent <id>]` and return bindings list."""
try:
result = service.agents_bindings_model(agent=agent)
bindings = result if isinstance(result, list) else []
return AgentsBindingsResponse(bindings=bindings)
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/gateway-status")
async def api_openclaw_gateway_status(
url: str | None = None,
token: str | None = None,
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> dict[str, Any]:
"""Read `openclaw gateway status --json [--url <url>] [--token <token>]`. Returns full gateway probe result."""
try:
result = service.gateway_status(url=url, token=token)
return result
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
@router.get("/memory-status")
async def api_openclaw_memory_status(
agent: str | None = None,
deep: bool = False,
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> list[dict[str, Any]]:
"""Read `openclaw memory status --json [--agent <id>] [--deep]`. Returns array of per-agent memory status."""
try:
result = service.memory_status(agent=agent, deep=deep)
return result if isinstance(result, list) else []
except OpenClawCliError as exc:
_raise_cli_http_error(exc)
class WorkspaceFilesResponse(BaseModel):
workspace: str
files: list[dict[str, Any]]
error: str | None = None
@router.get("/workspace-files")
async def api_openclaw_workspace_files(
workspace: str = Query(..., description="Path to the agent workspace directory"),
service: OpenClawCliService = Depends(get_openclaw_cli_service),
) -> WorkspaceFilesResponse:
"""List .md files in an OpenClaw agent workspace with their content previews."""
result = service.list_workspace_files(workspace)
return WorkspaceFilesResponse.model_validate(result, strict=False)

View File

@@ -8,6 +8,7 @@ import json
import logging
import os
import signal
import shutil
import subprocess
import sys
from datetime import datetime
@@ -16,20 +17,124 @@ from typing import Any, Dict, List, Optional
logger = logging.getLogger(__name__)
from fastapi import APIRouter, HTTPException, BackgroundTasks
from fastapi import APIRouter, BackgroundTasks, HTTPException, Request
from pydantic import BaseModel, Field
from backend.runtime.agent_runtime import AgentRuntimeState
from backend.runtime.manager import TradingRuntimeManager, get_global_runtime_manager
from backend.config.bootstrap_config import (
resolve_runtime_config,
update_bootstrap_values_for_run,
)
router = APIRouter(prefix="/api/runtime", tags=["runtime"])
runtime_manager: Optional[TradingRuntimeManager] = None
PROJECT_ROOT = Path(__file__).resolve().parents[2]
# Gateway process management
_gateway_process: Optional[subprocess.Popen] = None
_gateway_port: int = 8765
class RuntimeState:
"""Thread-safe singleton for managing runtime state.
Encapsulates runtime_manager, _gateway_process, and _gateway_port
with asyncio.Lock protection for concurrent access.
"""
_instance: Optional["RuntimeState"] = None
_lock: "threading.Lock" = __import__("threading").Lock()
def __new__(cls) -> "RuntimeState":
with cls._lock:
if cls._instance is None:
cls._instance = super().__new__(cls)
cls._instance._initialized = False
return cls._instance
def __init__(self) -> None:
if self._initialized:
return
self._runtime_manager: Optional[Any] = None
self._gateway_process: Optional[subprocess.Popen] = None
self._gateway_port: int = 8765
self._state_lock = asyncio.Lock()
self._initialized = True
@property
async def lock(self) -> asyncio.Lock:
"""Get the asyncio lock for state synchronization."""
return self._state_lock
@property
def runtime_manager(self) -> Optional[Any]:
"""Get the runtime manager (no lock - read only)."""
return self._runtime_manager
@runtime_manager.setter
def runtime_manager(self, value: Optional[Any]) -> None:
"""Set the runtime manager."""
self._runtime_manager = value
@property
def gateway_process(self) -> Optional[subprocess.Popen]:
"""Get the gateway process (no lock - read only)."""
return self._gateway_process
@gateway_process.setter
def gateway_process(self, value: Optional[subprocess.Popen]) -> None:
"""Set the gateway process."""
self._gateway_process = value
@property
def gateway_port(self) -> int:
"""Get the gateway port."""
return self._gateway_port
@gateway_port.setter
def gateway_port(self, value: int) -> None:
"""Set the gateway port."""
self._gateway_port = value
async def set_runtime_manager(self, manager: Any) -> None:
"""Set runtime manager with lock protection."""
async with self._state_lock:
self._runtime_manager = manager
async def get_runtime_manager(self) -> Optional[Any]:
"""Get runtime manager with lock protection."""
async with self._state_lock:
return self._runtime_manager
async def set_gateway_process(self, process: Optional[subprocess.Popen]) -> None:
"""Set gateway process with lock protection."""
async with self._state_lock:
self._gateway_process = process
async def get_gateway_process(self) -> Optional[subprocess.Popen]:
"""Get gateway process with lock protection."""
async with self._state_lock:
return self._gateway_process
async def set_gateway_port(self, port: int) -> None:
"""Set gateway port with lock protection."""
async with self._state_lock:
self._gateway_port = port
async def get_gateway_port(self) -> int:
"""Get gateway port with lock protection."""
async with self._state_lock:
return self._gateway_port
# Singleton instance
_runtime_state = RuntimeState()
def get_runtime_state() -> RuntimeState:
"""Get the RuntimeState singleton instance."""
return _runtime_state
# Backward compatibility: module-level runtime_manager for external imports
# This is set by register_runtime_manager() for backward compatibility
runtime_manager: Optional[Any] = None
class RunContextResponse(BaseModel):
@@ -62,6 +167,8 @@ class RuntimeEventsResponse(BaseModel):
class LaunchConfig(BaseModel):
"""Configuration for launching a new trading task."""
launch_mode: str = Field(default="fresh", description="启动形式: fresh, restore")
restore_run_id: Optional[str] = Field(default=None, description="历史任务 run_id用于恢复启动")
tickers: List[str] = Field(default_factory=list, description="股票池")
schedule_mode: str = Field(default="daily", description="调度模式: daily, interval")
interval_minutes: int = Field(default=60, ge=1, description="间隔分钟数")
@@ -74,7 +181,6 @@ class LaunchConfig(BaseModel):
start_date: Optional[str] = Field(default=None, description="回测开始日期 YYYY-MM-DD")
end_date: Optional[str] = Field(default=None, description="回测结束日期 YYYY-MM-DD")
poll_interval: int = Field(default=10, ge=1, le=300, description="市场数据轮询间隔(秒)")
enable_mock: bool = Field(default=False, description="是否启用模拟模式(使用模拟价格数据)")
class LaunchResponse(BaseModel):
@@ -85,17 +191,61 @@ class LaunchResponse(BaseModel):
message: str
class RuntimeHistoryItem(BaseModel):
run_id: str
run_dir: str
updated_at: Optional[str] = None
total_trades: int = 0
total_asset_value: Optional[float] = None
bootstrap: Dict[str, Any] = Field(default_factory=dict)
class RuntimeHistoryResponse(BaseModel):
runs: List[RuntimeHistoryItem]
class StopResponse(BaseModel):
status: str
message: str
class CleanupResponse(BaseModel):
status: str
kept: int
pruned_run_ids: List[str]
class GatewayStatusResponse(BaseModel):
is_running: bool
port: int
run_id: Optional[str] = None
class RuntimeConfigResponse(BaseModel):
run_id: str
is_running: bool
gateway_port: int
bootstrap: Dict[str, Any]
resolved: Dict[str, Any]
class RuntimeLogResponse(BaseModel):
run_id: Optional[str] = None
is_running: bool
log_path: Optional[str] = None
content: str = ""
class UpdateRuntimeConfigRequest(BaseModel):
schedule_mode: Optional[str] = None
interval_minutes: Optional[int] = Field(default=None, ge=1)
trigger_time: Optional[str] = None
max_comm_cycles: Optional[int] = Field(default=None, ge=1)
initial_cash: Optional[float] = Field(default=None, gt=0)
margin_requirement: Optional[float] = Field(default=None, ge=0)
enable_memory: Optional[bool] = None
def _generate_run_id() -> str:
"""Generate timestamp-based run ID: YYYYMMDD_HHMMSS"""
return datetime.now().strftime("%Y%m%d_%H%M%S")
@@ -106,6 +256,128 @@ def _get_run_dir(run_id: str) -> Path:
return PROJECT_ROOT / "runs" / run_id
def _load_run_snapshot(run_id: str) -> Dict[str, Any]:
"""Load a specific run snapshot by run_id."""
snapshot_path = _get_run_dir(run_id) / "state" / "runtime_state.json"
if not snapshot_path.exists():
raise HTTPException(status_code=404, detail=f"Run snapshot not found: {run_id}")
return json.loads(snapshot_path.read_text(encoding="utf-8"))
def _copy_path_if_exists(src: Path, dst: Path) -> None:
if not src.exists():
return
if src.is_dir():
shutil.copytree(src, dst, dirs_exist_ok=True)
else:
dst.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(src, dst)
def _restore_run_assets(source_run_id: str, target_run_dir: Path) -> None:
"""Seed a fresh run directory from a historical run snapshot."""
source_run_dir = _get_run_dir(source_run_id)
if not source_run_dir.exists():
raise HTTPException(status_code=404, detail=f"Source run not found: {source_run_id}")
for relative in [
"team_dashboard",
"agents",
"skills",
"memory",
"state/server_state.json",
"state/runtime.db",
"state/research.db",
]:
_copy_path_if_exists(source_run_dir / relative, target_run_dir / relative)
def _list_runs(limit: int = 50) -> list[RuntimeHistoryItem]:
runs_root = PROJECT_ROOT / "runs"
if not runs_root.exists():
return []
items: list[RuntimeHistoryItem] = []
run_dirs = sorted(
[path for path in runs_root.iterdir() if path.is_dir()],
key=lambda path: path.stat().st_mtime,
reverse=True,
)
for run_dir in run_dirs[: max(1, int(limit))]:
run_id = run_dir.name
runtime_state_path = run_dir / "state" / "runtime_state.json"
summary_path = run_dir / "team_dashboard" / "summary.json"
bootstrap: Dict[str, Any] = {}
updated_at: Optional[str] = None
total_trades = 0
total_asset_value: Optional[float] = None
if runtime_state_path.exists():
try:
snapshot = json.loads(runtime_state_path.read_text(encoding="utf-8"))
context = snapshot.get("context") or {}
bootstrap = dict(context.get("bootstrap_values") or {})
updated_at = snapshot.get("events", [{}])[-1].get("timestamp") if snapshot.get("events") else None
except Exception:
bootstrap = {}
if summary_path.exists():
try:
summary = json.loads(summary_path.read_text(encoding="utf-8"))
total_trades = int(summary.get("totalTrades") or 0)
total_asset_value = float(summary.get("totalAssetValue")) if summary.get("totalAssetValue") is not None else None
except Exception:
total_trades = 0
total_asset_value = None
items.append(
RuntimeHistoryItem(
run_id=run_id,
run_dir=str(run_dir),
updated_at=updated_at,
total_trades=total_trades,
total_asset_value=total_asset_value,
bootstrap=bootstrap,
)
)
return items
def _is_timestamped_run_dir(path: Path) -> bool:
try:
datetime.strptime(path.name, "%Y%m%d_%H%M%S")
return True
except ValueError:
return False
def _prune_old_timestamped_runs(*, keep: int = 20, exclude_run_ids: Optional[set[str]] = None) -> list[str]:
"""Prune old timestamped run directories, preserving the newest N and excluded ids."""
exclude = exclude_run_ids or set()
runs_root = PROJECT_ROOT / "runs"
if not runs_root.exists():
return []
candidates = sorted(
[
path
for path in runs_root.iterdir()
if path.is_dir() and _is_timestamped_run_dir(path) and path.name not in exclude
],
key=lambda path: path.name,
reverse=True,
)
pruned: list[str] = []
for path in candidates[max(0, keep):]:
shutil.rmtree(path, ignore_errors=True)
pruned.append(path.name)
return pruned
def _find_available_port(start_port: int = 8765, max_port: int = 9000) -> int:
"""Find an available port for Gateway."""
import socket
@@ -117,32 +389,42 @@ def _find_available_port(start_port: int = 8765, max_port: int = 9000) -> int:
def _is_gateway_running() -> bool:
"""Check if Gateway process is running."""
global _gateway_process
if _gateway_process is None:
"""Check if Gateway process is running.
Checks both the internally-managed gateway process and falls back to
port availability (for externally-managed gateway processes).
"""
process = _runtime_state.gateway_process
if process is not None and process.poll() is None:
return True
# Fallback: check if the gateway port is in use (for externally started gateway)
import socket
try:
with socket.create_connection(("127.0.0.1", _runtime_state.gateway_port), timeout=1):
return True
except OSError:
return False
return _gateway_process.poll() is None
def _stop_gateway() -> bool:
"""Stop the Gateway process."""
global _gateway_process
if _gateway_process is None:
process = _runtime_state.gateway_process
if process is None:
return False
try:
# Try graceful shutdown first
_gateway_process.terminate()
process.terminate()
try:
_gateway_process.wait(timeout=5)
process.wait(timeout=5)
except subprocess.TimeoutExpired:
# Force kill if graceful shutdown fails
_gateway_process.kill()
_gateway_process.wait()
process.kill()
process.wait()
except Exception as e:
logger.warning(f"Error during gateway shutdown: {e}")
finally:
_gateway_process = None
_runtime_state.gateway_process = None
return True
@@ -167,29 +449,29 @@ def _start_gateway_process(
"--bootstrap", json.dumps(bootstrap)
]
# Start process
process = subprocess.Popen(
cmd,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
cwd=PROJECT_ROOT
)
log_path = run_dir / "logs" / "gateway.log"
log_path.parent.mkdir(parents=True, exist_ok=True)
log_file = log_path.open("ab")
try:
process = subprocess.Popen(
cmd,
env=env,
stdout=log_file,
stderr=subprocess.STDOUT,
cwd=PROJECT_ROOT
)
finally:
log_file.close()
return process
@router.get("/context", response_model=RunContextResponse)
async def get_run_context() -> RunContextResponse:
"""Return the most recent run context."""
snapshot_path = PROJECT_ROOT.glob("runs/*/state/runtime_state.json")
snapshots = sorted(snapshot_path, key=lambda p: p.stat().st_mtime, reverse=True)
if not snapshots:
raise HTTPException(status_code=404, detail="No run context available")
latest = json.loads(snapshots[0].read_text(encoding="utf-8"))
context = latest.get("context")
"""Return active runtime context, or latest persisted context when stopped."""
snapshot = _get_active_runtime_snapshot() if _is_gateway_running() else _load_latest_runtime_snapshot()
context = snapshot.get("context")
if context is None:
raise HTTPException(status_code=404, detail="Run context is not ready")
@@ -202,15 +484,9 @@ async def get_run_context() -> RunContextResponse:
@router.get("/agents", response_model=RuntimeAgentsResponse)
async def get_runtime_agents() -> RuntimeAgentsResponse:
"""Return agent states from the most recent run."""
snapshot_path = PROJECT_ROOT.glob("runs/*/state/runtime_state.json")
snapshots = sorted(snapshot_path, key=lambda p: p.stat().st_mtime, reverse=True)
if not snapshots:
raise HTTPException(status_code=404, detail="No runtime state available")
latest = json.loads(snapshots[0].read_text(encoding="utf-8"))
agents = latest.get("agents", [])
"""Return agent states from the active runtime, or latest persisted run."""
snapshot = _get_active_runtime_snapshot() if _is_gateway_running() else _load_latest_runtime_snapshot()
agents = snapshot.get("agents", [])
return RuntimeAgentsResponse(
agents=[RuntimeAgentState(**a) for a in agents]
@@ -219,58 +495,219 @@ async def get_runtime_agents() -> RuntimeAgentsResponse:
@router.get("/events", response_model=RuntimeEventsResponse)
async def get_runtime_events() -> RuntimeEventsResponse:
"""Return events from the most recent run."""
snapshot_path = PROJECT_ROOT.glob("runs/*/state/runtime_state.json")
snapshots = sorted(snapshot_path, key=lambda p: p.stat().st_mtime, reverse=True)
if not snapshots:
raise HTTPException(status_code=404, detail="No runtime state available")
latest = json.loads(snapshots[0].read_text(encoding="utf-8"))
events = latest.get("events", [])
"""Return events from the active runtime, or latest persisted run."""
snapshot = _get_active_runtime_snapshot() if _is_gateway_running() else _load_latest_runtime_snapshot()
events = snapshot.get("events", [])
return RuntimeEventsResponse(
events=[RuntimeEvent(**e) for e in events]
)
@router.get("/history", response_model=RuntimeHistoryResponse)
async def get_runtime_history(limit: int = 20) -> RuntimeHistoryResponse:
"""List recent historical runs for restore/start selection."""
return RuntimeHistoryResponse(runs=_list_runs(limit=limit))
@router.get("/gateway/status", response_model=GatewayStatusResponse)
async def get_gateway_status() -> GatewayStatusResponse:
"""Get Gateway process status and port."""
global _gateway_port
is_running = _is_gateway_running()
run_id = None
if is_running:
# Try to find run_id from runtime state
snapshot_path = PROJECT_ROOT.glob("runs/*/state/runtime_state.json")
snapshots = sorted(snapshot_path, key=lambda p: p.stat().st_mtime, reverse=True)
if snapshots:
try:
latest = json.loads(snapshots[0].read_text(encoding="utf-8"))
run_id = latest.get("context", {}).get("config_name")
except Exception as e:
logger.warning(f"Failed to parse latest snapshot: {e}")
try:
run_id = _get_active_runtime_context().get("config_name")
except Exception as e:
logger.warning(f"Failed to resolve active runtime context: {e}")
return GatewayStatusResponse(
is_running=is_running,
port=_gateway_port,
port=_runtime_state.gateway_port,
run_id=run_id
)
@router.get("/gateway/port")
async def get_gateway_port() -> Dict[str, Any]:
async def get_gateway_port(request: Request) -> Dict[str, Any]:
"""Get WebSocket Gateway port for frontend connection."""
global _gateway_port
gateway_port = _runtime_state.gateway_port
return {
"port": _gateway_port,
"port": gateway_port,
"is_running": _is_gateway_running(),
"ws_url": f"ws://localhost:{_gateway_port}"
"ws_url": _build_gateway_ws_url(request, gateway_port),
}
@router.get("/logs", response_model=RuntimeLogResponse)
async def get_runtime_logs() -> RuntimeLogResponse:
"""Return current runtime log tail, or the latest run log if runtime is stopped."""
try:
context = _get_active_runtime_context() if _is_gateway_running() else _get_runtime_context_from_latest_snapshot()
except HTTPException:
return RuntimeLogResponse(is_running=False, content="")
run_id = str(context.get("config_name") or "").strip() or None
log_path = _get_gateway_log_path_for_run(run_id) if run_id else None
content = _read_log_tail(log_path) if log_path else ""
return RuntimeLogResponse(
run_id=run_id,
is_running=_is_gateway_running(),
log_path=str(log_path) if log_path else None,
content=content,
)
def _build_gateway_ws_url(request: Request, port: int) -> str:
"""Build a proxy-safe Gateway WebSocket URL."""
forwarded_proto = request.headers.get("x-forwarded-proto", "").split(",")[0].strip()
scheme = forwarded_proto or request.url.scheme
ws_scheme = "wss" if scheme == "https" else "ws"
forwarded_host = request.headers.get("x-forwarded-host", "").split(",")[0].strip()
host = forwarded_host or request.url.hostname or "localhost"
if ":" in host and not host.startswith("["):
host = host.split(":", 1)[0]
return f"{ws_scheme}://{host}:{port}"
def _load_latest_runtime_snapshot() -> Dict[str, Any]:
"""Load the latest persisted runtime snapshot."""
snapshots = sorted(
PROJECT_ROOT.glob("runs/*/state/runtime_state.json"),
key=lambda p: p.stat().st_mtime,
reverse=True,
)
if not snapshots:
raise HTTPException(status_code=404, detail="No runtime information available")
return json.loads(snapshots[0].read_text(encoding="utf-8"))
def _get_active_runtime_snapshot() -> Dict[str, Any]:
"""Return the active runtime snapshot, preferring in-memory manager state."""
if not _is_gateway_running():
raise HTTPException(status_code=404, detail="No runtime is currently running")
manager = _runtime_state.runtime_manager
if manager is not None and hasattr(manager, "build_snapshot"):
snapshot = manager.build_snapshot()
context = snapshot.get("context") or {}
if context.get("config_name"):
return snapshot
return _load_latest_runtime_snapshot()
def _get_runtime_context_from_latest_snapshot() -> Dict[str, Any]:
"""Return the latest persisted runtime context regardless of active process state."""
latest = _load_latest_runtime_snapshot()
context = latest.get("context") or {}
if not context.get("config_name"):
raise HTTPException(status_code=404, detail="No runtime context available")
return context
def _get_gateway_log_path_for_run(run_id: str) -> Path:
return _get_run_dir(run_id) / "logs" / "gateway.log"
def _read_log_tail(path: Path, max_chars: int = 120_000) -> str:
if not path.exists() or not path.is_file():
return ""
text = path.read_text(encoding="utf-8", errors="replace")
if len(text) <= max_chars:
return text
return text[-max_chars:]
def _get_current_runtime_context() -> Dict[str, Any]:
"""Return the active runtime context from the latest snapshot."""
if not _is_gateway_running():
raise HTTPException(status_code=404, detail="No runtime is currently running")
snapshot = _get_active_runtime_snapshot()
context = snapshot.get("context") or {}
if not context.get("config_name"):
raise HTTPException(status_code=404, detail="No runtime context available")
return context
def _get_active_runtime_context() -> Dict[str, Any]:
"""Return the active runtime context, preferring in-memory runtime manager state."""
return _get_current_runtime_context()
def _resolve_runtime_response(run_id: str) -> RuntimeConfigResponse:
"""Build a normalized runtime config response for the active run."""
context = _get_current_runtime_context()
bootstrap = dict(context.get("bootstrap_values") or {})
resolved = resolve_runtime_config(
project_root=PROJECT_ROOT,
config_name=run_id,
enable_memory=bool(bootstrap.get("enable_memory", False)),
schedule_mode=str(bootstrap.get("schedule_mode", "daily")),
interval_minutes=int(bootstrap.get("interval_minutes", 60) or 60),
trigger_time=str(bootstrap.get("trigger_time", "09:30") or "09:30"),
)
return RuntimeConfigResponse(
run_id=run_id,
is_running=True,
gateway_port=_runtime_state.gateway_port,
bootstrap=bootstrap,
resolved=resolved,
)
def _normalize_runtime_config_updates(
request: UpdateRuntimeConfigRequest,
) -> Dict[str, Any]:
"""Validate and normalize runtime config updates."""
updates: Dict[str, Any] = {}
if request.schedule_mode is not None:
schedule_mode = str(request.schedule_mode).strip().lower()
if schedule_mode not in {"daily", "intraday"}:
raise HTTPException(
status_code=400,
detail="schedule_mode must be 'daily' or 'intraday'",
)
updates["schedule_mode"] = schedule_mode
if request.interval_minutes is not None:
updates["interval_minutes"] = int(request.interval_minutes)
if request.trigger_time is not None:
trigger_time = str(request.trigger_time).strip()
if trigger_time and trigger_time != "now":
try:
datetime.strptime(trigger_time, "%H:%M")
except ValueError as exc:
raise HTTPException(
status_code=400,
detail="trigger_time must use HH:MM or 'now'",
) from exc
updates["trigger_time"] = trigger_time or "09:30"
if request.max_comm_cycles is not None:
updates["max_comm_cycles"] = int(request.max_comm_cycles)
if request.initial_cash is not None:
updates["initial_cash"] = float(request.initial_cash)
if request.margin_requirement is not None:
updates["margin_requirement"] = float(request.margin_requirement)
if request.enable_memory is not None:
updates["enable_memory"] = bool(request.enable_memory)
if not updates:
raise HTTPException(status_code=400, detail="No runtime config updates provided")
return updates
@router.post("/start", response_model=LaunchResponse)
async def start_runtime(
config: LaunchConfig,
@@ -284,33 +721,59 @@ async def start_runtime(
4. Start Gateway as subprocess (Data Plane)
5. Return Gateway port for WebSocket connection
"""
global _gateway_process, _gateway_port
# Lazy import to avoid circular dependency
from backend.runtime.manager import TradingRuntimeManager
# 1. Stop existing Gateway
if _is_gateway_running():
_stop_gateway()
await asyncio.sleep(1) # Wait for port release
# 2. Generate run ID and directory
run_id = _generate_run_id()
run_dir = _get_run_dir(run_id)
launch_mode = str(config.launch_mode or "fresh").strip().lower()
if launch_mode not in {"fresh", "restore"}:
raise HTTPException(status_code=400, detail="launch_mode must be 'fresh' or 'restore'")
# 3. Prepare bootstrap config
bootstrap = {
"tickers": config.tickers,
"schedule_mode": config.schedule_mode,
"interval_minutes": config.interval_minutes,
"trigger_time": config.trigger_time,
"max_comm_cycles": config.max_comm_cycles,
"initial_cash": config.initial_cash,
"margin_requirement": config.margin_requirement,
"enable_memory": config.enable_memory,
"mode": config.mode,
"start_date": config.start_date,
"end_date": config.end_date,
"poll_interval": config.poll_interval,
"enable_mock": config.enable_mock,
}
# 2. Resolve run ID, directory, and bootstrap
if launch_mode == "restore":
restore_run_id = str(config.restore_run_id or "").strip()
if not restore_run_id:
raise HTTPException(status_code=400, detail="restore_run_id is required when launch_mode=restore")
snapshot = _load_run_snapshot(restore_run_id)
context = snapshot.get("context") or {}
if not context.get("config_name"):
raise HTTPException(status_code=404, detail=f"Run context not found: {restore_run_id}")
run_id = restore_run_id
run_dir = _get_run_dir(run_id)
bootstrap = dict(context.get("bootstrap_values") or {})
bootstrap["launch_mode"] = "restore"
bootstrap["restore_run_id"] = restore_run_id
else:
run_id = _generate_run_id()
run_dir = _get_run_dir(run_id)
bootstrap = {
"launch_mode": "fresh",
"restore_run_id": None,
"tickers": config.tickers,
"schedule_mode": config.schedule_mode,
"interval_minutes": config.interval_minutes,
"trigger_time": config.trigger_time,
"max_comm_cycles": config.max_comm_cycles,
"initial_cash": config.initial_cash,
"margin_requirement": config.margin_requirement,
"enable_memory": config.enable_memory,
"mode": config.mode,
"start_date": config.start_date,
"end_date": config.end_date,
"poll_interval": config.poll_interval,
}
retention_keep = max(1, int(os.getenv("RUNS_RETENTION_COUNT", "20") or "20"))
pruned_run_ids = _prune_old_timestamped_runs(
keep=retention_keep,
exclude_run_ids={run_id},
)
if pruned_run_ids:
logger.info("Pruned old run directories: %s", ", ".join(pruned_run_ids))
# 4. Create runtime manager
manager = TradingRuntimeManager(
@@ -325,25 +788,28 @@ async def start_runtime(
_write_bootstrap_md(run_dir, bootstrap)
# 6. Find available port and start Gateway process
_gateway_port = _find_available_port(start_port=8765)
gateway_port = _find_available_port(start_port=8765)
_runtime_state.gateway_port = gateway_port
try:
_gateway_process = _start_gateway_process(
process = _start_gateway_process(
run_id=run_id,
run_dir=run_dir,
bootstrap=bootstrap,
port=_gateway_port
port=gateway_port
)
_runtime_state.gateway_process = process
# Wait briefly to check if process started successfully
await asyncio.sleep(2)
if not _is_gateway_running():
stdout, stderr = _gateway_process.communicate(timeout=1)
_gateway_process = None
_runtime_state.gateway_process = None
log_path = _get_gateway_log_path_for_run(run_id)
log_tail = _read_log_tail(log_path, max_chars=4000)
raise HTTPException(
status_code=500,
detail=f"Gateway failed to start: {stderr.decode() if stderr else 'Unknown error'}"
detail=f"Gateway failed to start: {log_tail or 'Unknown error'}"
)
except Exception as e:
@@ -354,16 +820,44 @@ async def start_runtime(
run_id=run_id,
status="started",
run_dir=str(run_dir),
gateway_port=_gateway_port,
message=f"Runtime started with run_id: {run_id}, Gateway on port: {_gateway_port}",
gateway_port=gateway_port,
message=f"Runtime started with run_id: {run_id}, Gateway on port: {gateway_port}",
)
@router.get("/config", response_model=RuntimeConfigResponse)
async def get_runtime_config() -> RuntimeConfigResponse:
"""Return the current runtime bootstrap and resolved settings."""
context = _get_current_runtime_context()
return _resolve_runtime_response(context["config_name"])
@router.put("/config", response_model=RuntimeConfigResponse)
async def update_runtime_config(
request: UpdateRuntimeConfigRequest,
) -> RuntimeConfigResponse:
"""Persist selected runtime configuration updates for the active run."""
context = _get_current_runtime_context()
run_id = context["config_name"]
updates = _normalize_runtime_config_updates(request)
updated = update_bootstrap_values_for_run(PROJECT_ROOT, run_id, updates)
manager = _runtime_state.runtime_manager
if manager is not None and getattr(manager, "config_name", None) == run_id:
manager.bootstrap.update(updates)
if getattr(manager, "context", None) is not None:
manager.context.bootstrap_values.update(updates)
if hasattr(manager, "_persist_snapshot"):
manager._persist_snapshot()
response = _resolve_runtime_response(run_id)
response.bootstrap = dict(updated.values)
return response
@router.post("/stop", response_model=StopResponse)
async def stop_runtime(force: bool = True) -> StopResponse:
"""Stop the current running runtime."""
global _gateway_process
was_running = _is_gateway_running()
if not was_running:
@@ -381,6 +875,25 @@ async def stop_runtime(force: bool = True) -> StopResponse:
)
@router.post("/cleanup", response_model=CleanupResponse)
async def cleanup_old_runs(keep: int = 20) -> CleanupResponse:
"""Prune old timestamped run directories while preserving named runs."""
keep_count = max(1, int(keep))
exclude: set[str] = set()
if _is_gateway_running():
try:
active_context = _get_active_runtime_context()
active_run_id = str(active_context.get("config_name") or "").strip()
if active_run_id:
exclude.add(active_run_id)
except HTTPException:
pass
pruned = _prune_old_timestamped_runs(keep=keep_count, exclude_run_ids=exclude)
return CleanupResponse(status="ok", kept=keep_count, pruned_run_ids=pruned)
@router.post("/restart")
async def restart_runtime(
config: LaunchConfig,
@@ -407,35 +920,31 @@ async def get_current_runtime():
if not _is_gateway_running():
raise HTTPException(status_code=404, detail="No runtime is currently running")
# Find latest runtime state
snapshot_path = PROJECT_ROOT.glob("runs/*/state/runtime_state.json")
snapshots = sorted(snapshot_path, key=lambda p: p.stat().st_mtime, reverse=True)
if not snapshots:
raise HTTPException(status_code=404, detail="No runtime information available")
latest = json.loads(snapshots[0].read_text(encoding="utf-8"))
context = latest.get("context", {})
context = _get_active_runtime_context()
return {
"run_id": context.get("config_name"),
"run_dir": context.get("run_dir"),
"is_running": True,
"gateway_port": _gateway_port,
"gateway_port": _runtime_state.gateway_port,
"bootstrap": context.get("bootstrap_values", {}),
}
def register_runtime_manager(manager: TradingRuntimeManager) -> None:
def register_runtime_manager(manager: Any) -> None:
"""Allow other modules to expose the runtime manager to the API."""
global runtime_manager
runtime_manager = manager
# Also update the RuntimeState singleton for internal consistency
_runtime_state.runtime_manager = manager
def unregister_runtime_manager() -> None:
"""Drop the runtime manager reference."""
global runtime_manager
runtime_manager = None
# Also update the RuntimeState singleton for internal consistency
_runtime_state.runtime_manager = None
def _write_bootstrap_md(run_dir: Path, bootstrap: Dict[str, Any]) -> None:

View File

@@ -1,115 +0,0 @@
# -*- coding: utf-8 -*-
"""
FastAPI Application - REST API for EvoTraders
Provides HTTP endpoints for:
- Agent management
- Workspace management
- Tool guard operations
- Health checks
"""
from contextlib import asynccontextmanager
from pathlib import Path
from typing import AsyncGenerator
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from backend.api import agents_router, workspaces_router, guard_router, runtime_router
from backend.agents import AgentFactory, WorkspaceManager, get_registry
# Global instances (initialized on startup)
agent_factory: AgentFactory | None = None
workspace_manager: WorkspaceManager | None = None
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncGenerator:
"""
Application lifespan manager.
Initializes global services on startup and cleans up on shutdown.
"""
global agent_factory, workspace_manager
# Startup: Initialize services
project_root = Path(__file__).parent.parent
# Initialize workspace manager
workspace_manager = WorkspaceManager(project_root=project_root)
# Initialize agent factory
agent_factory = AgentFactory(project_root=project_root)
# Ensure workspaces root exists
agent_factory.workspaces_root.mkdir(parents=True, exist_ok=True)
# Get or create global registry
registry = get_registry()
print(f"✓ EvoTraders API started")
print(f" - Workspaces root: {agent_factory.workspaces_root}")
print(f" - Registered agents: {registry.get_agent_count()}")
yield
# Shutdown: Cleanup
print("✓ EvoTraders API shutting down")
# Create FastAPI application
app = FastAPI(
title="EvoTraders API",
description="REST API for the EvoTraders multi-agent trading system",
version="0.1.0",
lifespan=lifespan,
)
# CORS middleware
app.add_middleware(
CORSMiddleware,
allow_origins=["*"], # Configure appropriately for production
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Health check endpoint
@app.get("/health")
async def health_check():
"""Health check endpoint."""
registry = get_registry()
return {
"status": "healthy",
"version": "0.1.0",
"agents_registered": registry.get_agent_count(),
"workspaces_available": len(workspace_manager.list_workspaces()) if workspace_manager else 0,
}
# API status endpoint
@app.get("/api/status")
async def api_status():
"""Get API status and system information."""
registry = get_registry()
stats = registry.get_stats()
return {
"status": "operational",
"registry": stats,
}
# Include routers
app.include_router(workspaces_router)
app.include_router(agents_router)
app.include_router(guard_router)
app.include_router(runtime_router)
# Main entry point for running with uvicorn
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8000)

34
backend/apps/__init__.py Normal file
View File

@@ -0,0 +1,34 @@
# -*- coding: utf-8 -*-
"""Application surfaces for progressive service extraction."""
from .agent_service import app as agent_app
from .agent_service import create_app as create_agent_app
from .news_service import app as news_app
from .news_service import create_app as create_news_app
from .openclaw_service import app as openclaw_app
from .openclaw_service import create_app as create_openclaw_app
from .runtime_service import app as runtime_app
from .runtime_service import create_app as create_runtime_app
from .trading_service import app as trading_app
from .trading_service import create_app as create_trading_app
from .cors import add_cors_middleware, get_cors_origins
app = agent_app
create_app = create_agent_app
__all__ = [
"app",
"create_app",
"agent_app",
"create_agent_app",
"news_app",
"create_news_app",
"openclaw_app",
"create_openclaw_app",
"runtime_app",
"create_runtime_app",
"trading_app",
"create_trading_app",
"add_cors_middleware",
"get_cors_origins",
]

View File

@@ -0,0 +1,89 @@
# -*- coding: utf-8 -*-
"""Agent control-plane FastAPI surface."""
from __future__ import annotations
from contextlib import asynccontextmanager
from pathlib import Path
from typing import AsyncGenerator
from fastapi import FastAPI
from backend.apps.cors import add_cors_middleware
from backend.api import agents_router, guard_router, workspaces_router
from backend.agents import AgentFactory, WorkspaceManager, get_registry
# Global instances (initialized on startup)
agent_factory: AgentFactory | None = None
workspace_manager: WorkspaceManager | None = None
def create_app(project_root: Path | None = None) -> FastAPI:
"""Create the agent control-plane app."""
resolved_project_root = project_root or Path(__file__).resolve().parents[2]
@asynccontextmanager
async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
"""Initialize workspace and registry state for the control plane."""
global agent_factory, workspace_manager
workspace_manager = WorkspaceManager(project_root=resolved_project_root)
agent_factory = AgentFactory(project_root=resolved_project_root)
agent_factory.workspaces_root.mkdir(parents=True, exist_ok=True)
registry = get_registry()
print("✓ 大时代 API started")
print(f" - Workspaces root: {agent_factory.workspaces_root}")
print(f" - Registered agents: {registry.get_agent_count()}")
yield
print("✓ 大时代 API shutting down")
app = FastAPI(
title="大时代 Agent Service",
description="REST API for the 大时代 multi-agent control plane",
version="0.1.0",
lifespan=lifespan,
)
add_cors_middleware(app)
@app.get("/health")
async def health_check() -> dict[str, object]:
"""Health check endpoint."""
registry = get_registry()
return {
"status": "healthy",
"version": "0.1.0",
"agents_registered": registry.get_agent_count(),
"workspaces_available": (
len(workspace_manager.list_workspaces())
if workspace_manager
else 0
),
}
@app.get("/api/status")
async def api_status() -> dict[str, object]:
"""Get API status and registry information."""
registry = get_registry()
return {
"status": "operational",
"registry": registry.get_stats(),
}
app.include_router(workspaces_router)
app.include_router(agents_router)
app.include_router(guard_router)
return app
app = create_app()
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8000)

30
backend/apps/cors.py Normal file
View File

@@ -0,0 +1,30 @@
# -*- coding: utf-8 -*-
"""Shared CORS configuration for all microservice apps."""
import os
from typing import Sequence
from fastapi.middleware.cors import CORSMiddleware
def get_cors_origins() -> Sequence[str]:
"""Get allowed CORS origins from environment variable.
Defaults to ["*"] for backward compatibility.
Set CORS_ALLOWED_ORIGINS env var (comma-separated) in production.
"""
origins = os.getenv("CORS_ALLOWED_ORIGINS", "").strip()
if not origins:
return ["*"]
return [o.strip() for o in origins.split(",") if o.strip()]
def add_cors_middleware(app: "FastAPI") -> None:
"""Add CORS middleware to app with environment-configured origins."""
app.add_middleware(
CORSMiddleware,
allow_origins=get_cors_origins(),
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)

View File

@@ -0,0 +1,154 @@
# -*- coding: utf-8 -*-
"""News and explain FastAPI surface."""
from __future__ import annotations
from typing import Any
from fastapi import Depends, FastAPI, Query
from backend.apps.cors import add_cors_middleware
from backend.data.market_store import MarketStore
from backend.domains import news as news_domain
def get_market_store() -> MarketStore:
"""Get the MarketStore singleton dependency."""
return MarketStore.get_instance()
def create_app() -> FastAPI:
"""Create the news/explain service app."""
app = FastAPI(
title="大时代 News Service",
description="Read-only news enrichment and explain service surface extracted from the monolith",
version="0.1.0",
)
add_cors_middleware(app)
@app.get("/health")
async def health_check() -> dict[str, str]:
return {"status": "healthy", "service": "news-service"}
@app.get("/api/enriched-news")
async def api_get_enriched_news(
ticker: str = Query(..., min_length=1),
start_date: str | None = Query(None),
end_date: str | None = Query(None),
limit: int = Query(100, ge=1, le=1000),
store: MarketStore = Depends(get_market_store),
) -> dict[str, Any]:
return news_domain.get_enriched_news(
store,
ticker=ticker,
start_date=start_date,
end_date=end_date,
limit=limit,
refresh_if_stale=False,
)
@app.get("/api/news-for-date")
async def api_get_news_for_date(
ticker: str = Query(..., min_length=1),
date: str = Query(...),
limit: int = Query(20, ge=1, le=100),
store: MarketStore = Depends(get_market_store),
) -> dict[str, Any]:
return news_domain.get_news_for_date(
store,
ticker=ticker,
date=date,
limit=limit,
refresh_if_stale=False,
)
@app.get("/api/news-timeline")
async def api_get_news_timeline(
ticker: str = Query(..., min_length=1),
start_date: str = Query(...),
end_date: str = Query(...),
store: MarketStore = Depends(get_market_store),
) -> dict[str, Any]:
return news_domain.get_news_timeline(
store,
ticker=ticker,
start_date=start_date,
end_date=end_date,
refresh_if_stale=False,
)
@app.get("/api/categories")
async def api_get_categories(
ticker: str = Query(..., min_length=1),
start_date: str | None = Query(None),
end_date: str | None = Query(None),
limit: int = Query(200, ge=1, le=1000),
store: MarketStore = Depends(get_market_store),
) -> dict[str, Any]:
return news_domain.get_news_categories(
store,
ticker=ticker,
start_date=start_date,
end_date=end_date,
limit=limit,
refresh_if_stale=False,
)
@app.get("/api/similar-days")
async def api_get_similar_days(
ticker: str = Query(..., min_length=1),
date: str = Query(...),
n_similar: int = Query(5, ge=1, le=20),
store: MarketStore = Depends(get_market_store),
) -> dict[str, Any]:
return news_domain.get_similar_days_payload(
store,
ticker=ticker,
date=date,
n_similar=n_similar,
refresh_if_stale=False,
)
@app.get("/api/stories/{ticker}")
async def api_get_story(
ticker: str,
as_of_date: str = Query(...),
store: MarketStore = Depends(get_market_store),
) -> dict[str, Any]:
return news_domain.get_story_payload(
store,
ticker=ticker,
as_of_date=as_of_date,
refresh_if_stale=False,
)
@app.get("/api/range-explain")
async def api_get_range_explain(
ticker: str = Query(..., min_length=1),
start_date: str = Query(...),
end_date: str = Query(...),
article_ids: list[str] = Query(default=[]),
limit: int = Query(100, ge=1, le=500),
store: MarketStore = Depends(get_market_store),
) -> dict[str, Any]:
return news_domain.get_range_explain_payload(
store,
ticker=ticker,
start_date=start_date,
end_date=end_date,
article_ids=article_ids,
limit=limit,
refresh_if_stale=False,
)
return app
app = create_app()
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8002)

View File

@@ -0,0 +1,49 @@
# -*- coding: utf-8 -*-
"""Read-only OpenClaw CLI FastAPI surface."""
from __future__ import annotations
from fastapi import Depends, FastAPI
from backend.api import openclaw_router
from backend.apps.cors import add_cors_middleware
from backend.api.openclaw import get_openclaw_cli_service
def create_app() -> FastAPI:
"""Create the OpenClaw service app."""
app = FastAPI(
title="大时代 OpenClaw Service",
description="Read-only OpenClaw CLI integration service surface",
version="0.1.0",
)
add_cors_middleware(app)
@app.get("/health")
async def health_check(
service=Depends(get_openclaw_cli_service),
) -> dict[str, object]:
return service.health()
@app.get("/api/status")
async def api_status(
service=Depends(get_openclaw_cli_service),
) -> dict[str, object]:
return {
"status": "operational",
"service": "openclaw-service",
"openclaw": service.health(),
}
app.include_router(openclaw_router)
return app
app = create_app()
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8004)

View File

@@ -0,0 +1,62 @@
# -*- coding: utf-8 -*-
"""Dedicated runtime service FastAPI surface."""
from __future__ import annotations
from fastapi import FastAPI
from backend.api import runtime_router
from backend.api.runtime import get_runtime_state
from backend.apps.cors import add_cors_middleware
def create_app() -> FastAPI:
"""Create the runtime service app."""
app = FastAPI(
title="大时代 Runtime Service",
description="Runtime lifecycle and gateway service surface extracted from the monolith",
version="0.1.0",
)
add_cors_middleware(app)
@app.get("/health")
async def health_check() -> dict[str, object]:
"""Health check for the runtime service."""
runtime_state = get_runtime_state()
process = runtime_state.gateway_process
is_running = process is not None and process.poll() is None
return {
"status": "healthy",
"service": "runtime-service",
"gateway_running": is_running,
"gateway_port": runtime_state.gateway_port,
}
@app.get("/api/status")
async def api_status() -> dict[str, object]:
"""Service-level status payload for runtime orchestration."""
runtime_state = get_runtime_state()
process = runtime_state.gateway_process
is_running = process is not None and process.poll() is None
return {
"status": "operational",
"service": "runtime-service",
"runtime": {
"gateway_running": is_running,
"gateway_port": runtime_state.gateway_port,
"has_runtime_manager": runtime_state.runtime_manager is not None,
},
}
app.include_router(runtime_router)
return app
app = create_app()
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8003)

View File

@@ -0,0 +1,136 @@
# -*- coding: utf-8 -*-
"""Trading data FastAPI surface."""
from __future__ import annotations
from typing import Any
from fastapi import FastAPI, Query
from backend.apps.cors import add_cors_middleware
from backend.domains import trading as trading_domain
from shared.schema import (
CompanyNewsResponse,
FinancialMetricsResponse,
InsiderTradeResponse,
LineItemResponse,
PriceResponse,
)
def create_app() -> FastAPI:
"""Create the trading data service app."""
app = FastAPI(
title="大时代 Trading Service",
description="Read-only trading data service surface extracted from the monolith",
version="0.1.0",
)
add_cors_middleware(app)
@app.get("/health")
async def health_check() -> dict[str, str]:
"""Health check endpoint."""
return {"status": "healthy", "service": "trading-service"}
@app.get("/api/prices", response_model=PriceResponse)
async def api_get_prices(
ticker: str = Query(..., min_length=1),
start_date: str = Query(...),
end_date: str = Query(...),
) -> PriceResponse:
payload = trading_domain.get_prices_payload(
ticker=ticker,
start_date=start_date,
end_date=end_date,
)
return PriceResponse(ticker=payload["ticker"], prices=payload["prices"])
@app.get("/api/financials", response_model=FinancialMetricsResponse)
async def api_get_financials(
ticker: str = Query(..., min_length=1),
end_date: str = Query(...),
period: str = Query("ttm"),
limit: int = Query(10, ge=1, le=100),
) -> FinancialMetricsResponse:
payload = trading_domain.get_financials_payload(
ticker=ticker,
end_date=end_date,
period=period,
limit=limit,
)
return FinancialMetricsResponse(financial_metrics=payload["financial_metrics"])
@app.get("/api/news", response_model=CompanyNewsResponse)
async def api_get_news(
ticker: str = Query(..., min_length=1),
end_date: str = Query(...),
start_date: str | None = Query(None),
limit: int = Query(1000, ge=1, le=5000),
) -> CompanyNewsResponse:
payload = trading_domain.get_news_payload(
ticker=ticker,
end_date=end_date,
start_date=start_date,
limit=limit,
)
return CompanyNewsResponse(news=payload["news"])
@app.get("/api/insider-trades", response_model=InsiderTradeResponse)
async def api_get_insider_trades(
ticker: str = Query(..., min_length=1),
end_date: str = Query(...),
start_date: str | None = Query(None),
limit: int = Query(1000, ge=1, le=5000),
) -> InsiderTradeResponse:
payload = trading_domain.get_insider_trades_payload(
ticker=ticker,
end_date=end_date,
start_date=start_date,
limit=limit,
)
return InsiderTradeResponse(insider_trades=payload["insider_trades"])
@app.get("/api/market/status")
async def api_get_market_status() -> dict[str, Any]:
"""Return current market status using the existing market service logic."""
return trading_domain.get_market_status_payload()
@app.get("/api/market-cap")
async def api_get_market_cap(
ticker: str = Query(..., min_length=1),
end_date: str = Query(...),
) -> dict[str, Any]:
"""Return market cap for one ticker/date."""
return trading_domain.get_market_cap_payload(
ticker=ticker,
end_date=end_date,
)
@app.get("/api/line-items", response_model=LineItemResponse)
async def api_get_line_items(
ticker: str = Query(..., min_length=1),
line_items: list[str] = Query(...),
end_date: str = Query(...),
period: str = Query("ttm"),
limit: int = Query(10, ge=1, le=100),
) -> LineItemResponse:
payload = trading_domain.get_line_items_payload(
ticker=ticker,
line_items=line_items,
end_date=end_date,
period=period,
limit=limit,
)
return LineItemResponse(search_results=payload["search_results"])
return app
app = create_app()
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8001)

View File

@@ -1,7 +1,7 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
EvoTraders CLI - Command-line interface for the EvoTraders trading system.
大时代 CLI - Command-line interface for the 大时代 trading system.
This module provides easy-to-use commands for running backtest, live trading,
and frontend development server.
@@ -29,7 +29,7 @@ from rich.table import Table
from dotenv import load_dotenv
from backend.agents.agent_workspace import load_agent_workspace_config
from backend.agents.prompt_loader import PromptLoader
from backend.agents.prompt_loader import get_prompt_loader
from backend.agents.skills_manager import SkillsManager
from backend.agents.team_pipeline_config import (
ensure_team_pipeline_config,
@@ -44,7 +44,7 @@ from backend.enrich.news_enricher import enrich_symbols
app = typer.Typer(
name="evotraders",
help="EvoTraders: A self-evolving multi-agent trading system",
help="大时代:自进化多智能体交易系统",
add_completion=False,
)
ingest_app = typer.Typer(help="Ingest Polygon market data into the research warehouse.")
@@ -55,7 +55,7 @@ team_app = typer.Typer(help="Inspect and manage run-scoped team pipeline config.
app.add_typer(team_app, name="team")
console = Console()
_prompt_loader = PromptLoader()
_prompt_loader = get_prompt_loader()
load_dotenv()
@@ -919,7 +919,7 @@ def backtest(
"""
console.print(
Panel.fit(
"[bold cyan]EvoTraders Backtest Mode[/bold cyan]",
"[bold cyan]大时代 Backtest Mode[/bold cyan]",
border_style="cyan",
),
)
@@ -1019,11 +1019,6 @@ def backtest(
@app.command()
def live(
mock: bool = typer.Option(
False,
"--mock",
help="Use mock mode with simulated prices (for testing)",
),
config_name: str = typer.Option(
"live",
"--config-name",
@@ -1078,7 +1073,6 @@ def live(
Example:
evotraders live # Run immediately (default)
evotraders live --mock # Mock mode
evotraders live -t 22:30 # Run at 22:30 local time daily
evotraders live --schedule-mode intraday --interval-minutes 60
evotraders live --trigger-time now # Run immediately
@@ -1086,33 +1080,31 @@ def live(
"""
schedule_mode = str(_normalize_typer_value(schedule_mode, "daily"))
interval_minutes = int(_normalize_typer_value(interval_minutes, 60))
mode_name = "MOCK" if mock else "LIVE"
console.print(
Panel.fit(
f"[bold cyan]EvoTraders {mode_name} Mode[/bold cyan]",
"[bold cyan]大时代 LIVE Mode[/bold cyan]",
border_style="cyan",
),
)
# Check for required API key in live mode
if not mock:
env_file = get_project_root() / ".env"
if not env_file.exists():
console.print("\n[yellow]Warning: .env file not found[/yellow]")
console.print("Creating from template...\n")
template = get_project_root() / "env.template"
if template.exists():
shutil.copy(template, env_file)
console.print("[green].env file created[/green]")
console.print(
"\n[red]Error: Please edit .env and set FINNHUB_API_KEY[/red]",
)
console.print(
"Get your free API key at: https://finnhub.io/register\n",
)
else:
console.print("[red]Error: env.template not found[/red]")
raise typer.Exit(1)
env_file = get_project_root() / ".env"
if not env_file.exists():
console.print("\n[yellow]Warning: .env file not found[/yellow]")
console.print("Creating from template...\n")
template = get_project_root() / "env.template"
if template.exists():
shutil.copy(template, env_file)
console.print("[green].env file created[/green]")
console.print(
"\n[red]Error: Please edit .env and set FINNHUB_API_KEY[/red]",
)
console.print(
"Get your free API key at: https://finnhub.io/register\n",
)
else:
console.print("[red]Error: env.template not found[/red]")
raise typer.Exit(1)
# Handle historical data cleanup
handle_history_cleanup(config_name, auto_clean=clean)
@@ -1168,12 +1160,9 @@ def live(
# Display configuration
console.print("\n[bold]Configuration:[/bold]")
if mock:
console.print(" Mode: [yellow]MOCK[/yellow] (Simulated prices)")
else:
console.print(
" Mode: [green]LIVE[/green] (Real-time prices via Finnhub)",
)
console.print(
" Mode: [green]LIVE[/green] (Real-time prices via Finnhub)",
)
console.print(f" Config: {config_name}")
console.print(f" Server: {host}:{port}")
console.print(f" Poll Interval: {poll_interval}s")
@@ -1188,22 +1177,17 @@ def live(
project_root = get_project_root()
os.chdir(project_root)
# Data update (if not mock mode)
if not mock:
run_data_updater(project_root)
auto_update_market_store(
config_name,
end_date=nyse_now.date().isoformat(),
)
auto_enrich_market_store(
config_name,
end_date=nyse_now.date().isoformat(),
force=False,
)
else:
console.print(
"\n[dim]Mock mode enabled - skipping data update[/dim]\n",
)
# Data update
run_data_updater(project_root)
auto_update_market_store(
config_name,
end_date=nyse_now.date().isoformat(),
)
auto_enrich_market_store(
config_name,
end_date=nyse_now.date().isoformat(),
force=False,
)
# Build command using backend.main
cmd = [
@@ -1229,8 +1213,6 @@ def live(
str(interval_minutes),
]
if mock:
cmd.append("--mock")
if enable_memory:
cmd.append("--enable-memory")
@@ -1269,7 +1251,7 @@ def frontend(
"""
console.print(
Panel.fit(
"[bold cyan]EvoTraders Frontend[/bold cyan]",
"[bold cyan]大时代 Frontend[/bold cyan]",
border_style="cyan",
),
)
@@ -1337,16 +1319,16 @@ def frontend(
@app.command()
def version():
"""Show the version of EvoTraders."""
"""Show the version of 大时代."""
console.print(
"\n[bold cyan]EvoTraders[/bold cyan] version [green]0.1.0[/green]\n",
"\n[bold cyan]大时代[/bold cyan] version [green]0.1.0[/green]\n",
)
@app.callback()
def main():
"""
EvoTraders: A self-evolving multi-agent trading system
大时代:自进化多智能体交易系统
Use 'evotraders --help' to see available commands.
"""

View File

@@ -4,6 +4,22 @@
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Dict
DEFAULT_TICKERS = [
"AAPL",
"MSFT",
"GOOGL",
"AMZN",
"NVDA",
"META",
"TSLA",
"AMD",
"NFLX",
"AVGO",
"PLTR",
"COIN",
]
import re
import yaml
@@ -127,7 +143,7 @@ def resolve_runtime_config(
bootstrap = get_bootstrap_config_for_run(project_root, config_name)
return {
"tickers": bootstrap.get("tickers")
or get_env_list("TICKERS", ["AAPL", "MSFT"]),
or get_env_list("TICKERS", DEFAULT_TICKERS),
"initial_cash": float(
bootstrap.get(
"initial_cash",

View File

@@ -76,27 +76,19 @@ def _resolve_config() -> DataSourceConfig:
"""
Resolve data source configuration based on available API keys.
Priority:
1. FINNHUB_API_KEY (if set)
2. FINANCIAL_DATASETS_API_KEY (if set)
3. Raises error if neither is available
The effective source should always match the first item in the resolved
ordered source list.
"""
sources = _ordered_sources()
if "finnhub" in sources:
return DataSourceConfig(
source="finnhub",
api_key=os.getenv("FINNHUB_API_KEY", "").strip(),
sources=sources,
)
if "financial_datasets" in sources:
return DataSourceConfig(
source="financial_datasets",
api_key=os.getenv("FINANCIAL_DATASETS_API_KEY", "").strip(),
sources=sources,
)
if "yfinance" in sources:
return DataSourceConfig(source="yfinance", api_key="", sources=sources)
return DataSourceConfig(source="local_csv", api_key="", sources=sources)
source = sources[0] if sources else "local_csv"
api_key = ""
if source == "finnhub":
api_key = os.getenv("FINNHUB_API_KEY", "").strip()
elif source == "financial_datasets":
api_key = os.getenv("FINANCIAL_DATASETS_API_KEY", "").strip()
return DataSourceConfig(source=source, api_key=api_key, sources=sources)
def get_config() -> DataSourceConfig:

View File

@@ -1,7 +1,35 @@
# -*- coding: utf-8 -*-
"""Core pipeline and orchestration logic"""
"""Core pipeline and orchestration logic.
Keep ``pipeline_runner`` behind lazy wrappers so importing ``backend.core`` does
not immediately pull in the gateway runtime graph.
"""
from .pipeline import TradingPipeline
from .state_sync import StateSync
__all__ = ["TradingPipeline", "StateSync"]
def create_agents(*args, **kwargs):
from .pipeline_runner import create_agents as _create_agents
return _create_agents(*args, **kwargs)
def create_long_term_memory(*args, **kwargs):
from .pipeline_runner import create_long_term_memory as _create_long_term_memory
return _create_long_term_memory(*args, **kwargs)
def stop_gateway(*args, **kwargs):
from .pipeline_runner import stop_gateway as _stop_gateway
return _stop_gateway(*args, **kwargs)
__all__ = [
"TradingPipeline",
"StateSync",
"create_agents",
"create_long_term_memory",
"stop_gateway",
]

View File

@@ -18,7 +18,6 @@ from agentscope.message import Msg
from agentscope.pipeline import MsgHub
from backend.utils.settlement import SettlementCoordinator
from backend.utils.terminal_dashboard import get_dashboard
from backend.core.state_sync import StateSync
from backend.utils.trade_executor import PortfolioTradeExecutor
from backend.runtime.manager import TradingRuntimeManager
@@ -30,7 +29,7 @@ from backend.agents.team_pipeline_config import (
from backend.agents import AnalystAgent
from backend.agents.toolkit_factory import create_agent_toolkit
from backend.agents.workspace_manager import WorkspaceManager
from backend.agents.prompt_loader import PromptLoader
from backend.agents.prompt_loader import get_prompt_loader
from backend.llm.models import get_agent_formatter, get_agent_model
from backend.config.constants import ANALYST_TYPES
@@ -48,13 +47,9 @@ except ImportError:
logger = logging.getLogger(__name__)
def _log(msg: str):
"""Log to dashboard if available, otherwise to logger"""
dashboard = get_dashboard()
if dashboard.live:
dashboard.log(msg)
else:
logger.info(msg)
def _log(msg: str) -> None:
"""Helper function for pipeline logging."""
logger.info(msg)
class TradingPipeline:
@@ -71,7 +66,7 @@ class TradingPipeline:
Real-time updates via StateSync after each agent completes.
Supports both legacy agent lists and new workspace-based agent loading.
Supports both legacy agent lists and run-scoped agent loading.
"""
def __init__(
@@ -1623,16 +1618,15 @@ class TradingPipeline:
config_name = getattr(self.pm, "config", {}).get("config_name", "default")
project_root = Path(__file__).resolve().parents[2]
personas = PromptLoader().load_yaml_config("analyst", "personas")
personas = get_prompt_loader().load_yaml_config("analyst", "personas")
persona = personas.get(analyst_type, {})
WorkspaceManager(project_root=project_root).ensure_agent_assets(
workspace_manager = WorkspaceManager(project_root=project_root)
workspace_manager.ensure_agent_assets(
config_name=config_name,
agent_id=agent_id,
role_seed=persona.get("description", "").strip(),
style_seed="\n".join(f"- {item}" for item in persona.get("focus", [])),
policy_seed=(
"State a clear signal, confidence, and the conditions "
"that would invalidate the thesis."
file_contents=workspace_manager.build_default_agent_files(
agent_id=agent_id,
persona=persona,
),
)

View File

@@ -17,7 +17,7 @@ from typing import Any, Dict, Optional, Callable
from backend.agents import AnalystAgent, PMAgent, RiskAgent
from backend.agents.skills_manager import SkillsManager
from backend.agents.toolkit_factory import create_agent_toolkit, load_agent_profiles
from backend.agents.prompt_loader import PromptLoader
from backend.agents.prompt_loader import get_prompt_loader
from backend.agents.workspace_manager import WorkspaceManager
from backend.config.constants import ANALYST_TYPES
from backend.core.pipeline import TradingPipeline
@@ -36,7 +36,7 @@ from backend.services.storage import StorageService
from backend.services.gateway import Gateway
from backend.utils.settlement import SettlementCoordinator
_prompt_loader = PromptLoader()
_prompt_loader = get_prompt_loader()
# Global gateway reference for cleanup
_gateway_instance: Optional[Gateway] = None
@@ -232,7 +232,7 @@ async def run_pipeline(
try:
# Extract config values
tickers = bootstrap.get("tickers", ["AAPL", "MSFT"])
tickers = bootstrap.get("tickers", ["AAPL", "MSFT", "GOOGL", "AMZN", "NVDA", "META", "TSLA", "AMD", "NFLX", "AVGO", "PLTR", "COIN"])
initial_cash = float(bootstrap.get("initial_cash", 100000.0))
margin_requirement = float(bootstrap.get("margin_requirement", 0.0))
max_comm_cycles = int(bootstrap.get("max_comm_cycles", 2))
@@ -244,10 +244,8 @@ async def run_pipeline(
start_date = bootstrap.get("start_date")
end_date = bootstrap.get("end_date")
enable_memory = bootstrap.get("enable_memory", False)
enable_mock = bootstrap.get("enable_mock", False)
is_backtest = mode == "backtest"
is_mock = enable_mock or mode == "mock" or (not is_backtest and os.getenv("MOCK_MODE", "false").lower() == "true")
# ======================================================================
# PHASE 0: Initialize runtime manager
@@ -266,10 +264,6 @@ async def run_pipeline(
set_global_runtime_manager(runtime_manager)
# Register runtime manager with API
from backend.api.runtime import register_runtime_manager
register_runtime_manager(runtime_manager)
# ======================================================================
# PHASE 1 & 2: Create infrastructure services (Market, Storage)
# These will be started by Gateway in the correct order
@@ -292,9 +286,8 @@ async def run_pipeline(
market_service = MarketService(
tickers=tickers,
poll_interval=10,
mock_mode=is_mock and not is_backtest,
backtest_mode=is_backtest,
api_key=os.getenv("FINNHUB_API_KEY") if not is_mock and not is_backtest else None,
api_key=os.getenv("FINNHUB_API_KEY") if not is_backtest else None,
backtest_start_date=start_date if is_backtest else None,
backtest_end_date=end_date if is_backtest else None,
)
@@ -391,7 +384,6 @@ async def run_pipeline(
scheduler_callback=scheduler_callback,
config={
"mode": mode,
"mock_mode": is_mock,
"backtest_mode": is_backtest,
"tickers": tickers,
"config_name": run_id,

View File

@@ -465,7 +465,6 @@ class StateSync:
payload = {
"server_mode": self._state.get("server_mode", "live"),
"is_mock_mode": self._state.get("is_mock_mode", False),
"is_backtest": self._state.get("is_backtest", False),
"tickers": self._state.get("tickers"),
"runtime_config": self._state.get("runtime_config"),
@@ -488,12 +487,13 @@ class StateSync:
}
if include_dashboard:
dashboard_snapshot = self.storage.build_dashboard_snapshot_from_state(self._state)
payload["dashboard"] = {
"summary": self.storage.load_file("summary"),
"holdings": self.storage.load_file("holdings"),
"stats": self.storage.load_file("stats"),
"trades": self.storage.load_file("trades"),
"leaderboard": self.storage.load_file("leaderboard"),
"summary": dashboard_snapshot.get("summary"),
"holdings": dashboard_snapshot.get("holdings"),
"stats": dashboard_snapshot.get("stats"),
"trades": dashboard_snapshot.get("trades"),
"leaderboard": dashboard_snapshot.get("leaderboard"),
}
return payload

View File

@@ -1,6 +1,5 @@
# -*- coding: utf-8 -*-
from backend.data.historical_price_manager import HistoricalPriceManager
from backend.data.mock_price_manager import MockPriceManager
from backend.data.polling_price_manager import PollingPriceManager
__all__ = ["MockPriceManager", "PollingPriceManager", "HistoricalPriceManager"]
__all__ = ["PollingPriceManager", "HistoricalPriceManager"]

View File

@@ -8,6 +8,7 @@ from typing import Iterable
from backend.data.market_store import MarketStore
from backend.data.news_alignment import align_news_for_symbol
from backend.data.provider_router import DataProviderRouter
from backend.data.polygon_client import (
fetch_news,
fetch_ohlc,
@@ -24,6 +25,65 @@ def _default_start(years: int = 2) -> str:
return (datetime.now(timezone.utc).date() - timedelta(days=years * 366)).isoformat()
def _max_news_date(news_rows: Iterable[dict]) -> str | None:
dates = [
str(item.get("published_utc") or "").strip()[:10]
for item in news_rows
if str(item.get("published_utc") or "").strip()
]
dates = [value for value in dates if value]
return max(dates) if dates else None
def _effective_last_news_fetch(
market_store: MarketStore,
*,
ticker: str,
end_date: str,
watermark_value: str | None,
) -> str | None:
"""Clamp stale/future watermarks to the latest actually stored news date."""
raw = str(watermark_value or "").strip()[:10]
if not raw:
return None
if raw <= end_date:
return raw
latest_stored = market_store.get_latest_news_date(ticker)
if latest_stored and latest_stored <= end_date:
return latest_stored
return end_date
def _normalize_provider_news_rows(ticker: str, news_items: Iterable[Any]) -> list[dict]:
rows: list[dict] = []
for item in news_items:
payload = item.model_dump() if hasattr(item, "model_dump") else dict(item or {})
related = payload.get("related")
if isinstance(related, str):
related_list = [value.strip().upper() for value in related.split(",") if value.strip()]
elif isinstance(related, list):
related_list = [str(value).strip().upper() for value in related if str(value).strip()]
else:
related_list = []
if ticker not in related_list:
related_list.append(ticker)
rows.append(
{
"title": payload.get("title"),
"description": payload.get("summary"),
"summary": payload.get("summary"),
"article_url": payload.get("url"),
"published_utc": payload.get("date"),
"publisher": payload.get("source"),
"tickers": related_list,
"category": payload.get("category"),
"raw_json": payload,
}
)
return rows
def ingest_ticker_history(
symbol: str,
*,
@@ -50,7 +110,11 @@ def ingest_ticker_history(
price_count = market_store.upsert_ohlc(ticker, ohlc_rows, source="polygon")
news_count = market_store.upsert_news(ticker, news_rows, source="polygon")
aligned_count = align_news_for_symbol(market_store, ticker)
market_store.update_fetch_watermark(symbol=ticker, price_date=end, news_date=end)
market_store.update_fetch_watermark(
symbol=ticker,
price_date=end,
news_date=_max_news_date(news_rows),
)
return {
"symbol": ticker,
@@ -78,9 +142,15 @@ def update_ticker_incremental(
if watermarks.get("last_price_fetch")
else _default_start()
)
effective_last_news_fetch = _effective_last_news_fetch(
market_store,
ticker=ticker,
end_date=end,
watermark_value=watermarks.get("last_news_fetch"),
)
start_news = (
(datetime.fromisoformat(watermarks["last_news_fetch"]) + timedelta(days=1)).date().isoformat()
if watermarks.get("last_news_fetch")
(datetime.fromisoformat(effective_last_news_fetch) + timedelta(days=1)).date().isoformat()
if effective_last_news_fetch
else _default_start()
)
@@ -100,7 +170,7 @@ def update_ticker_incremental(
market_store.update_fetch_watermark(
symbol=ticker,
price_date=end if ohlc_rows or watermarks.get("last_price_fetch") else None,
news_date=end if news_rows or watermarks.get("last_news_fetch") else None,
news_date=_max_news_date(news_rows),
)
return {
@@ -114,6 +184,86 @@ def update_ticker_incremental(
}
def refresh_news_incremental(
symbol: str,
*,
end_date: str | None = None,
store: MarketStore | None = None,
) -> dict:
"""Incrementally fetch company news using the configured provider router."""
ticker = normalize_symbol(symbol)
market_store = store or MarketStore()
watermarks = market_store.get_ticker_watermarks(ticker)
end = end_date or _today_utc()
effective_last_news_fetch = _effective_last_news_fetch(
market_store,
ticker=ticker,
end_date=end,
watermark_value=watermarks.get("last_news_fetch"),
)
start_news = (
(datetime.fromisoformat(effective_last_news_fetch) + timedelta(days=1)).date().isoformat()
if effective_last_news_fetch
else _default_start()
)
if start_news > end:
return {
"symbol": ticker,
"start_news_date": start_news,
"end_date": end,
"news": 0,
"aligned": 0,
}
router = DataProviderRouter()
news_items, source = router.get_company_news(
ticker=ticker,
start_date=start_news,
end_date=end,
limit=1000,
)
news_rows = _normalize_provider_news_rows(ticker, news_items)
news_count = market_store.upsert_news(ticker, news_rows, source=source) if news_rows else 0
aligned_count = align_news_for_symbol(market_store, ticker)
market_store.update_fetch_watermark(
symbol=ticker,
news_date=_max_news_date(news_rows),
)
return {
"symbol": ticker,
"start_news_date": start_news,
"end_date": end,
"news": news_count,
"aligned": aligned_count,
"source": source,
}
def refresh_news_for_symbols(
symbols: Iterable[str],
*,
end_date: str | None = None,
store: MarketStore | None = None,
) -> list[dict]:
"""Incrementally refresh company news for a list of tickers."""
market_store = store or MarketStore()
results = []
for symbol in symbols:
ticker = normalize_symbol(symbol)
if not ticker:
continue
results.append(
refresh_news_incremental(
ticker,
end_date=end_date,
store=market_store,
)
)
return results
def ingest_symbols(
symbols: Iterable[str],
*,

View File

@@ -9,7 +9,7 @@ import os
import sqlite3
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Iterable
from typing import Any, Iterable, Optional
SCHEMA = """
@@ -147,12 +147,30 @@ def _utc_timestamp() -> str:
class MarketStore:
"""SQLite-backed market research warehouse."""
"""SQLite-backed market research warehouse. Use get_instance() for the singleton."""
_instance: Optional["MarketStore"] = None
def __new__(cls, db_path: Path | None = None) -> "MarketStore":
if cls._instance is not None:
if db_path is None or cls._instance.db_path == Path(db_path or get_market_db_path()):
return cls._instance
instance = super().__new__(cls)
cls._instance = instance
return instance
def __init__(self, db_path: Path | None = None):
if getattr(self, "_initialized", False):
return
self.db_path = Path(db_path or get_market_db_path())
self.db_path.parent.mkdir(parents=True, exist_ok=True)
self._init_db()
self._initialized = True
@classmethod
def get_instance(cls, db_path: Path | None = None) -> "MarketStore":
"""Get the MarketStore singleton instance."""
return cls(db_path)
def _connect(self) -> sqlite3.Connection:
conn = sqlite3.connect(self.db_path)
@@ -226,6 +244,20 @@ class MarketStore:
"last_news_fetch": None,
}
def get_latest_news_date(self, symbol: str) -> str | None:
"""Return the latest stored published news date for one ticker."""
with self._connect() as conn:
row = conn.execute(
"""
SELECT MAX(substr(nr.published_utc, 1, 10)) AS latest_date
FROM news_ticker nt
JOIN news_raw nr ON nr.id = nt.news_id
WHERE nt.symbol = ?
""",
(symbol,),
).fetchone()
return str(row["latest_date"]).strip() if row and row["latest_date"] else None
def upsert_ohlc(self, symbol: str, rows: Iterable[dict[str, Any]], *, source: str = "polygon") -> int:
timestamp = _utc_timestamp()
count = 0

View File

@@ -1,244 +0,0 @@
# -*- coding: utf-8 -*-
"""
Mock Price Manager - For testing during non-trading hours
Generates virtual real-time price data
"""
import logging
import os
import random
import threading
import time
from typing import Callable, Dict, List, Optional
from backend.data.provider_utils import normalize_symbol
logger = logging.getLogger(__name__)
class MockPriceManager:
"""Mock Price Manager - Generates virtual prices for testing"""
def __init__(self, poll_interval: int = 10, volatility: float = 0.5):
"""
Args:
poll_interval: Price update interval in seconds
volatility: Price volatility percentage
"""
if poll_interval is None:
poll_interval = int(os.getenv("MOCK_POLL_INTERVAL", "5"))
if volatility is None:
volatility = float(os.getenv("MOCK_VOLATILITY", "0.5"))
self.poll_interval = poll_interval
self.volatility = volatility
self.subscribed_symbols: List[str] = []
self.base_prices: Dict[str, float] = {}
self.open_prices: Dict[str, float] = {}
self.latest_prices: Dict[str, float] = {}
self.price_callbacks: List[Callable] = []
self.running = False
self._thread: Optional[threading.Thread] = None
self.default_base_prices = {
"AAPL": 237.50,
"MSFT": 425.30,
"GOOGL": 161.50,
"AMZN": 218.45,
"NVDA": 950.00,
"META": 573.22,
"TSLA": 342.15,
"AMD": 168.90,
"NFLX": 688.25,
"INTC": 42.18,
"COIN": 285.50,
"PLTR": 45.80,
"BABA": 88.30,
"DIS": 112.50,
"BKNG": 4850.00,
}
logger.info(
f"MockPriceManager initialized (interval: {self.poll_interval}s, "
f"volatility: {self.volatility}%)",
)
def subscribe(
self,
symbols: List[str],
base_prices: Dict[str, float] = None,
):
"""Subscribe to stock symbols"""
for symbol in symbols:
symbol = normalize_symbol(symbol)
if symbol not in self.subscribed_symbols:
self.subscribed_symbols.append(symbol)
if base_prices and symbol in base_prices:
base_price = base_prices[symbol]
elif symbol in self.default_base_prices:
base_price = self.default_base_prices[symbol]
else:
base_price = random.uniform(50, 500)
self.base_prices[symbol] = base_price
self.open_prices[symbol] = base_price
self.latest_prices[symbol] = base_price
logger.info(
f"Subscribed to mock price: {symbol} (base: ${base_price:.2f})", # noqa: E501
)
def unsubscribe(self, symbols: List[str]):
"""Unsubscribe from symbols"""
for symbol in symbols:
symbol = normalize_symbol(symbol)
if symbol in self.subscribed_symbols:
self.subscribed_symbols.remove(symbol)
self.base_prices.pop(symbol, None)
self.open_prices.pop(symbol, None)
self.latest_prices.pop(symbol, None)
logger.info(f"Unsubscribed: {symbol}")
def add_price_callback(self, callback: Callable):
"""Add price update callback"""
self.price_callbacks.append(callback)
def _generate_price_update(self, symbol: str) -> float:
"""Generate price update based on random walk"""
current_price = self.latest_prices.get(
symbol,
self.base_prices[symbol],
)
change_percent = random.uniform(-self.volatility, self.volatility)
new_price = current_price * (1 + change_percent / 100)
# 10% chance of larger movement
if random.random() < 0.1:
trend_factor = random.uniform(-2, 2)
new_price = new_price * (1 + trend_factor / 100)
# Limit intraday movement to +/-10%
open_price = self.open_prices[symbol]
max_price = open_price * 1.10
min_price = open_price * 0.90
new_price = max(min_price, min(max_price, new_price))
return new_price
def _update_prices(self):
"""Update prices for all subscribed stocks"""
timestamp = int(time.time() * 1000)
for symbol in self.subscribed_symbols:
try:
new_price = self._generate_price_update(symbol)
self.latest_prices[symbol] = new_price
open_price = self.open_prices[symbol]
ret = ((new_price - open_price) / open_price) * 100
price_data = {
"symbol": symbol,
"price": new_price,
"timestamp": timestamp,
"volume": random.randint(1000000, 10000000),
"open": open_price,
"high": max(new_price, open_price),
"low": min(new_price, open_price),
"previous_close": open_price,
"ret": ret,
}
for callback in self.price_callbacks:
try:
callback(price_data)
except Exception as e:
logger.error(
f"Mock price callback error ({symbol}): {e}",
)
logger.debug(
f"Mock {symbol}: ${new_price:.2f} [ret: {ret:+.2f}%]",
)
except Exception as e:
logger.error(f"Failed to generate mock price ({symbol}): {e}")
def _polling_loop(self):
"""Main polling loop"""
logger.info(
f"Mock price generation started (interval: {self.poll_interval}s)",
)
while self.running:
try:
start_time = time.time()
self._update_prices()
elapsed = time.time() - start_time
sleep_time = max(0, self.poll_interval - elapsed)
if sleep_time > 0:
time.sleep(sleep_time)
except Exception as e:
logger.error(f"Mock polling loop error: {e}")
time.sleep(5)
def start(self):
"""Start mock price generation"""
if self.running:
logger.warning("Mock price manager already running")
return
if not self.subscribed_symbols:
logger.warning("No stocks subscribed")
return
self.running = True
self._thread = threading.Thread(target=self._polling_loop, daemon=True)
self._thread.start()
logger.info(
f"Mock price manager started: {', '.join(self.subscribed_symbols)}", # noqa: E501
)
def stop(self):
"""Stop mock price generation"""
self.running = False
if self._thread:
self._thread.join(timeout=5)
logger.info("Mock price manager stopped")
def get_latest_price(self, symbol: str) -> Optional[float]:
"""Get latest price for symbol"""
return self.latest_prices.get(symbol)
def get_all_latest_prices(self) -> Dict[str, float]:
"""Get all latest prices"""
return self.latest_prices.copy()
def get_open_price(self, symbol: str) -> Optional[float]:
"""Get open price for symbol"""
return self.open_prices.get(symbol)
def reset_open_prices(self):
"""Reset open prices for new trading day"""
for symbol in self.subscribed_symbols:
last_close = self.latest_prices[symbol]
gap_percent = random.uniform(-1, 1)
new_open = last_close * (1 + gap_percent / 100)
self.open_prices[symbol] = new_open
self.latest_prices[symbol] = new_open
logger.info("Open prices reset")
def set_base_price(self, symbol: str, price: float):
"""Manually set base price for testing"""
if symbol in self.subscribed_symbols:
self.base_prices[symbol] = price
self.open_prices[symbol] = price
self.latest_prices[symbol] = price
logger.info(f"{symbol} base price set to: ${price:.2f}")
else:
logger.warning(f"{symbol} not subscribed")

View File

@@ -15,6 +15,9 @@ from backend.data.provider_utils import normalize_symbol
logger = logging.getLogger(__name__)
_SUPPRESSED_LOG_EVERY = 20
class PollingPriceManager:
"""Polling-based price manager using Finnhub or yfinance."""
@@ -43,6 +46,7 @@ class PollingPriceManager:
self.latest_prices: Dict[str, float] = {}
self.open_prices: Dict[str, float] = {}
self.price_callbacks: List[Callable] = []
self._failure_counts: Dict[str, int] = {}
self.running = False
self._thread: Optional[threading.Thread] = None
@@ -77,6 +81,8 @@ class PollingPriceManager:
for symbol in self.subscribed_symbols:
try:
quote_data = self._fetch_quote(symbol)
if not isinstance(quote_data, dict):
raise ValueError(f"{symbol}: Empty quote payload")
current_price = quote_data.get("c")
open_price = quote_data.get("o")
@@ -103,6 +109,13 @@ class PollingPriceManager:
)
self.latest_prices[symbol] = current_price
previous_failures = self._failure_counts.pop(symbol, 0)
if previous_failures > 0:
logger.info(
"%s quote polling recovered after %d consecutive failures",
symbol,
previous_failures,
)
price_data = {
"symbol": symbol,
@@ -128,7 +141,20 @@ class PollingPriceManager:
)
except Exception as e:
logger.error(f"Failed to fetch {symbol} price: {e}")
failure_count = self._failure_counts.get(symbol, 0) + 1
self._failure_counts[symbol] = failure_count
message = f"Failed to fetch {symbol} price: {e}"
if failure_count == 1:
logger.warning(message)
elif failure_count % _SUPPRESSED_LOG_EVERY == 0:
logger.warning(
"%s (repeated %d times; suppressing intermediate failures)",
message,
failure_count,
)
else:
logger.debug(message)
def _fetch_quote(self, symbol: str) -> Dict[str, float]:
"""Fetch a normalized quote payload from the configured provider."""
@@ -136,7 +162,10 @@ class PollingPriceManager:
return self._fetch_yfinance_quote(symbol)
if not self.finnhub_client:
raise ValueError("Finnhub API key required for finnhub polling")
return self.finnhub_client.quote(symbol)
quote = self.finnhub_client.quote(symbol)
if not isinstance(quote, dict):
raise ValueError(f"{symbol}: Invalid Finnhub quote payload")
return quote
def _fetch_yfinance_quote(self, symbol: str) -> Dict[str, float]:
"""Fetch quote data from yfinance and normalize to Finnhub-like keys."""
@@ -162,6 +191,8 @@ class PollingPriceManager:
if current_price is None:
history = ticker.history(period="1d", interval="1m", auto_adjust=False)
if history is None:
raise ValueError(f"{symbol}: yfinance returned no history frame")
if history.empty:
raise ValueError(f"{symbol}: No yfinance quote data")
latest = history.iloc[-1]

View File

@@ -11,7 +11,7 @@ import pandas as pd
import yfinance as yf
from backend.config.data_config import DataSource, get_data_sources
from backend.data.schema import (
from shared.schema import (
CompanyFactsResponse,
CompanyNews,
CompanyNewsResponse,

View File

@@ -1,194 +1,50 @@
# -*- coding: utf-8 -*-
from pydantic import BaseModel
"""Compatibility schema bridge.
This module preserves the legacy ``backend.data.schema`` import path while
delegating the actual schema definitions to ``shared.schema``. Keeping one
canonical DTO set avoids drift as the monolith is split into service-specific
packages.
"""
class Price(BaseModel):
open: float
close: float
high: float
low: float
volume: int
time: str
from shared.schema import (
AgentStateData,
AgentStateMetadata,
AnalystSignal,
CompanyFacts,
CompanyFactsResponse,
CompanyNews,
CompanyNewsResponse,
FinancialMetrics,
FinancialMetricsResponse,
InsiderTrade,
InsiderTradeResponse,
LineItem,
LineItemResponse,
Portfolio,
Position,
Price,
PriceResponse,
TickerAnalysis,
)
class PriceResponse(BaseModel):
ticker: str
prices: list[Price]
class FinancialMetrics(BaseModel):
ticker: str
report_period: str
period: str
currency: str
market_cap: float | None
enterprise_value: float | None
price_to_earnings_ratio: float | None
price_to_book_ratio: float | None
price_to_sales_ratio: float | None
enterprise_value_to_ebitda_ratio: float | None
enterprise_value_to_revenue_ratio: float | None
free_cash_flow_yield: float | None
peg_ratio: float | None
gross_margin: float | None
operating_margin: float | None
net_margin: float | None
return_on_equity: float | None
return_on_assets: float | None
return_on_invested_capital: float | None
asset_turnover: float | None
inventory_turnover: float | None
receivables_turnover: float | None
days_sales_outstanding: float | None
operating_cycle: float | None
working_capital_turnover: float | None
current_ratio: float | None
quick_ratio: float | None
cash_ratio: float | None
operating_cash_flow_ratio: float | None
debt_to_equity: float | None
debt_to_assets: float | None
interest_coverage: float | None
revenue_growth: float | None
earnings_growth: float | None
book_value_growth: float | None
earnings_per_share_growth: float | None
free_cash_flow_growth: float | None
operating_income_growth: float | None
ebitda_growth: float | None
payout_ratio: float | None
earnings_per_share: float | None
book_value_per_share: float | None
free_cash_flow_per_share: float | None
class FinancialMetricsResponse(BaseModel):
financial_metrics: list[FinancialMetrics]
class LineItem(BaseModel):
ticker: str
report_period: str
period: str
currency: str
# Allow additional fields dynamically
model_config = {"extra": "allow"}
class LineItemResponse(BaseModel):
search_results: list[LineItem]
class InsiderTrade(BaseModel):
ticker: str
issuer: str | None
name: str | None
title: str | None
is_board_director: bool | None
transaction_date: str | None
transaction_shares: float | None
transaction_price_per_share: float | None
transaction_value: float | None
shares_owned_before_transaction: float | None
shares_owned_after_transaction: float | None
security_title: str | None
filing_date: str
class InsiderTradeResponse(BaseModel):
insider_trades: list[InsiderTrade]
class CompanyNews(BaseModel):
category: str | None = None
ticker: str
title: str
related: str | None = None
source: str
date: str | None = None
url: str
summary: str | None = None
class CompanyNewsResponse(BaseModel):
news: list[CompanyNews]
class CompanyFacts(BaseModel):
ticker: str
name: str
cik: str | None = None
industry: str | None = None
sector: str | None = None
category: str | None = None
exchange: str | None = None
is_active: bool | None = None
listing_date: str | None = None
location: str | None = None
market_cap: float | None = None
number_of_employees: int | None = None
sec_filings_url: str | None = None
sic_code: str | None = None
sic_industry: str | None = None
sic_sector: str | None = None
website_url: str | None = None
weighted_average_shares: int | None = None
class CompanyFactsResponse(BaseModel):
company_facts: CompanyFacts
class Position(BaseModel):
"""Position information - for Portfolio mode"""
long: int = 0 # Long position quantity (shares)
short: int = 0 # Short position quantity (shares)
long_cost_basis: float = 0.0 # Long position average cost
short_cost_basis: float = 0.0 # Short position average cost
class Portfolio(BaseModel):
"""Portfolio - for Portfolio mode"""
cash: float = 100000.0 # Available cash
positions: dict[str, Position] = {} # ticker -> Position mapping
# Margin requirement (0.0 means shorting disabled, 0.5 means 50% margin)
margin_requirement: float = 0.0
margin_used: float = 0.0 # Margin used
class AnalystSignal(BaseModel):
signal: str | None = None
confidence: float | None = None
reasoning: dict | str | None = None
# Extended fields for richer signal information
reasons: list[str] | None = None # Core drivers/reasons for the signal
risks: list[str] | None = None # Key risk factors
invalidation: str | None = None # Conditions that would invalidate the thesis
next_action: str | None = None # Suggested next action for PM
# Valuation-related fields
intrinsic_value: float | None = None # DCF intrinsic value
fair_value_range: dict | None = None # {bear, base, bull} fair value range
value_gap_pct: float | None = None # Value gap percentage
valuation_methods: list[str] | None = None # List of valuation methods used
max_position_size: float | None = None # For risk management signals
class TickerAnalysis(BaseModel):
ticker: str
analyst_signals: dict[str, AnalystSignal] # agent_name -> signal mapping
class AgentStateData(BaseModel):
tickers: list[str]
portfolio: Portfolio
start_date: str
end_date: str
ticker_analyses: dict[str, TickerAnalysis] # ticker -> analysis mapping
class AgentStateMetadata(BaseModel):
show_reasoning: bool = False
model_config = {"extra": "allow"}
__all__ = [
"Price",
"PriceResponse",
"FinancialMetrics",
"FinancialMetricsResponse",
"LineItem",
"LineItemResponse",
"InsiderTrade",
"InsiderTradeResponse",
"CompanyNews",
"CompanyNewsResponse",
"CompanyFacts",
"CompanyFactsResponse",
"Position",
"Portfolio",
"AnalystSignal",
"TickerAnalysis",
"AgentStateData",
"AgentStateMetadata",
]

View File

@@ -0,0 +1,2 @@
# -*- coding: utf-8 -*-
"""Domain modules for split service internals."""

320
backend/domains/news.py Normal file
View File

@@ -0,0 +1,320 @@
# -*- coding: utf-8 -*-
"""News/explain domain helpers shared by app surfaces and gateway fallbacks."""
from __future__ import annotations
from typing import Any
from backend.data.market_store import MarketStore
from backend.data.market_ingest import update_ticker_incremental
from backend.enrich.news_enricher import enrich_news_for_symbol
from backend.explain.range_explainer import build_range_explanation
from backend.explain.similarity_service import find_similar_days
from backend.explain.story_service import get_or_create_stock_story
def news_rows_need_enrichment(rows: list[dict[str, Any]]) -> bool:
"""Return whether news rows are missing explain-oriented analysis fields."""
if not rows:
return True
return all(
not row.get("sentiment")
and not row.get("relevance")
and not row.get("key_discussion")
for row in rows
)
def ensure_news_fresh(
store: MarketStore,
*,
ticker: str,
target_date: str | None = None,
refresh_if_stale: bool = True,
) -> dict[str, Any]:
"""Refresh raw news incrementally when stored watermarks are stale."""
normalized_target = str(target_date or "").strip()[:10]
if not normalized_target:
return {
"ticker": ticker,
"target_date": None,
"last_news_fetch": None,
"refreshed": False,
}
watermarks = store.get_ticker_watermarks(ticker)
last_news_fetch = str(watermarks.get("last_news_fetch") or "").strip()[:10]
refreshed = False
if refresh_if_stale and (not last_news_fetch or last_news_fetch < normalized_target):
update_ticker_incremental(
ticker,
end_date=normalized_target,
store=store,
)
refreshed = True
watermarks = store.get_ticker_watermarks(ticker)
last_news_fetch = str(watermarks.get("last_news_fetch") or "").strip()[:10]
return {
"ticker": ticker,
"target_date": normalized_target,
"last_news_fetch": last_news_fetch or None,
"refreshed": refreshed,
}
def get_enriched_news(
store: MarketStore,
*,
ticker: str,
start_date: str | None = None,
end_date: str | None = None,
limit: int = 100,
refresh_if_stale: bool = False,
) -> dict[str, Any]:
freshness = ensure_news_fresh(
store,
ticker=ticker,
target_date=end_date,
refresh_if_stale=refresh_if_stale,
)
rows = store.get_news_items_enriched(
ticker,
start_date=start_date,
end_date=end_date,
limit=limit,
)
if news_rows_need_enrichment(rows):
enrich_news_for_symbol(
store,
ticker,
start_date=start_date,
end_date=end_date,
limit=limit,
)
rows = store.get_news_items_enriched(
ticker,
start_date=start_date,
end_date=end_date,
limit=limit,
)
return {"ticker": ticker, "news": rows, "freshness": freshness}
def get_news_for_date(
store: MarketStore,
*,
ticker: str,
date: str,
limit: int = 20,
refresh_if_stale: bool = False,
) -> dict[str, Any]:
freshness = ensure_news_fresh(
store,
ticker=ticker,
target_date=date,
refresh_if_stale=refresh_if_stale,
)
rows = store.get_news_items_enriched(
ticker,
trade_date=date,
limit=limit,
)
if news_rows_need_enrichment(rows):
enrich_news_for_symbol(
store,
ticker,
start_date=date,
end_date=date,
limit=limit,
)
rows = store.get_news_items_enriched(
ticker,
trade_date=date,
limit=limit,
)
return {"ticker": ticker, "date": date, "news": rows, "freshness": freshness}
def get_news_timeline(
store: MarketStore,
*,
ticker: str,
start_date: str,
end_date: str,
refresh_if_stale: bool = False,
) -> dict[str, Any]:
freshness = ensure_news_fresh(
store,
ticker=ticker,
target_date=end_date,
refresh_if_stale=refresh_if_stale,
)
timeline = store.get_news_timeline_enriched(
ticker,
start_date=start_date,
end_date=end_date,
)
if not timeline:
enrich_news_for_symbol(
store,
ticker,
start_date=start_date,
end_date=end_date,
limit=200,
)
timeline = store.get_news_timeline_enriched(
ticker,
start_date=start_date,
end_date=end_date,
)
return {
"ticker": ticker,
"timeline": timeline,
"start_date": start_date,
"end_date": end_date,
"freshness": freshness,
}
def get_news_categories(
store: MarketStore,
*,
ticker: str,
start_date: str | None = None,
end_date: str | None = None,
limit: int = 200,
refresh_if_stale: bool = False,
) -> dict[str, Any]:
freshness = ensure_news_fresh(
store,
ticker=ticker,
target_date=end_date,
refresh_if_stale=refresh_if_stale,
)
rows = store.get_news_items_enriched(
ticker,
start_date=start_date,
end_date=end_date,
limit=limit,
)
if news_rows_need_enrichment(rows):
enrich_news_for_symbol(
store,
ticker,
start_date=start_date,
end_date=end_date,
limit=limit,
)
categories = store.get_news_categories_enriched(
ticker,
start_date=start_date,
end_date=end_date,
limit=limit,
)
return {"ticker": ticker, "categories": categories, "freshness": freshness}
def get_similar_days_payload(
store: MarketStore,
*,
ticker: str,
date: str,
n_similar: int = 5,
refresh_if_stale: bool = False,
) -> dict[str, Any]:
freshness = ensure_news_fresh(
store,
ticker=ticker,
target_date=date,
refresh_if_stale=refresh_if_stale,
)
result = find_similar_days(
store,
symbol=ticker,
target_date=date,
top_k=n_similar,
)
result["freshness"] = freshness
return result
def get_story_payload(
store: MarketStore,
*,
ticker: str,
as_of_date: str,
refresh_if_stale: bool = False,
) -> dict[str, Any]:
freshness = ensure_news_fresh(
store,
ticker=ticker,
target_date=as_of_date,
refresh_if_stale=refresh_if_stale,
)
enrich_news_for_symbol(
store,
ticker,
end_date=as_of_date,
limit=80,
)
result = get_or_create_stock_story(
store,
symbol=ticker,
as_of_date=as_of_date,
)
result["freshness"] = freshness
return result
def get_range_explain_payload(
store: MarketStore,
*,
ticker: str,
start_date: str,
end_date: str,
article_ids: list[str] | None = None,
limit: int = 100,
refresh_if_stale: bool = False,
) -> dict[str, Any]:
freshness = ensure_news_fresh(
store,
ticker=ticker,
target_date=end_date,
refresh_if_stale=refresh_if_stale,
)
news_rows = []
if article_ids:
news_rows = store.get_news_by_ids_enriched(ticker, article_ids)
if not news_rows:
news_rows = store.get_news_items_enriched(
ticker,
start_date=start_date,
end_date=end_date,
limit=limit,
)
if news_rows_need_enrichment(news_rows):
enrich_news_for_symbol(
store,
ticker,
start_date=start_date,
end_date=end_date,
limit=limit,
)
news_rows = (
store.get_news_by_ids_enriched(ticker, article_ids)
if article_ids
else store.get_news_items_enriched(
ticker,
start_date=start_date,
end_date=end_date,
limit=limit,
)
)
result = build_range_explanation(
ticker=ticker,
start_date=start_date,
end_date=end_date,
news_rows=news_rows,
)
return {"ticker": ticker, "result": result, "freshness": freshness}

106
backend/domains/trading.py Normal file
View File

@@ -0,0 +1,106 @@
# -*- coding: utf-8 -*-
"""Trading domain helpers shared by app surfaces and gateway fallbacks."""
from __future__ import annotations
from typing import Any
from backend.services.market import MarketService
from backend.tools.data_tools import (
get_company_news,
get_financial_metrics,
get_insider_trades,
get_market_cap,
get_prices,
search_line_items,
)
def get_prices_payload(*, ticker: str, start_date: str, end_date: str) -> dict[str, Any]:
return {
"ticker": ticker,
"prices": get_prices(ticker, start_date, end_date),
}
def get_financials_payload(
*,
ticker: str,
end_date: str,
period: str = "ttm",
limit: int = 10,
) -> dict[str, Any]:
return {
"financial_metrics": get_financial_metrics(
ticker=ticker,
end_date=end_date,
period=period,
limit=limit,
)
}
def get_news_payload(
*,
ticker: str,
end_date: str,
start_date: str | None = None,
limit: int = 1000,
) -> dict[str, Any]:
return {
"news": get_company_news(
ticker=ticker,
end_date=end_date,
start_date=start_date,
limit=limit,
)
}
def get_insider_trades_payload(
*,
ticker: str,
end_date: str,
start_date: str | None = None,
limit: int = 1000,
) -> dict[str, Any]:
return {
"insider_trades": get_insider_trades(
ticker=ticker,
end_date=end_date,
start_date=start_date,
limit=limit,
)
}
def get_market_status_payload() -> dict[str, Any]:
market_service = MarketService(tickers=[])
return market_service.get_market_status()
def get_market_cap_payload(*, ticker: str, end_date: str) -> dict[str, Any]:
return {
"ticker": ticker,
"end_date": end_date,
"market_cap": get_market_cap(ticker, end_date),
}
def get_line_items_payload(
*,
ticker: str,
line_items: list[str],
end_date: str,
period: str = "ttm",
limit: int = 10,
) -> dict[str, Any]:
return {
"search_results": search_line_items(
ticker=ticker,
line_items=line_items,
end_date=end_date,
period=period,
limit=limit,
)
}

309
backend/gateway_server.py Normal file
View File

@@ -0,0 +1,309 @@
# -*- coding: utf-8 -*-
"""Gateway Server - Entry point for Gateway subprocess.
This module is launched as a subprocess by the Control Plane (FastAPI)
to run the Data Plane (Gateway + Pipeline).
"""
import argparse
import asyncio
import json
import logging
import os
import sys
from contextlib import AsyncExitStack
from pathlib import Path
from dotenv import load_dotenv
# Load environment variables
load_dotenv()
from backend.agents import AnalystAgent, PMAgent, RiskAgent
from backend.agents.skills_manager import SkillsManager
from backend.agents.toolkit_factory import create_agent_toolkit, load_agent_profiles
from backend.agents.prompt_loader import get_prompt_loader
from backend.agents.workspace_manager import WorkspaceManager
from backend.config.constants import ANALYST_TYPES
from backend.core.pipeline import TradingPipeline
from backend.core.pipeline_runner import create_agents, create_long_term_memory
from backend.core.scheduler import BacktestScheduler, Scheduler
from backend.llm.models import get_agent_formatter, get_agent_model
from backend.runtime.manager import (
TradingRuntimeManager,
set_global_runtime_manager,
clear_global_runtime_manager,
)
from backend.services.gateway import Gateway
from backend.services.market import MarketService
from backend.services.storage import StorageService
from backend.utils.settlement import SettlementCoordinator
logger = logging.getLogger(__name__)
_prompt_loader = get_prompt_loader()
INFO_LOGGER_PREFIXES = (
"backend.agents",
"backend.core.pipeline",
"backend.core.scheduler",
"backend.services.gateway_cycle_support",
)
NOISY_LOGGER_LEVELS = {
"aiohttp": logging.WARNING,
"asyncio": logging.WARNING,
"dashscope": logging.WARNING,
"finnhub": logging.WARNING,
"httpcore": logging.WARNING,
"httpx": logging.WARNING,
"urllib3": logging.WARNING,
"websockets": logging.WARNING,
"yfinance": logging.WARNING,
"backend.data.polling_price_manager": logging.WARNING,
"backend.services.gateway": logging.WARNING,
"backend.services.market": logging.WARNING,
"backend.services.storage": logging.WARNING,
}
class SuppressNoisyInfoFilter(logging.Filter):
"""Filter out low-signal library INFO logs while keeping warnings/errors."""
def filter(self, record: logging.LogRecord) -> bool:
message = record.getMessage()
if record.name == "httpx" and message.startswith("HTTP Request:"):
return False
if record.name.startswith("websockets") and "connection open" in message:
return False
if record.name.startswith("websockets") and "opening handshake failed" in message:
return False
if record.levelno >= logging.WARNING:
return True
return True
def configure_gateway_logging(verbose: bool = False) -> None:
"""Configure gateway logging with low-noise defaults for runtime logs."""
root_level = logging.DEBUG if verbose else logging.WARNING
logging.basicConfig(
level=root_level,
format="%(asctime)s | %(levelname)-7s | %(name)s:%(lineno)d - %(message)s",
force=True,
)
if not verbose:
suppress_filter = SuppressNoisyInfoFilter()
for handler in logging.getLogger().handlers:
handler.addFilter(suppress_filter)
for logger_name, level in NOISY_LOGGER_LEVELS.items():
logging.getLogger(logger_name).setLevel(logging.DEBUG if verbose else level)
if not verbose:
for prefix in INFO_LOGGER_PREFIXES:
logging.getLogger(prefix).setLevel(logging.INFO)
logging.getLogger(__name__).setLevel(logging.INFO if not verbose else logging.DEBUG)
async def run_gateway(
run_id: str,
run_dir: Path,
bootstrap: dict,
port: int
):
"""Run Gateway with Pipeline."""
# Extract config
tickers = bootstrap.get("tickers", ["AAPL", "MSFT", "GOOGL", "AMZN", "NVDA", "META", "TSLA", "AMD", "NFLX", "AVGO", "PLTR", "COIN"])
initial_cash = float(bootstrap.get("initial_cash", 100000.0))
margin_requirement = float(bootstrap.get("margin_requirement", 0.0))
max_comm_cycles = int(bootstrap.get("max_comm_cycles", 2))
schedule_mode = bootstrap.get("schedule_mode", "daily")
trigger_time = bootstrap.get("trigger_time", "09:30")
interval_minutes = int(bootstrap.get("interval_minutes", 60))
heartbeat_interval = int(bootstrap.get("heartbeat_interval", 0)) # 0 = disabled
mode = bootstrap.get("mode", "live")
start_date = bootstrap.get("start_date")
end_date = bootstrap.get("end_date")
enable_memory = bootstrap.get("enable_memory", False)
poll_interval = int(bootstrap.get("poll_interval", 10))
is_backtest = mode == "backtest"
logger.info(f"[Gateway Server] Starting run {run_id} on port {port}")
# Create runtime manager
runtime_manager = TradingRuntimeManager(
config_name=run_id,
run_dir=run_dir,
bootstrap=bootstrap,
)
runtime_manager.prepare_run()
set_global_runtime_manager(runtime_manager)
try:
async with AsyncExitStack() as stack:
# Create services
market_service = MarketService(
tickers=tickers,
poll_interval=poll_interval,
backtest_mode=is_backtest,
api_key=os.getenv("FINNHUB_API_KEY") if not is_backtest else None,
backtest_start_date=start_date if is_backtest else None,
backtest_end_date=end_date if is_backtest else None,
)
storage_service = StorageService(
dashboard_dir=run_dir / "team_dashboard",
initial_cash=initial_cash,
config_name=run_id,
)
if not storage_service.files["summary"].exists():
storage_service.initialize_empty_dashboard()
else:
storage_service.update_leaderboard_model_info()
# Create agents
analysts, risk_manager, pm, long_term_memories = create_agents(
run_id=run_id,
run_dir=run_dir,
initial_cash=initial_cash,
margin_requirement=margin_requirement,
enable_long_term_memory=enable_memory,
)
# Register agents
for agent in analysts + [risk_manager, pm]:
agent_id = getattr(agent, "agent_id", None) or getattr(agent, "name", None)
if agent_id:
runtime_manager.register_agent(agent_id)
# Load portfolio state
portfolio_state = storage_service.load_portfolio_state()
pm.load_portfolio_state(portfolio_state)
# Create settlement coordinator
settlement_coordinator = SettlementCoordinator(
storage=storage_service,
initial_capital=initial_cash,
)
# Create pipeline
pipeline = TradingPipeline(
analysts=analysts,
risk_manager=risk_manager,
portfolio_manager=pm,
settlement_coordinator=settlement_coordinator,
max_comm_cycles=max_comm_cycles,
runtime_manager=runtime_manager,
)
# Create scheduler
scheduler_callback = None
live_scheduler = None
if is_backtest:
backtest_scheduler = BacktestScheduler(
start_date=start_date,
end_date=end_date,
trading_calendar="NYSE",
delay_between_days=0.5,
)
async def scheduler_callback_fn(callback):
await backtest_scheduler.start(callback)
scheduler_callback = scheduler_callback_fn
else:
live_scheduler = Scheduler(
mode=schedule_mode,
trigger_time=trigger_time,
interval_minutes=interval_minutes,
heartbeat_interval=heartbeat_interval if heartbeat_interval > 0 else None,
config={"config_name": run_id},
)
async def scheduler_callback_fn(callback):
await live_scheduler.start(callback)
scheduler_callback = scheduler_callback_fn
# Enter long-term memory contexts
for memory in long_term_memories:
await stack.enter_async_context(memory)
# Create Gateway
gateway = Gateway(
market_service=market_service,
storage_service=storage_service,
pipeline=pipeline,
scheduler_callback=scheduler_callback,
config={
"mode": mode,
"backtest_mode": is_backtest,
"tickers": tickers,
"config_name": run_id,
"schedule_mode": schedule_mode,
"interval_minutes": interval_minutes,
"trigger_time": trigger_time,
"heartbeat_interval": heartbeat_interval,
"initial_cash": initial_cash,
"margin_requirement": margin_requirement,
"max_comm_cycles": max_comm_cycles,
"enable_memory": enable_memory,
},
scheduler=live_scheduler,
)
# Start Gateway (blocks until shutdown)
logger.info(f"[Gateway Server] Gateway starting on port {port}")
await gateway.start(host="0.0.0.0", port=port)
except asyncio.CancelledError:
logger.info("[Gateway Server] Cancelled")
raise
finally:
logger.info("[Gateway Server] Cleaning up")
clear_global_runtime_manager()
def main():
"""Main entry point."""
parser = argparse.ArgumentParser(description="Gateway Server")
parser.add_argument("--run-id", required=True, help="Run identifier")
parser.add_argument("--run-dir", required=True, help="Run directory path")
parser.add_argument("--port", type=int, default=8765, help="WebSocket port")
parser.add_argument("--bootstrap", required=True, help="Bootstrap config as JSON")
parser.add_argument("--verbose", action="store_true", help="Verbose logging")
args = parser.parse_args()
# Setup logging
configure_gateway_logging(verbose=args.verbose)
# Parse bootstrap
bootstrap = json.loads(args.bootstrap)
run_dir = Path(args.run_dir)
# Run
try:
asyncio.run(run_gateway(
run_id=args.run_id,
run_dir=run_dir,
bootstrap=bootstrap,
port=args.port
))
except KeyboardInterrupt:
logger.info("[Gateway Server] Interrupted by user")
except Exception as e:
logger.exception(f"[Gateway Server] Fatal error: {e}")
sys.exit(1)
if __name__ == "__main__":
main()

View File

@@ -3,6 +3,8 @@
AgentScope Native Model Factory
Uses native AgentScope model classes for LLM calls
"""
import asyncio
import inspect
import os
import time
import logging
@@ -34,6 +36,27 @@ logger = logging.getLogger(__name__)
T = TypeVar("T")
def _usage_value(usage: Any, key: str, default: Any = 0) -> Any:
"""Read usage fields from both object-style and dict-style usage payloads."""
if usage is None:
return default
if isinstance(usage, dict):
return usage.get(key, default)
try:
return getattr(usage, key)
except (AttributeError, KeyError):
return default
def _usage_total_tokens(usage: Any) -> int:
total = _usage_value(usage, "total_tokens", None)
if total is not None:
return int(total or 0)
input_tokens = _usage_value(usage, "input_tokens", 0)
output_tokens = _usage_value(usage, "output_tokens", 0)
return int((input_tokens or 0) + (output_tokens or 0))
class RetryChatModel:
"""Wraps an AgentScope model with automatic retry for transient errors.
@@ -55,6 +78,7 @@ class RetryChatModel:
"502",
"504",
"connection",
"disconnected",
"temporary",
"overloaded",
"too_many_requests",
@@ -150,8 +174,8 @@ class RetryChatModel:
# Track usage if available
if hasattr(result, "usage") and result.usage:
usage = result.usage
self._total_tokens_used += getattr(usage, "total_tokens", 0)
self._total_cost += getattr(usage, "cost", 0.0)
self._total_tokens_used += _usage_total_tokens(usage)
self._total_cost += float(_usage_value(usage, "cost", 0.0) or 0.0)
return result
@@ -192,9 +216,66 @@ class RetryChatModel:
raise last_error
raise RuntimeError("RetryChatModel: Unexpected state, no error but no result")
async def _call_with_retry_async(self, func: Callable[..., T], *args, **kwargs) -> T:
"""Call an async function with retry logic for transient errors."""
last_error: Optional[Exception] = None
for attempt in range(1, self._max_retries + 1):
try:
result = await func(*args, **kwargs)
if hasattr(result, "usage") and result.usage:
usage = result.usage
self._total_tokens_used += _usage_total_tokens(usage)
self._total_cost += float(_usage_value(usage, "cost", 0.0) or 0.0)
return result
except Exception as e:
last_error = e
if attempt >= self._max_retries:
logger.error(
"RetryChatModel: Max retries (%d) exhausted for %s",
self._max_retries,
self.model_name,
)
break
if not self._is_transient_error(e):
logger.warning(
"RetryChatModel: Non-transient error, not retrying: %s",
str(e),
)
break
delay = self._calculate_delay(attempt)
logger.warning(
"RetryChatModel: Transient async error on attempt %d/%d, "
"retrying in %.1fs: %s",
attempt,
self._max_retries,
delay,
str(e)[:200],
)
if self._on_retry:
self._on_retry(attempt, e, delay)
await asyncio.sleep(delay)
if last_error is not None:
raise last_error
raise RuntimeError("RetryChatModel: Unexpected async state, no error but no result")
def __call__(self, *args, **kwargs) -> Any:
"""Forward calls to the wrapped model with retry logic."""
return self._call_with_retry(self._model, *args, **kwargs)
model_call = getattr(self._model, "__call__", None)
if inspect.iscoroutinefunction(self._model) or inspect.iscoroutinefunction(model_call):
return self._call_with_retry_async(self._model, *args, **kwargs)
result = self._model(*args, **kwargs)
return result
def __getattr__(self, name: str) -> Any:
"""Proxy attribute access to the wrapped model."""
@@ -248,10 +329,18 @@ class TokenRecordingModelWrapper:
if usage is None:
return
self._prompt_tokens += getattr(usage, "prompt_tokens", 0)
self._completion_tokens += getattr(usage, "completion_tokens", 0)
self._total_tokens += getattr(usage, "total_tokens", 0)
self._total_cost += getattr(usage, "cost", 0.0)
prompt_tokens = _usage_value(usage, "prompt_tokens", None)
completion_tokens = _usage_value(usage, "completion_tokens", None)
if prompt_tokens is None:
prompt_tokens = _usage_value(usage, "input_tokens", 0)
if completion_tokens is None:
completion_tokens = _usage_value(usage, "output_tokens", 0)
self._prompt_tokens += int(prompt_tokens or 0)
self._completion_tokens += int(completion_tokens or 0)
self._total_tokens += _usage_total_tokens(usage)
self._total_cost += float(_usage_value(usage, "cost", 0.0) or 0.0)
def __call__(self, *args, **kwargs) -> Any:
"""Forward calls and record usage."""
@@ -401,7 +490,8 @@ def create_model(
if host:
model_kwargs["host"] = host
return model_class(**model_kwargs)
model = model_class(**model_kwargs)
return RetryChatModel(model)
def get_agent_model(agent_id: str, stream: bool = False):

View File

@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
"""
Main Entry Point
Supports: backtest, live, mock modes
Supports: backtest, live modes
"""
import argparse
import asyncio
@@ -16,7 +16,7 @@ from dotenv import load_dotenv
from backend.agents import AnalystAgent, PMAgent, RiskAgent
from backend.agents.skills_manager import SkillsManager
from backend.agents.toolkit_factory import create_agent_toolkit, load_agent_profiles
from backend.agents.prompt_loader import PromptLoader
from backend.agents.prompt_loader import get_prompt_loader
from backend.agents.workspace_manager import WorkspaceManager
from backend.config.bootstrap_config import resolve_runtime_config
from backend.config.constants import ANALYST_TYPES
@@ -29,6 +29,7 @@ from backend.runtime.manager import (
set_global_runtime_manager,
clear_global_runtime_manager,
)
from backend.gateway_server import configure_gateway_logging
from backend.services.gateway import Gateway
from backend.services.market import MarketService
from backend.services.storage import StorageService
@@ -38,7 +39,8 @@ load_dotenv()
logger = logging.getLogger(__name__)
loguru.logger.disable("flowllm")
loguru.logger.disable("reme_ai")
_prompt_loader = PromptLoader()
configure_gateway_logging(verbose=os.getenv("LOG_LEVEL", "").upper() == "DEBUG")
_prompt_loader = get_prompt_loader()
def _get_run_dir(config_name: str) -> Path:
@@ -226,17 +228,13 @@ async def run_with_gateway(args):
)
runtime_manager.prepare_run()
set_global_runtime_manager(runtime_manager)
register_runtime_manager(runtime_manager)
# Create market service
market_service = MarketService(
tickers=tickers,
poll_interval=args.poll_interval,
mock_mode=args.mock and not is_backtest,
backtest_mode=is_backtest,
api_key=os.getenv("FINNHUB_API_KEY")
if not args.mock and not is_backtest
else None,
api_key=os.getenv("FINNHUB_API_KEY") if not is_backtest else None,
backtest_start_date=args.start_date if is_backtest else None,
backtest_end_date=args.end_date if is_backtest else None,
)
@@ -321,7 +319,6 @@ async def run_with_gateway(args):
scheduler_callback=scheduler_callback,
config={
"mode": args.mode,
"mock_mode": args.mock,
"backtest_mode": is_backtest,
"tickers": tickers,
"config_name": config_name,
@@ -354,8 +351,7 @@ def main():
"""Main entry point"""
parser = argparse.ArgumentParser(description="Trading System")
parser.add_argument("--mode", choices=["live", "backtest"], default="live")
parser.add_argument("--mock", action="store_true")
parser.add_argument("--config-name", default="mock")
parser.add_argument("--config-name", default="live")
parser.add_argument("--host", default="0.0.0.0")
parser.add_argument("--port", type=int, default=8765)
parser.add_argument(

View File

@@ -13,15 +13,30 @@ from .registry import RuntimeRegistry
_global_runtime_manager: Optional["TradingRuntimeManager"] = None
_shutdown_event: Optional[asyncio.Event] = None
# Lazy import to avoid circular dependency
_api_runtime = None
def _get_api_runtime():
global _api_runtime
if _api_runtime is None:
from backend.api import runtime as api_runtime_module
_api_runtime = api_runtime_module
return _api_runtime
def set_global_runtime_manager(manager: "TradingRuntimeManager") -> None:
global _global_runtime_manager
_global_runtime_manager = manager
# Sync to RuntimeState for consistency
_get_api_runtime().register_runtime_manager(manager)
def clear_global_runtime_manager() -> None:
global _global_runtime_manager
_global_runtime_manager = None
# Sync to RuntimeState for consistency
_get_api_runtime().unregister_runtime_manager()
def get_global_runtime_manager() -> Optional["TradingRuntimeManager"]:

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,426 @@
# -*- coding: utf-8 -*-
"""Runtime/workspace/skills handlers extracted from the main Gateway module.
Deprecated note:
Agent/workspace/skill read-write operations are being migrated to
agent_service REST endpoints. These websocket handlers remain as a
compatibility fallback and should not be considered the primary control
plane path for frontend reads/writes.
"""
from __future__ import annotations
import json
from datetime import datetime
from typing import Any
from backend.agents.agent_workspace import load_agent_workspace_config
from backend.agents.skills_manager import SkillsManager
from backend.agents.toolkit_factory import load_agent_profiles
from backend.config.bootstrap_config import (
get_bootstrap_config_for_run,
resolve_runtime_config,
update_bootstrap_values_for_run,
)
from backend.data.market_ingest import ingest_symbols
from backend.llm.models import get_agent_model_info
async def handle_reload_runtime_assets(gateway: Any) -> None:
config_name = gateway.config.get("config_name", "default")
runtime_config = resolve_runtime_config(
project_root=gateway._project_root,
config_name=config_name,
enable_memory=gateway.config.get("enable_memory", False),
schedule_mode=gateway.config.get("schedule_mode", "daily"),
interval_minutes=gateway.config.get("interval_minutes", 60),
trigger_time=gateway.config.get("trigger_time", "09:30"),
)
result = gateway.pipeline.reload_runtime_assets(runtime_config=runtime_config)
runtime_updates = gateway._apply_runtime_config(runtime_config)
await gateway.state_sync.on_system_message("Runtime assets reloaded.")
await gateway.broadcast({"type": "runtime_assets_reloaded", **result, **runtime_updates})
async def handle_update_runtime_config(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
updates: dict[str, Any] = {}
schedule_mode = str(data.get("schedule_mode", "")).strip().lower()
if schedule_mode:
if schedule_mode not in {"daily", "intraday"}:
await websocket.send(json.dumps({"type": "error", "message": "schedule_mode must be 'daily' or 'intraday'."}, ensure_ascii=False))
return
updates["schedule_mode"] = schedule_mode
interval_minutes = data.get("interval_minutes")
if interval_minutes is not None:
try:
parsed_interval = int(interval_minutes)
except (TypeError, ValueError):
parsed_interval = 0
if parsed_interval <= 0:
await websocket.send(json.dumps({"type": "error", "message": "interval_minutes must be a positive integer."}, ensure_ascii=False))
return
updates["interval_minutes"] = parsed_interval
trigger_time = data.get("trigger_time")
if trigger_time is not None:
raw_trigger = str(trigger_time).strip()
if raw_trigger and raw_trigger != "now":
try:
datetime.strptime(raw_trigger, "%H:%M")
except ValueError:
await websocket.send(json.dumps({"type": "error", "message": "trigger_time must use HH:MM or 'now'."}, ensure_ascii=False))
return
updates["trigger_time"] = raw_trigger or "09:30"
max_comm_cycles = data.get("max_comm_cycles")
if max_comm_cycles is not None:
try:
parsed_cycles = int(max_comm_cycles)
except (TypeError, ValueError):
parsed_cycles = 0
if parsed_cycles <= 0:
await websocket.send(json.dumps({"type": "error", "message": "max_comm_cycles must be a positive integer."}, ensure_ascii=False))
return
updates["max_comm_cycles"] = parsed_cycles
initial_cash = data.get("initial_cash")
if initial_cash is not None:
try:
parsed_initial_cash = float(initial_cash)
except (TypeError, ValueError):
parsed_initial_cash = 0.0
if parsed_initial_cash <= 0:
await websocket.send(json.dumps({"type": "error", "message": "initial_cash must be a positive number."}, ensure_ascii=False))
return
updates["initial_cash"] = parsed_initial_cash
margin_requirement = data.get("margin_requirement")
if margin_requirement is not None:
try:
parsed_margin_requirement = float(margin_requirement)
except (TypeError, ValueError):
parsed_margin_requirement = -1.0
if parsed_margin_requirement < 0:
await websocket.send(json.dumps({"type": "error", "message": "margin_requirement must be a non-negative number."}, ensure_ascii=False))
return
updates["margin_requirement"] = parsed_margin_requirement
enable_memory = data.get("enable_memory")
if enable_memory is not None:
updates["enable_memory"] = bool(enable_memory)
if not updates:
await websocket.send(json.dumps({"type": "error", "message": "No runtime settings were provided."}, ensure_ascii=False))
return
config_name = gateway.config.get("config_name", "default")
update_bootstrap_values_for_run(
project_root=gateway._project_root,
config_name=config_name,
updates=updates,
)
await gateway.state_sync.on_system_message("运行时调度配置已保存,正在热更新")
await handle_reload_runtime_assets(gateway)
async def handle_update_watchlist(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
tickers = gateway._normalize_watchlist(data.get("tickers"))
if not tickers:
await websocket.send(json.dumps({"type": "error", "message": "update_watchlist requires at least one valid ticker."}, ensure_ascii=False))
return
config_name = gateway.config.get("config_name", "default")
update_bootstrap_values_for_run(
project_root=gateway._project_root,
config_name=config_name,
updates={"tickers": tickers},
)
await gateway.state_sync.on_system_message(f"Watchlist updated: {', '.join(tickers)}")
await gateway.broadcast({"type": "watchlist_updated", "config_name": config_name, "tickers": tickers})
await handle_reload_runtime_assets(gateway)
gateway._schedule_watchlist_market_store_refresh(tickers)
async def handle_get_agent_skills(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
agent_id = str(data.get("agent_id", "")).strip()
if not agent_id:
await websocket.send(json.dumps({"type": "error", "message": "get_agent_skills requires agent_id."}, ensure_ascii=False))
return
config_name = gateway.config.get("config_name", "default")
skills_manager = SkillsManager(project_root=gateway._project_root)
agent_asset_dir = skills_manager.get_agent_asset_dir(config_name, agent_id)
agent_config = load_agent_workspace_config(agent_asset_dir / "agent.yaml")
resolved_skills = set(skills_manager.resolve_agent_skill_names(config_name=config_name, agent_id=agent_id, default_skills=[]))
enabled = set(agent_config.enabled_skills)
disabled = set(agent_config.disabled_skills)
payload = []
for item in skills_manager.list_agent_skill_catalog(config_name, agent_id):
if item.skill_name in disabled:
status = "disabled"
elif item.skill_name in enabled:
status = "enabled"
elif item.skill_name in resolved_skills:
status = "active"
else:
status = "available"
payload.append({
"skill_name": item.skill_name,
"name": item.name,
"description": item.description,
"version": item.version,
"source": item.source,
"tools": item.tools,
"status": status,
})
await websocket.send(json.dumps({
"type": "agent_skills_loaded",
"config_name": config_name,
"agent_id": agent_id,
"skills": payload,
}, ensure_ascii=False))
async def handle_get_agent_profile(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
agent_id = str(data.get("agent_id", "")).strip()
if not agent_id:
await websocket.send(json.dumps({"type": "error", "message": "get_agent_profile requires agent_id."}, ensure_ascii=False))
return
config_name = gateway.config.get("config_name", "default")
skills_manager = SkillsManager(project_root=gateway._project_root)
asset_dir = skills_manager.get_agent_asset_dir(config_name, agent_id)
agent_config = load_agent_workspace_config(asset_dir / "agent.yaml")
profiles = load_agent_profiles()
profile = profiles.get(agent_id, {})
bootstrap = get_bootstrap_config_for_run(gateway._project_root, config_name)
override = bootstrap.agent_override(agent_id)
active_tool_groups = override.get("active_tool_groups", agent_config.active_tool_groups or profile.get("active_tool_groups", []))
if not isinstance(active_tool_groups, list):
active_tool_groups = []
disabled_tool_groups = agent_config.disabled_tool_groups
if disabled_tool_groups:
disabled_set = set(disabled_tool_groups)
active_tool_groups = [group_name for group_name in active_tool_groups if group_name not in disabled_set]
default_skills = profile.get("skills", [])
if not isinstance(default_skills, list):
default_skills = []
resolved_skills = skills_manager.resolve_agent_skill_names(
config_name=config_name,
agent_id=agent_id,
default_skills=default_skills,
)
prompt_files = agent_config.prompt_files or ["SOUL.md", "PROFILE.md", "AGENTS.md", "POLICY.md", "MEMORY.md"]
model_name, model_provider = get_agent_model_info(agent_id)
await websocket.send(json.dumps({
"type": "agent_profile_loaded",
"config_name": config_name,
"agent_id": agent_id,
"profile": {
"model_name": model_name,
"model_provider": model_provider,
"prompt_files": prompt_files,
"default_skills": default_skills,
"resolved_skills": resolved_skills,
"active_tool_groups": active_tool_groups,
"disabled_tool_groups": disabled_tool_groups,
"enabled_skills": agent_config.enabled_skills,
"disabled_skills": agent_config.disabled_skills,
},
}, ensure_ascii=False))
async def handle_get_skill_detail(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
agent_id = str(data.get("agent_id", "")).strip()
skill_name = str(data.get("skill_name", "")).strip()
if not skill_name:
await websocket.send(json.dumps({"type": "error", "message": "get_skill_detail requires skill_name."}, ensure_ascii=False))
return
skills_manager = SkillsManager(project_root=gateway._project_root)
try:
if agent_id:
config_name = gateway.config.get("config_name", "default")
detail = skills_manager.load_agent_skill_document(config_name=config_name, agent_id=agent_id, skill_name=skill_name)
else:
detail = skills_manager.load_skill_document(skill_name)
except FileNotFoundError:
await websocket.send(json.dumps({"type": "error", "message": f"Unknown skill: {skill_name}"}, ensure_ascii=False))
return
await websocket.send(json.dumps({
"type": "skill_detail_loaded",
"agent_id": agent_id,
"skill": detail,
}, ensure_ascii=False))
async def handle_create_agent_local_skill(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
agent_id = str(data.get("agent_id", "")).strip()
skill_name = str(data.get("skill_name", "")).strip()
if not agent_id or not skill_name:
await websocket.send(json.dumps({"type": "error", "message": "create_agent_local_skill requires agent_id and skill_name."}, ensure_ascii=False))
return
config_name = gateway.config.get("config_name", "default")
skills_manager = SkillsManager(project_root=gateway._project_root)
try:
skills_manager.create_agent_local_skill(config_name=config_name, agent_id=agent_id, skill_name=skill_name)
except (ValueError, FileExistsError) as exc:
await websocket.send(json.dumps({"type": "error", "message": str(exc)}, ensure_ascii=False))
return
await gateway.state_sync.on_system_message(f"Created local skill {skill_name} for {agent_id}")
await gateway._handle_reload_runtime_assets()
await websocket.send(json.dumps({"type": "agent_local_skill_created", "agent_id": agent_id, "skill_name": skill_name}, ensure_ascii=False))
await handle_get_agent_skills(gateway, websocket, {"agent_id": agent_id})
await handle_get_skill_detail(gateway, websocket, {"agent_id": agent_id, "skill_name": skill_name})
async def handle_update_agent_local_skill(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
agent_id = str(data.get("agent_id", "")).strip()
skill_name = str(data.get("skill_name", "")).strip()
content = data.get("content")
if not agent_id or not skill_name or not isinstance(content, str):
await websocket.send(json.dumps({"type": "error", "message": "update_agent_local_skill requires agent_id, skill_name, and string content."}, ensure_ascii=False))
return
config_name = gateway.config.get("config_name", "default")
skills_manager = SkillsManager(project_root=gateway._project_root)
try:
skills_manager.update_agent_local_skill(config_name=config_name, agent_id=agent_id, skill_name=skill_name, content=content)
except (ValueError, FileNotFoundError) as exc:
await websocket.send(json.dumps({"type": "error", "message": str(exc)}, ensure_ascii=False))
return
await gateway.state_sync.on_system_message(f"Updated local skill {skill_name} for {agent_id}")
await gateway._handle_reload_runtime_assets()
await websocket.send(json.dumps({"type": "agent_local_skill_updated", "agent_id": agent_id, "skill_name": skill_name}, ensure_ascii=False))
await handle_get_skill_detail(gateway, websocket, {"agent_id": agent_id, "skill_name": skill_name})
async def handle_delete_agent_local_skill(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
agent_id = str(data.get("agent_id", "")).strip()
skill_name = str(data.get("skill_name", "")).strip()
if not agent_id or not skill_name:
await websocket.send(json.dumps({"type": "error", "message": "delete_agent_local_skill requires agent_id and skill_name."}, ensure_ascii=False))
return
config_name = gateway.config.get("config_name", "default")
skills_manager = SkillsManager(project_root=gateway._project_root)
try:
skills_manager.delete_agent_local_skill(config_name=config_name, agent_id=agent_id, skill_name=skill_name)
skills_manager.forget_agent_skill_overrides(config_name=config_name, agent_id=agent_id, skill_names=[skill_name])
except (ValueError, FileNotFoundError) as exc:
await websocket.send(json.dumps({"type": "error", "message": str(exc)}, ensure_ascii=False))
return
await gateway.state_sync.on_system_message(f"Deleted local skill {skill_name} for {agent_id}")
await gateway._handle_reload_runtime_assets()
await websocket.send(json.dumps({"type": "agent_local_skill_deleted", "agent_id": agent_id, "skill_name": skill_name}, ensure_ascii=False))
await handle_get_agent_skills(gateway, websocket, {"agent_id": agent_id})
async def handle_remove_agent_skill(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
agent_id = str(data.get("agent_id", "")).strip()
skill_name = str(data.get("skill_name", "")).strip()
if not agent_id or not skill_name:
await websocket.send(json.dumps({"type": "error", "message": "remove_agent_skill requires agent_id and skill_name."}, ensure_ascii=False))
return
config_name = gateway.config.get("config_name", "default")
skills_manager = SkillsManager(project_root=gateway._project_root)
skill_names = {
item.skill_name
for item in skills_manager.list_agent_skill_catalog(config_name, agent_id)
if item.source != "local"
}
if skill_name not in skill_names:
await websocket.send(json.dumps({"type": "error", "message": f"Unknown shared skill: {skill_name}"}, ensure_ascii=False))
return
skills_manager.update_agent_skill_overrides(config_name=config_name, agent_id=agent_id, disable=[skill_name])
await gateway.state_sync.on_system_message(f"Removed shared skill {skill_name} from {agent_id}")
await gateway._handle_reload_runtime_assets()
await websocket.send(json.dumps({"type": "agent_skill_removed", "agent_id": agent_id, "skill_name": skill_name}, ensure_ascii=False))
await handle_get_agent_skills(gateway, websocket, {"agent_id": agent_id})
async def handle_update_agent_skill(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
agent_id = str(data.get("agent_id", "")).strip()
skill_name = str(data.get("skill_name", "")).strip()
enabled = data.get("enabled")
if not agent_id or not skill_name or not isinstance(enabled, bool):
await websocket.send(json.dumps({"type": "error", "message": "update_agent_skill requires agent_id, skill_name, and boolean enabled."}, ensure_ascii=False))
return
config_name = gateway.config.get("config_name", "default")
skills_manager = SkillsManager(project_root=gateway._project_root)
skill_names = {item.skill_name for item in skills_manager.list_agent_skill_catalog(config_name, agent_id)}
if skill_name not in skill_names:
await websocket.send(json.dumps({"type": "error", "message": f"Unknown skill: {skill_name}"}, ensure_ascii=False))
return
if enabled:
skills_manager.update_agent_skill_overrides(config_name=config_name, agent_id=agent_id, enable=[skill_name])
await gateway.state_sync.on_system_message(f"Enabled skill {skill_name} for {agent_id}")
else:
skills_manager.update_agent_skill_overrides(config_name=config_name, agent_id=agent_id, disable=[skill_name])
await gateway.state_sync.on_system_message(f"Disabled skill {skill_name} for {agent_id}")
await websocket.send(json.dumps({
"type": "agent_skill_updated",
"agent_id": agent_id,
"skill_name": skill_name,
"enabled": enabled,
}, ensure_ascii=False))
await gateway._handle_reload_runtime_assets()
await handle_get_agent_skills(gateway, websocket, {"agent_id": agent_id})
async def handle_get_agent_workspace_file(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
agent_id = str(data.get("agent_id", "")).strip()
filename = gateway._normalize_agent_workspace_filename(data.get("filename"))
if not agent_id or not filename:
await websocket.send(json.dumps({"type": "error", "message": "get_agent_workspace_file requires agent_id and supported filename."}, ensure_ascii=False))
return
config_name = gateway.config.get("config_name", "default")
skills_manager = SkillsManager(project_root=gateway._project_root)
asset_dir = skills_manager.get_agent_asset_dir(config_name, agent_id)
asset_dir.mkdir(parents=True, exist_ok=True)
path = asset_dir / filename
content = path.read_text(encoding="utf-8") if path.exists() else ""
await websocket.send(json.dumps({
"type": "agent_workspace_file_loaded",
"config_name": config_name,
"agent_id": agent_id,
"filename": filename,
"content": content,
}, ensure_ascii=False))
async def handle_update_agent_workspace_file(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
agent_id = str(data.get("agent_id", "")).strip()
filename = gateway._normalize_agent_workspace_filename(data.get("filename"))
content = data.get("content")
if not agent_id or not filename or not isinstance(content, str):
await websocket.send(json.dumps({"type": "error", "message": "update_agent_workspace_file requires agent_id, supported filename, and string content."}, ensure_ascii=False))
return
config_name = gateway.config.get("config_name", "default")
skills_manager = SkillsManager(project_root=gateway._project_root)
asset_dir = skills_manager.get_agent_asset_dir(config_name, agent_id)
asset_dir.mkdir(parents=True, exist_ok=True)
path = asset_dir / filename
path.write_text(content, encoding="utf-8")
await gateway.state_sync.on_system_message(f"Updated {filename} for {agent_id}")
await websocket.send(json.dumps({"type": "agent_workspace_file_updated", "agent_id": agent_id, "filename": filename}, ensure_ascii=False))
await gateway._handle_reload_runtime_assets()
await handle_get_agent_workspace_file(gateway, websocket, {"agent_id": agent_id, "filename": filename})

View File

@@ -0,0 +1,372 @@
# -*- coding: utf-8 -*-
"""Cycle and monitoring helpers extracted from the main Gateway module."""
from __future__ import annotations
import asyncio
import logging
from typing import Any
from backend.data.market_ingest import ingest_symbols, refresh_news_for_symbols
from backend.domains import trading as trading_domain
from backend.utils.msg_adapter import FrontendAdapter
logger = logging.getLogger(__name__)
def schedule_watchlist_market_store_refresh(gateway: Any, tickers: list[str]) -> None:
"""Kick off a non-blocking market-store refresh for an updated watchlist."""
if not tickers:
return
if gateway._watchlist_ingest_task and not gateway._watchlist_ingest_task.done():
gateway._watchlist_ingest_task.cancel()
gateway._watchlist_ingest_task = asyncio.create_task(
refresh_market_store_for_watchlist(gateway, tickers),
)
async def refresh_market_store_for_watchlist(gateway: Any, tickers: list[str]) -> None:
"""Refresh the long-lived market store after a watchlist update."""
try:
await gateway.state_sync.on_system_message(
f"正在同步自选股市场数据: {', '.join(tickers)}",
)
results = await asyncio.to_thread(
ingest_symbols,
tickers,
mode="incremental",
)
summary = ", ".join(
f"{item['symbol']} prices={item['prices']} news={item['news']}"
for item in results
)
await gateway.state_sync.on_system_message(
f"自选股市场数据已同步: {summary}",
)
except asyncio.CancelledError:
raise
except Exception as exc:
logger.warning("Watchlist market store refresh failed: %s", exc)
await gateway.state_sync.on_system_message(
f"自选股市场数据同步失败: {exc}",
)
async def market_status_monitor(gateway: Any) -> None:
"""Periodically check and broadcast market status changes."""
while True:
try:
await gateway.market_service.check_and_broadcast_market_status()
status = gateway.market_service.get_market_status()
if status["status"] == "open" and not gateway.storage.is_live_session_active:
gateway.storage.start_live_session()
summary = gateway.storage.build_dashboard_snapshot_from_state(gateway.state_sync.state).get("summary") or {}
gateway._session_start_portfolio_value = summary.get(
"totalAssetValue",
gateway.storage.initial_cash,
)
logger.info(
"Session start portfolio: $%s",
f"{gateway._session_start_portfolio_value:,.2f}",
)
elif status["status"] != "open" and gateway.storage.is_live_session_active:
gateway.storage.end_live_session()
gateway._session_start_portfolio_value = None
if gateway.storage.is_live_session_active:
await update_and_broadcast_live_returns(gateway)
await asyncio.sleep(60)
except asyncio.CancelledError:
break
except Exception as exc:
logger.error("Market status monitor error: %s", exc)
await asyncio.sleep(60)
async def update_and_broadcast_live_returns(gateway: Any) -> None:
"""Calculate and broadcast live returns for current session."""
if not gateway.storage.is_live_session_active:
return
prices = gateway.market_service.get_all_prices()
if not prices or not any(p > 0 for p in prices.values()):
return
state = gateway.storage.load_internal_state()
equity_history = state.get("equity_history", [])
baseline_history = state.get("baseline_history", [])
baseline_vw_history = state.get("baseline_vw_history", [])
momentum_history = state.get("momentum_history", [])
current_equity = equity_history[-1]["v"] if equity_history else None
current_baseline = baseline_history[-1]["v"] if baseline_history else None
current_baseline_vw = baseline_vw_history[-1]["v"] if baseline_vw_history else None
current_momentum = momentum_history[-1]["v"] if momentum_history else None
point = gateway.storage.update_live_returns(
current_equity=current_equity,
current_baseline=current_baseline,
current_baseline_vw=current_baseline_vw,
current_momentum=current_momentum,
)
if point:
live_returns = gateway.storage.get_live_returns()
await gateway.broadcast(
{
"type": "team_summary",
"equity_return": live_returns["equity_return"],
"baseline_return": live_returns["baseline_return"],
"baseline_vw_return": live_returns["baseline_vw_return"],
"momentum_return": live_returns["momentum_return"],
},
)
async def on_strategy_trigger(gateway: Any, date: str) -> None:
"""Handle trading cycle trigger."""
if gateway._cycle_lock.locked():
logger.warning("Trading cycle already running, skipping trigger for %s", date)
await gateway.state_sync.on_system_message(f"已有交易周期在运行,跳过本次触发: {date}")
return
async with gateway._cycle_lock:
logger.info("Strategy triggered for %s", date)
tickers = gateway.config.get("tickers", [])
if gateway.is_backtest:
await run_backtest_cycle(gateway, date, tickers)
else:
await run_live_cycle(gateway, date, tickers)
async def on_heartbeat_trigger(gateway: Any, date: str) -> None:
"""Run lightweight heartbeat check for all analysts."""
logger.info("[Heartbeat] Running heartbeat check for %s", date)
analysts = gateway.pipeline._all_analysts()
for analyst in analysts:
try:
logger.debug(
"[Heartbeat] No heartbeat configured for %s, skipping",
analyst.name,
)
except Exception as exc:
logger.error("[Heartbeat] %s failed: %s", analyst.name, exc, exc_info=True)
async def run_backtest_cycle(gateway: Any, date: str, tickers: list[str]) -> None:
gateway.market_service.set_backtest_date(date)
await gateway.market_service.emit_market_open()
await gateway.state_sync.on_cycle_start(date)
prices = gateway.market_service.get_open_prices()
close_prices = gateway.market_service.get_close_prices()
market_caps = await get_market_caps(gateway, tickers, date)
result = await gateway.pipeline.run_cycle(
tickers=tickers,
date=date,
prices=prices,
close_prices=close_prices,
market_caps=market_caps,
)
await gateway.market_service.emit_market_close()
settlement_result = result.get("settlement_result")
save_cycle_results(gateway, result, date, close_prices, settlement_result)
await broadcast_portfolio_updates(gateway, result, close_prices)
await finalize_cycle(gateway, date)
async def run_live_cycle(gateway: Any, date: str, tickers: list[str]) -> None:
trading_date = gateway.market_service.get_live_trading_date()
logger.info("Live cycle: triggered=%s, trading_date=%s", date, trading_date)
try:
news_refresh = await asyncio.to_thread(
refresh_news_for_symbols,
tickers,
end_date=trading_date,
store=gateway.storage.market_store,
)
logger.info(
"News refresh complete: %s",
", ".join(
f"{item['symbol']} news={item['news']}"
for item in news_refresh
) or "no symbols",
)
except Exception as exc:
logger.warning("Live cycle news refresh failed: %s", exc)
await gateway.state_sync.on_cycle_start(trading_date)
market_caps = await get_market_caps(gateway, tickers, trading_date)
schedule_mode = gateway.config.get("schedule_mode", "daily")
market_status = gateway.market_service.get_market_status()
current_prices = gateway.market_service.get_all_prices()
if schedule_mode == "intraday":
execute_decisions = market_status.get("status") == "open"
if execute_decisions:
await gateway.state_sync.on_system_message("定时任务触发:当前处于交易时段,本轮将执行交易决策")
else:
await gateway.state_sync.on_system_message("定时任务触发:当前非交易时段,本轮仅更新数据与分析,不执行交易")
result = await gateway.pipeline.run_cycle(
tickers=tickers,
date=trading_date,
prices=current_prices,
market_caps=market_caps,
execute_decisions=execute_decisions,
)
close_prices = current_prices
else:
result = await gateway.pipeline.run_cycle(
tickers=tickers,
date=trading_date,
market_caps=market_caps,
get_open_prices_fn=gateway.market_service.wait_for_open_prices,
get_close_prices_fn=gateway.market_service.wait_for_close_prices,
)
close_prices = gateway.market_service.get_all_prices()
settlement_result = result.get("settlement_result")
save_cycle_results(gateway, result, trading_date, close_prices, settlement_result)
await broadcast_portfolio_updates(gateway, result, close_prices)
await finalize_cycle(gateway, trading_date)
async def finalize_cycle(gateway: Any, date: str) -> None:
dashboard_snapshot = gateway.storage.build_dashboard_snapshot_from_state(gateway.state_sync.state)
summary = dashboard_snapshot.get("summary") or {}
if gateway.storage.is_live_session_active:
summary.update(gateway.storage.get_live_returns())
await gateway.state_sync.on_cycle_end(date, portfolio_summary=summary)
leaderboard = dashboard_snapshot.get("leaderboard") or []
if leaderboard:
await gateway.state_sync.on_leaderboard_update(leaderboard)
async def get_market_caps(gateway: Any, tickers: list[str], date: str) -> dict[str, float]:
market_caps: dict[str, float] = {}
for ticker in tickers:
try:
market_cap = None
response = await gateway._call_trading_service(
f"get_market_cap for {ticker}",
lambda client, symbol=ticker: client.get_market_cap(ticker=symbol, end_date=date),
)
if response is not None:
market_cap = response.get("market_cap")
if market_cap is None:
payload = trading_domain.get_market_cap_payload(ticker=ticker, end_date=date)
market_cap = payload.get("market_cap")
market_caps[ticker] = market_cap if market_cap else 1e9
except Exception as exc:
logger.warning("Failed to get market cap for %s, using default 1e9: %s", ticker, exc)
market_caps[ticker] = 1e9
return market_caps
async def broadcast_portfolio_updates(gateway: Any, result: dict[str, Any], prices: dict[str, float]) -> None:
portfolio = result.get("portfolio", {})
if portfolio:
holdings = FrontendAdapter.build_holdings(portfolio, prices)
if holdings:
await gateway.state_sync.on_holdings_update(holdings)
stats = FrontendAdapter.build_stats(portfolio, prices)
if stats:
await gateway.state_sync.on_stats_update(stats)
executed_trades = result.get("executed_trades", [])
if executed_trades:
await gateway.state_sync.on_trades_executed(executed_trades)
def save_cycle_results(
gateway: Any,
result: dict[str, Any],
date: str,
prices: dict[str, float],
settlement_result: dict[str, Any] | None = None,
) -> None:
portfolio = result.get("portfolio", {})
executed_trades = result.get("executed_trades", [])
baseline_values = settlement_result.get("baseline_values") if settlement_result else None
if portfolio:
gateway.storage.update_dashboard_after_cycle(
portfolio=portfolio,
prices=prices,
date=date,
executed_trades=executed_trades,
baseline_values=baseline_values,
)
async def run_backtest_dates(gateway: Any, dates: list[str]) -> None:
gateway.state_sync.set_backtest_dates(dates)
await gateway.state_sync.on_system_message(f"Starting backtest - {len(dates)} trading days")
try:
for date in dates:
await gateway.on_strategy_trigger(date=date)
await asyncio.sleep(0.1)
await gateway.state_sync.on_system_message(f"Backtest complete - {len(dates)} days")
except Exception as exc:
error_msg = f"Backtest failed: {type(exc).__name__}: {str(exc)}"
logger.error(error_msg, exc_info=True)
asyncio.create_task(gateway.state_sync.on_system_message(error_msg))
raise
finally:
gateway._backtest_task = None
def handle_backtest_exception(gateway: Any, task: asyncio.Task) -> None:
try:
task.result()
except asyncio.CancelledError:
logger.info("Backtest task was cancelled")
except Exception as exc:
logger.error("Backtest task failed with exception:%s:%s", type(exc).__name__, exc, exc_info=True)
def handle_manual_cycle_exception(gateway: Any, task: asyncio.Task) -> None:
gateway._manual_cycle_task = None
try:
task.result()
except asyncio.CancelledError:
logger.info("Manual cycle task was cancelled")
except Exception as exc:
logger.error("Manual cycle task failed with exception:%s:%s", type(exc).__name__, exc, exc_info=True)
def set_backtest_dates(gateway: Any, dates: list[str]) -> None:
gateway.state_sync.set_backtest_dates(dates)
if dates:
gateway._backtest_start_date = dates[0]
gateway._backtest_end_date = dates[-1]
def stop_gateway(gateway: Any) -> None:
gateway.state_sync.save_state()
gateway.market_service.stop()
if gateway._backtest_task:
gateway._backtest_task.cancel()
if gateway._market_status_task:
gateway._market_status_task.cancel()
if gateway._watchlist_ingest_task:
gateway._watchlist_ingest_task.cancel()
# Close OpenClaw WebSocket connection
if gateway._openclaw_ws:
import asyncio
try:
loop = asyncio.get_event_loop()
if loop.is_running():
loop.create_task(gateway._openclaw_ws.disconnect())
else:
loop.run_until_complete(gateway._openclaw_ws.disconnect())
except Exception:
pass

View File

@@ -0,0 +1,534 @@
# -*- coding: utf-8 -*-
"""OpenClaw WebSocket handlers — gateway calls OpenClaw Gateway via WebSocket."""
from __future__ import annotations
import json
import logging
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from backend.services.gateway import Gateway
logger = logging.getLogger(__name__)
def _ensure_session_bridge(gateway) -> None:
"""Forward OpenClaw session events into 大时代 frontend websockets."""
if getattr(gateway, "_openclaw_session_bridge_ready", False):
return
async def _forward(event) -> None:
payload = event.payload or {}
session_key = str(payload.get("sessionKey") or payload.get("key") or "").strip()
if not session_key:
return
subscriber_map = getattr(gateway, "_openclaw_session_subscribers", {})
targets = [
ws
for ws, session_keys in list(subscriber_map.items())
if session_key in session_keys
]
if not targets:
return
message = json.dumps(
{
"type": "openclaw_session_event",
"event": event.event,
"session_key": session_key,
"payload": payload,
}
)
stale = []
for ws in targets:
try:
await ws.send(message)
except Exception:
stale.append(ws)
for ws in stale:
try:
subscriber_map.pop(ws, None)
except Exception:
pass
def _handler(event) -> None:
try:
import asyncio
asyncio.create_task(_forward(event))
except Exception as exc:
logger.debug("OpenClaw session bridge skipped event: %s", exc)
client = _get_ws_client(gateway)
client.add_event_handler(_handler)
gateway._openclaw_session_bridge_ready = True
gateway._openclaw_session_bridge_handler = _handler
if not hasattr(gateway, "_openclaw_session_subscribers"):
gateway._openclaw_session_subscribers = {}
def _get_ws_client(gateway) -> "OpenClawWebSocketClient":
"""Get the OpenClaw WebSocket client from gateway."""
from shared.client.openclaw_websocket_client import OpenClawWebSocketClient
client = gateway._openclaw_ws
if client is None:
raise RuntimeError("OpenClaw Gateway not connected")
return client
async def _ws_call(gateway, method: str, params: dict | None = None) -> dict:
"""Call OpenClaw Gateway via WebSocket and return result."""
try:
client = _get_ws_client(gateway)
return await client.call_method(method, params)
except Exception as exc:
logger.warning("OpenClaw Gateway call failed for %s: %s", method, exc)
return {"error": str(exc)[:200]}
async def handle_get_openclaw_status(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "status")
await websocket.send(json.dumps({"type": "openclaw_status_loaded", "data": result}))
async def handle_get_openclaw_sessions(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "sessions.list", {"limit": 50, "includeLastMessage": True})
await websocket.send(json.dumps({"type": "openclaw_sessions_loaded", "data": result}))
async def handle_get_openclaw_session_detail(gateway, websocket, data: dict) -> None:
session_key = data.get("session_key", "")
result = await _ws_call(gateway, "sessions.list", {"limit": 200, "includeLastMessage": True})
session = None
if isinstance(result, dict):
for item in result.get("sessions", []) or []:
if not isinstance(item, dict):
continue
if item.get("key") == session_key or item.get("sessionKey") == session_key:
session = item
break
await websocket.send(json.dumps({
"type": "openclaw_session_detail_loaded",
"data": {"session": session, "error": None if session else f"session '{session_key}' not found"},
"session_key": session_key,
}))
async def handle_get_openclaw_session_history(gateway, websocket, data: dict) -> None:
session_key = data.get("session_key", "")
limit = data.get("limit", 20)
try:
from backend.services.openclaw_cli import OpenClawCliService
result = OpenClawCliService().get_session_history_model(session_key, limit=limit)
payload = {
"session_key": result.session_key,
"session_id": result.session_id,
"history": result.events,
"events": result.events,
"raw_text": result.raw_text,
}
except Exception as exc:
payload = {"error": str(exc)[:200], "history": []}
await websocket.send(json.dumps({
"type": "openclaw_session_history_loaded",
"data": payload,
"session_key": session_key,
}))
async def handle_openclaw_resolve_session(gateway, websocket, data: dict) -> None:
params = {}
agent_id = str(data.get("agent_id") or "").strip()
label = str(data.get("label") or "").strip()
channel = str(data.get("channel") or "").strip()
if agent_id:
params["agentId"] = agent_id
if label:
params["label"] = label
if channel:
params["channel"] = channel
params["includeGlobal"] = bool(data.get("include_global", True))
result = await _ws_call(gateway, "sessions.resolve", params)
await websocket.send(json.dumps({"type": "openclaw_session_resolved", "data": result}))
async def handle_openclaw_create_session(gateway, websocket, data: dict) -> None:
params = {}
agent_id = str(data.get("agent_id") or "").strip()
label = str(data.get("label") or "").strip()
model = str(data.get("model") or "").strip()
initial_message = str(data.get("initial_message") or "").strip()
if agent_id:
params["agentId"] = agent_id
if label:
params["label"] = label
if model:
params["model"] = model
if initial_message:
params["message"] = initial_message
result = await _ws_call(gateway, "sessions.create", params)
await websocket.send(json.dumps({"type": "openclaw_session_created", "data": result}))
async def handle_openclaw_send_message(gateway, websocket, data: dict) -> None:
session_key = str(data.get("session_key") or "").strip()
message = str(data.get("message") or "").strip()
thinking = str(data.get("thinking") or "").strip()
if not session_key or not message:
await websocket.send(
json.dumps(
{
"type": "openclaw_message_sent",
"data": {"error": "session_key and message are required"},
}
)
)
return
params = {"key": session_key, "message": message}
if thinking:
params["thinking"] = thinking
result = await _ws_call(gateway, "sessions.send", params)
await websocket.send(
json.dumps(
{
"type": "openclaw_message_sent",
"data": result,
"session_key": session_key,
}
)
)
async def handle_openclaw_subscribe_session(gateway, websocket, data: dict) -> None:
session_key = str(data.get("session_key") or "").strip()
if not session_key:
await websocket.send(
json.dumps(
{
"type": "openclaw_session_subscribed",
"data": {"error": "session_key is required"},
}
)
)
return
_ensure_session_bridge(gateway)
result = await _ws_call(gateway, "sessions.messages.subscribe", {"key": session_key})
if not isinstance(result, dict) or not result.get("error"):
subscriber_map = getattr(gateway, "_openclaw_session_subscribers", {})
subscriber_map.setdefault(websocket, set()).add(session_key)
gateway._openclaw_session_subscribers = subscriber_map
await websocket.send(
json.dumps(
{
"type": "openclaw_session_subscribed",
"data": result,
"session_key": session_key,
}
)
)
async def handle_openclaw_unsubscribe_session(gateway, websocket, data: dict) -> None:
session_key = str(data.get("session_key") or "").strip()
if not session_key:
await websocket.send(
json.dumps(
{
"type": "openclaw_session_unsubscribed",
"data": {"error": "session_key is required"},
}
)
)
return
result = await _ws_call(gateway, "sessions.messages.unsubscribe", {"key": session_key})
subscriber_map = getattr(gateway, "_openclaw_session_subscribers", {})
session_keys = subscriber_map.get(websocket)
if isinstance(session_keys, set):
session_keys.discard(session_key)
if not session_keys:
subscriber_map.pop(websocket, None)
gateway._openclaw_session_subscribers = subscriber_map
await websocket.send(
json.dumps(
{
"type": "openclaw_session_unsubscribed",
"data": result,
"session_key": session_key,
}
)
)
async def handle_openclaw_reset_session(gateway, websocket, data: dict) -> None:
session_key = str(data.get("session_key") or "").strip()
if not session_key:
await websocket.send(
json.dumps(
{
"type": "openclaw_session_reset",
"data": {"error": "session_key is required"},
}
)
)
return
result = await _ws_call(gateway, "sessions.reset", {"key": session_key})
await websocket.send(
json.dumps(
{
"type": "openclaw_session_reset",
"data": result,
"session_key": session_key,
}
)
)
async def handle_openclaw_delete_session(gateway, websocket, data: dict) -> None:
session_key = str(data.get("session_key") or "").strip()
if not session_key:
await websocket.send(
json.dumps(
{
"type": "openclaw_session_deleted",
"data": {"error": "session_key is required"},
}
)
)
return
result = await _ws_call(gateway, "sessions.delete", {"key": session_key})
await websocket.send(
json.dumps(
{
"type": "openclaw_session_deleted",
"data": result,
"session_key": session_key,
}
)
)
async def handle_get_openclaw_cron(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "cron.list")
await websocket.send(json.dumps({"type": "openclaw_cron_loaded", "data": result}))
async def handle_get_openclaw_approvals(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "exec.approvals.get")
await websocket.send(json.dumps({"type": "openclaw_approvals_loaded", "data": result}))
async def handle_get_openclaw_agents(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "agents.list")
sessions_result = await _ws_call(
gateway,
"sessions.list",
{"limit": 200, "includeLastMessage": True},
)
config_result = await _ws_call(gateway, "config.get")
session_model_by_agent: dict[str, str] = {}
default_session_model: str | None = None
agent_skills_by_id: dict[str, list[str] | None] = {}
default_agent_skills: list[str] | None = None
parsed_config = config_result.get("parsed") if isinstance(config_result, dict) else None
if isinstance(parsed_config, dict):
agents_cfg = parsed_config.get("agents")
if isinstance(agents_cfg, dict):
defaults_cfg = agents_cfg.get("defaults")
if isinstance(defaults_cfg, dict):
default_skills = defaults_cfg.get("skills")
if isinstance(default_skills, list):
default_agent_skills = [
str(skill).strip()
for skill in default_skills
if str(skill).strip()
]
list_cfg = agents_cfg.get("list")
if isinstance(list_cfg, list):
for entry in list_cfg:
if not isinstance(entry, dict):
continue
agent_id = str(entry.get("id") or "").strip()
if not agent_id:
continue
skills = entry.get("skills")
if isinstance(skills, list):
agent_skills_by_id[agent_id] = [
str(skill).strip()
for skill in skills
if str(skill).strip()
]
elif skills == []:
agent_skills_by_id[agent_id] = []
if isinstance(sessions_result, dict) and isinstance(sessions_result.get("sessions"), list):
defaults = sessions_result.get("defaults")
if isinstance(defaults, dict):
value = (
defaults.get("model")
or defaults.get("modelName")
or defaults.get("model_name")
)
if value:
default_session_model = str(value)
for session in sessions_result.get("sessions", []):
if not isinstance(session, dict):
continue
agent_id = str(
session.get("agentId")
or session.get("agent_id")
or ""
).strip()
if not agent_id:
key = str(session.get("key") or session.get("sessionKey") or "").strip()
parts = key.split(":")
if len(parts) >= 3 and parts[0] == "agent":
agent_id = parts[1]
model_value = (
session.get("model")
or session.get("modelName")
or session.get("model_name")
or session.get("resolvedModel")
or session.get("resolved_model")
or session.get("defaultModel")
or session.get("default_model")
)
if agent_id and model_value and agent_id not in session_model_by_agent:
session_model_by_agent[agent_id] = str(model_value)
if isinstance(result, dict) and isinstance(result.get("agents"), list):
normalized_agents = []
for agent in result.get("agents", []):
if not isinstance(agent, dict):
normalized_agents.append(agent)
continue
normalized = dict(agent)
if not normalized.get("model"):
normalized["model"] = (
normalized.get("modelName")
or normalized.get("model_name")
or normalized.get("resolvedModel")
or normalized.get("resolved_model")
or normalized.get("defaultModel")
or normalized.get("default_model")
or session_model_by_agent.get(str(normalized.get("id") or "").strip())
or default_session_model
)
agent_id = str(normalized.get("id") or "").strip()
if "skills" not in normalized:
normalized["skills"] = agent_skills_by_id.get(agent_id, default_agent_skills)
normalized_agents.append(normalized)
result = {**result, "agents": normalized_agents}
await websocket.send(json.dumps({"type": "openclaw_agents_loaded", "data": result}))
async def handle_get_openclaw_agents_presence(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "node.list")
await websocket.send(json.dumps({"type": "openclaw_agents_presence_loaded", "data": result}))
async def handle_get_openclaw_skills(gateway, websocket, data: dict) -> None:
agent_id = str(data.get("agent_id") or "").strip()
params = {"agentId": agent_id} if agent_id else {}
result = await _ws_call(gateway, "skills.status", params)
await websocket.send(json.dumps({"type": "openclaw_skills_loaded", "data": result}))
async def handle_get_openclaw_models(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "models.list")
await websocket.send(json.dumps({"type": "openclaw_models_loaded", "data": result}))
async def handle_get_openclaw_hooks(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "tools.catalog")
await websocket.send(json.dumps({"type": "openclaw_hooks_loaded", "data": result}))
async def handle_get_openclaw_plugins(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "config.get")
await websocket.send(json.dumps({"type": "openclaw_plugins_loaded", "data": result}))
async def handle_get_openclaw_secrets_audit(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "secrets.reload")
await websocket.send(json.dumps({"type": "openclaw_secrets_audit_loaded", "data": result}))
async def handle_get_openclaw_security_audit(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "gateway.identity.get")
await websocket.send(json.dumps({"type": "openclaw_security_audit_loaded", "data": result}))
async def handle_get_openclaw_daemon_status(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "doctor.memory.status")
await websocket.send(json.dumps({"type": "openclaw_daemon_status_loaded", "data": result}))
async def handle_get_openclaw_pairing(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "device.pair.list")
await websocket.send(json.dumps({"type": "openclaw_pairing_loaded", "data": result}))
async def handle_get_openclaw_qr(gateway, websocket, data: dict) -> None:
await websocket.send(json.dumps({"type": "openclaw_qr_loaded", "data": {"error": "QR code not available via WebSocket"}}))
async def handle_get_openclaw_update_status(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "update.run")
await websocket.send(json.dumps({"type": "openclaw_update_status_loaded", "data": result}))
async def handle_get_openclaw_models_aliases(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "models.list")
await websocket.send(json.dumps({"type": "openclaw_models_aliases_loaded", "data": result}))
async def handle_get_openclaw_models_fallbacks(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "models.list")
await websocket.send(json.dumps({"type": "openclaw_models_fallbacks_loaded", "data": result}))
async def handle_get_openclaw_models_image_fallbacks(gateway, websocket, data: dict) -> None:
result = await _ws_call(gateway, "models.list")
await websocket.send(json.dumps({"type": "openclaw_models_image_fallbacks_loaded", "data": result}))
async def handle_get_openclaw_skill_update(gateway, websocket, data: dict) -> None:
slug = data.get("slug")
all_flag = data.get("all", False)
params = {}
if slug is not None:
params["slug"] = slug
if all_flag:
params["all"] = "true"
result = await _ws_call(gateway, "skills.update", params)
await websocket.send(json.dumps({"type": "openclaw_skill_update_loaded", "data": result}))
async def handle_get_openclaw_workspace_files(gateway, websocket, data: dict) -> None:
raw_workspace = data.get("workspace", "")
# Use the workspace param (which is actually the agent.id from frontend) as agent_id
agent_id = raw_workspace or "main"
result = await _ws_call(gateway, "agents.files.list", {"agentId": agent_id})
if isinstance(result, dict):
result["workspace"] = agent_id
await websocket.send(json.dumps({"type": "openclaw_workspace_files_loaded", "data": result}))
async def handle_get_openclaw_workspace_file(gateway, websocket, data: dict) -> None:
agent_id = data.get("agent_id", "main")
file_name = data.get("file_name", "")
if not file_name:
await websocket.send(json.dumps({"type": "openclaw_workspace_file_loaded", "data": {"error": "file_name is required"}}))
return
result = await _ws_call(gateway, "agents.files.get", {"agentId": agent_id, "name": file_name})
await websocket.send(json.dumps({"type": "openclaw_workspace_file_loaded", "data": result}))

View File

@@ -0,0 +1,161 @@
# -*- coding: utf-8 -*-
"""Runtime/state support helpers extracted from the main Gateway module."""
from __future__ import annotations
from typing import Any
from backend.data.provider_utils import normalize_symbol
def normalize_watchlist(raw_tickers: Any) -> list[str]:
"""Parse watchlist payloads from websocket messages."""
if raw_tickers is None:
return []
if isinstance(raw_tickers, str):
candidates = raw_tickers.split(",")
elif isinstance(raw_tickers, list):
candidates = raw_tickers
else:
candidates = [raw_tickers]
tickers: list[str] = []
for candidate in candidates:
symbol = normalize_symbol(str(candidate).strip().strip("\"'"))
if symbol and symbol not in tickers:
tickers.append(symbol)
return tickers
def normalize_agent_workspace_filename(
raw_name: Any,
*,
allowlist: set[str],
) -> str | None:
"""Restrict editable workspace files to a safe allowlist."""
filename = str(raw_name or "").strip()
if filename in allowlist:
return filename
return None
def apply_runtime_config(gateway: Any, runtime_config: dict[str, Any]) -> dict[str, Any]:
"""Apply runtime config to gateway-owned services and state."""
warnings: list[str] = []
ticker_changes = gateway.market_service.update_tickers(
runtime_config.get("tickers", []),
)
gateway.config["tickers"] = ticker_changes["active"]
gateway.pipeline.max_comm_cycles = int(runtime_config["max_comm_cycles"])
gateway.config["max_comm_cycles"] = gateway.pipeline.max_comm_cycles
gateway.config["schedule_mode"] = runtime_config.get(
"schedule_mode",
gateway.config.get("schedule_mode", "daily"),
)
gateway.config["interval_minutes"] = int(
runtime_config.get(
"interval_minutes",
gateway.config.get("interval_minutes", 60),
),
)
gateway.config["trigger_time"] = runtime_config.get(
"trigger_time",
gateway.config.get("trigger_time", "09:30"),
)
if gateway.scheduler:
gateway.scheduler.reconfigure(
mode=gateway.config["schedule_mode"],
trigger_time=gateway.config["trigger_time"],
interval_minutes=gateway.config["interval_minutes"],
)
pm_apply_result = gateway.pipeline.pm.apply_runtime_portfolio_config(
margin_requirement=runtime_config["margin_requirement"],
)
gateway.config["margin_requirement"] = gateway.pipeline.pm.portfolio.get(
"margin_requirement",
runtime_config["margin_requirement"],
)
requested_initial_cash = float(runtime_config["initial_cash"])
current_initial_cash = float(gateway.storage.initial_cash)
initial_cash_applied = requested_initial_cash == current_initial_cash
if not initial_cash_applied:
if (
gateway.storage.can_apply_initial_cash()
and gateway.pipeline.pm.can_apply_initial_cash()
):
initial_cash_applied = gateway.storage.apply_initial_cash(
requested_initial_cash,
)
if initial_cash_applied:
gateway.pipeline.pm.apply_runtime_portfolio_config(
initial_cash=requested_initial_cash,
)
gateway.config["initial_cash"] = gateway.storage.initial_cash
else:
warnings.append(
"initial_cash changed in BOOTSTRAP.md but was not applied "
"because the run already has positions, margin usage, or trades.",
)
requested_enable_memory = bool(runtime_config["enable_memory"])
current_enable_memory = bool(gateway.config.get("enable_memory", False))
if requested_enable_memory != current_enable_memory:
warnings.append(
"enable_memory changed in BOOTSTRAP.md but still requires a restart "
"because long-term memory contexts are created at startup.",
)
sync_runtime_state(gateway)
return {
"runtime_config_requested": runtime_config,
"runtime_config_applied": {
"tickers": list(gateway.config.get("tickers", [])),
"schedule_mode": gateway.config.get("schedule_mode", "daily"),
"interval_minutes": gateway.config.get("interval_minutes", 60),
"trigger_time": gateway.config.get("trigger_time", "09:30"),
"initial_cash": gateway.storage.initial_cash,
"margin_requirement": gateway.config["margin_requirement"],
"max_comm_cycles": gateway.config["max_comm_cycles"],
"enable_memory": gateway.config.get("enable_memory", False),
},
"runtime_config_status": {
"tickers": True,
"schedule_mode": True,
"interval_minutes": True,
"trigger_time": True,
"initial_cash": initial_cash_applied,
"margin_requirement": pm_apply_result["margin_requirement"],
"max_comm_cycles": True,
"enable_memory": requested_enable_memory == current_enable_memory,
},
"ticker_changes": ticker_changes,
"runtime_config_warnings": warnings,
}
def sync_runtime_state(gateway: Any) -> None:
"""Refresh persisted state after runtime config changes."""
gateway.state_sync.update_state("tickers", gateway.config.get("tickers", []))
gateway.state_sync.update_state(
"runtime_config",
{
"tickers": gateway.config.get("tickers", []),
"schedule_mode": gateway.config.get("schedule_mode", "daily"),
"interval_minutes": gateway.config.get("interval_minutes", 60),
"trigger_time": gateway.config.get("trigger_time", "09:30"),
"initial_cash": gateway.storage.initial_cash,
"margin_requirement": gateway.config.get("margin_requirement"),
"max_comm_cycles": gateway.config.get("max_comm_cycles"),
"enable_memory": gateway.config.get("enable_memory", False),
},
)
gateway.storage.update_server_state_from_dashboard(gateway.state_sync.state)
gateway.state_sync.save_state()

View File

@@ -0,0 +1,716 @@
# -*- coding: utf-8 -*-
"""Stock-related Gateway handlers extracted from the main Gateway module."""
from __future__ import annotations
import asyncio
import json
import logging
from datetime import datetime, timedelta
from typing import Any
from backend.data.provider_utils import normalize_symbol
from backend.domains import news as news_domain
from backend.domains import trading as trading_domain
from backend.enrich.news_enricher import enrich_news_for_symbol
from backend.enrich.llm_enricher import llm_enrichment_enabled
from backend.tools.data_tools import prices_to_df
from shared.client import NewsServiceClient, TradingServiceClient
logger = logging.getLogger(__name__)
async def handle_get_stock_history(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
ticker = normalize_symbol(data.get("ticker", ""))
if not ticker:
await websocket.send(json.dumps({
"type": "stock_history_loaded",
"ticker": "",
"prices": [],
"source": None,
"error": "invalid ticker",
}, ensure_ascii=False))
return
lookback_days = data.get("lookback_days", 90)
try:
lookback_days = max(7, min(int(lookback_days), 365))
except (TypeError, ValueError):
lookback_days = 90
end_date = gateway.state_sync.state.get("current_date") or datetime.now().strftime("%Y-%m-%d")
try:
end_dt = datetime.strptime(end_date, "%Y-%m-%d")
except ValueError:
end_dt = datetime.now()
end_date = end_dt.strftime("%Y-%m-%d")
start_date = (end_dt - timedelta(days=lookback_days)).strftime("%Y-%m-%d")
prices = []
source = "polygon"
response = await gateway._call_trading_service(
"get_prices for history",
lambda client: client.get_prices(ticker=ticker, start_date=start_date, end_date=end_date),
)
if response is not None:
prices = response.prices
source = "trading_service"
if not prices:
prices = await asyncio.to_thread(gateway.storage.market_store.get_ohlc, ticker, start_date, end_date)
if not prices:
payload = await asyncio.to_thread(
trading_domain.get_prices_payload,
ticker=ticker,
start_date=start_date,
end_date=end_date,
)
prices = payload.get("prices") or []
usage_snapshot = gateway._provider_router.get_usage_snapshot()
source = usage_snapshot.get("last_success", {}).get("prices")
if prices:
await asyncio.to_thread(
gateway.storage.market_store.upsert_ohlc,
ticker,
[price.model_dump() for price in prices],
source=source or "provider",
)
await websocket.send(json.dumps({
"type": "stock_history_loaded",
"ticker": ticker,
"prices": [price if isinstance(price, dict) else price.model_dump() for price in prices][-120:],
"source": source,
"start_date": start_date,
"end_date": end_date,
}, ensure_ascii=False, default=str))
async def handle_get_stock_explain_events(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
ticker = normalize_symbol(data.get("ticker", ""))
snapshot = gateway.storage.runtime_db.get_stock_explain_snapshot(ticker)
await websocket.send(json.dumps({
"type": "stock_explain_events_loaded",
"ticker": ticker,
"events": snapshot.get("events", []),
"signals": snapshot.get("signals", []),
"trades": snapshot.get("trades", []),
}, ensure_ascii=False, default=str))
async def handle_get_stock_news(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
ticker = normalize_symbol(data.get("ticker", ""))
if not ticker:
await websocket.send(json.dumps({
"type": "stock_news_loaded",
"ticker": "",
"news": [],
"source": None,
"error": "invalid ticker",
}, ensure_ascii=False))
return
lookback_days = data.get("lookback_days", 30)
limit = data.get("limit", 12)
try:
lookback_days = max(7, min(int(lookback_days), 180))
except (TypeError, ValueError):
lookback_days = 30
try:
limit = max(1, min(int(limit), 30))
except (TypeError, ValueError):
limit = 12
end_date = gateway.state_sync.state.get("current_date") or datetime.now().strftime("%Y-%m-%d")
try:
end_dt = datetime.strptime(end_date, "%Y-%m-%d")
except ValueError:
end_dt = datetime.now()
end_date = end_dt.strftime("%Y-%m-%d")
start_date = (end_dt - timedelta(days=lookback_days)).strftime("%Y-%m-%d")
news_rows = []
source = "polygon"
response = await gateway._call_news_service(
"get_enriched_news",
lambda client: client.get_enriched_news(
ticker=ticker,
start_date=start_date,
end_date=end_date,
limit=limit,
),
)
if response is not None:
news_rows = response.get("news") or []
source = "news_service"
if not news_rows:
payload = await asyncio.to_thread(
news_domain.get_enriched_news,
gateway.storage.market_store,
ticker=ticker,
start_date=start_date,
end_date=end_date,
limit=max(limit, 50),
refresh_if_stale=False,
)
news_rows = (payload.get("news") or [])[-limit:]
source = "market_store"
await websocket.send(json.dumps({
"type": "stock_news_loaded",
"ticker": ticker,
"news": news_rows[-limit:],
"source": source,
"start_date": start_date,
"end_date": end_date,
}, ensure_ascii=False, default=str))
async def handle_get_stock_news_for_date(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
ticker = normalize_symbol(data.get("ticker", ""))
trade_date = str(data.get("date") or "").strip()
if not ticker or not trade_date:
await websocket.send(json.dumps({
"type": "stock_news_for_date_loaded",
"ticker": ticker,
"date": trade_date,
"news": [],
"error": "ticker and date are required",
}, ensure_ascii=False))
return
limit = data.get("limit", 20)
try:
limit = max(1, min(int(limit), 50))
except (TypeError, ValueError):
limit = 20
source = "market_store"
news_rows = []
response = await gateway._call_news_service(
"get_news_for_date",
lambda client: client.get_news_for_date(ticker=ticker, date=trade_date, limit=limit),
)
if response is not None:
news_rows = response.get("news") or []
source = "news_service"
if not news_rows:
payload = await asyncio.to_thread(
news_domain.get_news_for_date,
gateway.storage.market_store,
ticker=ticker,
date=trade_date,
limit=limit,
refresh_if_stale=False,
)
news_rows = payload.get("news") or []
source = "market_store"
await websocket.send(json.dumps({
"type": "stock_news_for_date_loaded",
"ticker": ticker,
"date": trade_date,
"news": news_rows,
"source": source,
}, ensure_ascii=False, default=str))
async def handle_get_stock_news_timeline(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
ticker = normalize_symbol(data.get("ticker", ""))
if not ticker:
await websocket.send(json.dumps({
"type": "stock_news_timeline_loaded",
"ticker": "",
"timeline": [],
"error": "invalid ticker",
}, ensure_ascii=False))
return
lookback_days = data.get("lookback_days", 90)
try:
lookback_days = max(7, min(int(lookback_days), 365))
except (TypeError, ValueError):
lookback_days = 90
end_date = gateway.state_sync.state.get("current_date") or datetime.now().strftime("%Y-%m-%d")
try:
end_dt = datetime.strptime(end_date, "%Y-%m-%d")
except ValueError:
end_dt = datetime.now()
end_date = end_dt.strftime("%Y-%m-%d")
start_date = (end_dt - timedelta(days=lookback_days)).strftime("%Y-%m-%d")
timeline = []
response = await gateway._call_news_service(
"get_news_timeline",
lambda client: client.get_news_timeline(ticker=ticker, start_date=start_date, end_date=end_date),
)
if response is not None:
timeline = response.get("timeline") or []
if not timeline:
payload = await asyncio.to_thread(
news_domain.get_news_timeline,
gateway.storage.market_store,
ticker=ticker,
start_date=start_date,
end_date=end_date,
refresh_if_stale=False,
)
timeline = payload.get("timeline") or []
await websocket.send(json.dumps({
"type": "stock_news_timeline_loaded",
"ticker": ticker,
"timeline": timeline,
"start_date": start_date,
"end_date": end_date,
}, ensure_ascii=False, default=str))
async def handle_get_stock_news_categories(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
ticker = normalize_symbol(data.get("ticker", ""))
if not ticker:
await websocket.send(json.dumps({
"type": "stock_news_categories_loaded",
"ticker": "",
"categories": {},
"error": "invalid ticker",
}, ensure_ascii=False))
return
lookback_days = data.get("lookback_days", 90)
try:
lookback_days = max(7, min(int(lookback_days), 365))
except (TypeError, ValueError):
lookback_days = 90
end_date = gateway.state_sync.state.get("current_date") or datetime.now().strftime("%Y-%m-%d")
try:
end_dt = datetime.strptime(end_date, "%Y-%m-%d")
except ValueError:
end_dt = datetime.now()
end_date = end_dt.strftime("%Y-%m-%d")
start_date = (end_dt - timedelta(days=lookback_days)).strftime("%Y-%m-%d")
categories = {}
response = await gateway._call_news_service(
"get_categories",
lambda client: client.get_categories(
ticker=ticker,
start_date=start_date,
end_date=end_date,
limit=200,
),
)
if response is not None:
categories = response.get("categories") or {}
if not categories:
payload = await asyncio.to_thread(
news_domain.get_news_categories,
gateway.storage.market_store,
ticker=ticker,
start_date=start_date,
end_date=end_date,
limit=200,
refresh_if_stale=False,
)
categories = payload.get("categories") or {}
await websocket.send(json.dumps({
"type": "stock_news_categories_loaded",
"ticker": ticker,
"categories": categories,
"start_date": start_date,
"end_date": end_date,
}, ensure_ascii=False, default=str))
async def handle_get_stock_range_explain(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
ticker = normalize_symbol(data.get("ticker", ""))
start_date = str(data.get("start_date") or "").strip()
end_date = str(data.get("end_date") or "").strip()
if not ticker or not start_date or not end_date:
await websocket.send(json.dumps({
"type": "stock_range_explain_loaded",
"ticker": ticker,
"result": {"error": "ticker, start_date, end_date are required"},
}, ensure_ascii=False))
return
article_ids = data.get("article_ids")
result = None
response = await gateway._call_news_service(
"get_range_explain",
lambda client: client.get_range_explain(
ticker=ticker,
start_date=start_date,
end_date=end_date,
article_ids=article_ids if isinstance(article_ids, list) else None,
limit=100,
),
)
if response is not None:
result = response.get("result")
if result is None:
payload = await asyncio.to_thread(
news_domain.get_range_explain_payload,
gateway.storage.market_store,
ticker=ticker,
start_date=start_date,
end_date=end_date,
article_ids=article_ids if isinstance(article_ids, list) else None,
limit=100,
refresh_if_stale=False,
)
result = payload.get("result")
await websocket.send(json.dumps({
"type": "stock_range_explain_loaded",
"ticker": ticker,
"result": result,
}, ensure_ascii=False, default=str))
async def handle_get_stock_insider_trades(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
ticker = normalize_symbol(data.get("ticker", ""))
if not ticker:
await websocket.send(json.dumps({
"type": "stock_insider_trades_loaded",
"ticker": "",
"trades": [],
"error": "invalid ticker",
}, ensure_ascii=False))
return
end_date = str(data.get("end_date") or gateway.state_sync.state.get("current_date") or datetime.now().strftime("%Y-%m-%d")).strip()[:10]
start_date = str(data.get("start_date") or "").strip()[:10]
limit = int(data.get("limit", 50))
trades = []
response = await gateway._call_trading_service(
"get_insider_trades",
lambda client: client.get_insider_trades(
ticker=ticker,
end_date=end_date,
start_date=start_date if start_date else None,
limit=limit,
),
)
if response is not None:
trades = response.insider_trades
if not trades:
payload = await asyncio.to_thread(
trading_domain.get_insider_trades_payload,
ticker=ticker,
end_date=end_date,
start_date=start_date if start_date else None,
limit=limit,
)
trades = payload.get("insider_trades") or []
sorted_trades = sorted(trades, key=lambda t: t.transaction_date or "", reverse=True)
formatted_trades = [{
"ticker": t.ticker,
"name": t.name,
"title": t.title,
"is_board_director": t.is_board_director,
"transaction_date": t.transaction_date,
"transaction_shares": t.transaction_shares,
"transaction_price_per_share": t.transaction_price_per_share,
"transaction_value": t.transaction_value,
"shares_owned_before_transaction": t.shares_owned_before_transaction,
"shares_owned_after_transaction": t.shares_owned_after_transaction,
"security_title": t.security_title,
"filing_date": t.filing_date,
"holding_change": (
(t.shares_owned_after_transaction or 0) - (t.shares_owned_before_transaction or 0)
if t.shares_owned_after_transaction and t.shares_owned_before_transaction else None
),
"is_buy": ((t.transaction_shares or 0) > 0) if t.transaction_shares is not None else None,
} for t in sorted_trades]
await websocket.send(json.dumps({
"type": "stock_insider_trades_loaded",
"ticker": ticker,
"start_date": start_date or None,
"end_date": end_date,
"trades": formatted_trades,
}, ensure_ascii=False, default=str))
async def handle_get_stock_story(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
ticker = normalize_symbol(data.get("ticker", ""))
if not ticker:
await websocket.send(json.dumps({
"type": "stock_story_loaded",
"ticker": "",
"story": "",
"error": "invalid ticker",
}, ensure_ascii=False))
return
as_of_date = str(data.get("as_of_date") or gateway.state_sync.state.get("current_date") or datetime.now().strftime("%Y-%m-%d")).strip()[:10]
result = await gateway._call_news_service(
"get_story",
lambda client: client.get_story(ticker=ticker, as_of_date=as_of_date),
)
if result is None:
result = await asyncio.to_thread(
news_domain.get_story_payload,
gateway.storage.market_store,
ticker=ticker,
as_of_date=as_of_date,
)
await websocket.send(json.dumps({
"type": "stock_story_loaded",
"ticker": ticker,
"as_of_date": as_of_date,
"story": result.get("story") or "",
"source": result.get("source") or "local",
}, ensure_ascii=False, default=str))
async def handle_get_stock_similar_days(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
ticker = normalize_symbol(data.get("ticker", ""))
target_date = str(data.get("date") or "").strip()[:10]
if not ticker or not target_date:
await websocket.send(json.dumps({
"type": "stock_similar_days_loaded",
"ticker": ticker,
"date": target_date,
"items": [],
"error": "ticker and date are required",
}, ensure_ascii=False))
return
top_k = data.get("top_k", 8)
try:
top_k = max(1, min(int(top_k), 20))
except (TypeError, ValueError):
top_k = 8
result = await gateway._call_news_service(
"get_similar_days",
lambda client: client.get_similar_days(ticker=ticker, date=target_date, n_similar=top_k),
)
if result is None:
result = await asyncio.to_thread(
news_domain.get_similar_days_payload,
gateway.storage.market_store,
ticker=ticker,
date=target_date,
n_similar=top_k,
)
await websocket.send(json.dumps({
"type": "stock_similar_days_loaded",
"ticker": ticker,
"date": target_date,
**result,
}, ensure_ascii=False, default=str))
async def handle_get_stock_technical_indicators(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
ticker = normalize_symbol(data.get("ticker", ""))
if not ticker:
await websocket.send(json.dumps({
"type": "stock_technical_indicators_loaded",
"ticker": ticker,
"indicators": None,
"error": "ticker is required",
}, ensure_ascii=False))
return
try:
end_date = datetime.now()
start_date = end_date - timedelta(days=250)
prices = None
response = await gateway._call_trading_service(
"get_prices",
lambda client: client.get_prices(
ticker=ticker,
start_date=start_date.strftime("%Y-%m-%d"),
end_date=end_date.strftime("%Y-%m-%d"),
),
)
if response is not None:
prices = response.prices
if prices is None:
payload = trading_domain.get_prices_payload(
ticker=ticker,
start_date=start_date.strftime("%Y-%m-%d"),
end_date=end_date.strftime("%Y-%m-%d"),
)
prices = payload.get("prices") or []
if not prices or len(prices) < 20:
await websocket.send(json.dumps({
"type": "stock_technical_indicators_loaded",
"ticker": ticker,
"indicators": None,
"error": "Insufficient price data",
}, ensure_ascii=False))
return
df = prices_to_df(prices)
signal = gateway._technical_analyzer.analyze(ticker, df)
import pandas as pd
df_sorted = df.sort_values("time").reset_index(drop=True)
df_sorted["returns"] = df_sorted["close"].pct_change()
vol_10 = float(df_sorted["returns"].tail(10).std() * (252**0.5) * 100) if len(df_sorted) >= 10 else None
vol_20 = float(df_sorted["returns"].tail(20).std() * (252**0.5) * 100) if len(df_sorted) >= 20 else None
vol_60 = float(df_sorted["returns"].tail(60).std() * (252**0.5) * 100) if len(df_sorted) >= 60 else None
ma_distance = {}
for ma_key in ["ma5", "ma10", "ma20", "ma50", "ma200"]:
ma_value = getattr(signal, ma_key, None)
ma_distance[ma_key] = ((signal.current_price - ma_value) / ma_value) * 100 if ma_value and ma_value > 0 else None
indicators = {
"ticker": ticker,
"current_price": signal.current_price,
"ma": {
"ma5": signal.ma5,
"ma10": signal.ma10,
"ma20": signal.ma20,
"ma50": signal.ma50,
"ma200": signal.ma200,
"distance": ma_distance,
},
"rsi": {
"rsi14": signal.rsi14,
"status": "oversold" if signal.rsi14 < 30 else "overbought" if signal.rsi14 > 70 else "neutral",
},
"macd": {
"macd": signal.macd,
"signal": signal.macd_signal,
"histogram": signal.macd - signal.macd_signal,
},
"bollinger": {
"upper": signal.bollinger_upper,
"mid": signal.bollinger_mid,
"lower": signal.bollinger_lower,
},
"volatility": {
"vol_10d": vol_10,
"vol_20d": vol_20,
"vol_60d": vol_60,
"annualized": signal.annualized_volatility_pct,
"risk_level": signal.risk_level,
},
"trend": signal.trend,
"mean_reversion": signal.mean_reversion_signal,
}
await websocket.send(json.dumps({
"type": "stock_technical_indicators_loaded",
"ticker": ticker,
"indicators": indicators,
}, ensure_ascii=False, default=str))
except Exception as exc:
logger.exception("Error getting technical indicators for %s", ticker)
await websocket.send(json.dumps({
"type": "stock_technical_indicators_loaded",
"ticker": ticker,
"indicators": None,
"error": str(exc),
}, ensure_ascii=False))
async def handle_run_stock_enrich(gateway: Any, websocket: Any, data: dict[str, Any]) -> None:
ticker = normalize_symbol(data.get("ticker", ""))
start_date = str(data.get("start_date") or "").strip()[:10]
end_date = str(data.get("end_date") or "").strip()[:10]
story_date = str(data.get("story_date") or end_date or "").strip()[:10]
target_date = str(data.get("target_date") or "").strip()[:10]
force = bool(data.get("force", False))
rebuild_story = bool(data.get("rebuild_story", True))
rebuild_similar_days = bool(data.get("rebuild_similar_days", True))
only_local_to_llm = bool(data.get("only_local_to_llm", False))
limit = data.get("limit", 200)
try:
limit = max(10, min(int(limit), 500))
except (TypeError, ValueError):
limit = 200
if not ticker or not start_date or not end_date:
await websocket.send(json.dumps({
"type": "stock_enrich_completed",
"ticker": ticker,
"start_date": start_date,
"end_date": end_date,
"error": "ticker, start_date, end_date are required",
}, ensure_ascii=False))
return
if only_local_to_llm and not llm_enrichment_enabled():
await websocket.send(json.dumps({
"type": "stock_enrich_completed",
"ticker": ticker,
"start_date": start_date,
"end_date": end_date,
"error": "only_local_to_llm requires EXPLAIN_ENRICH_USE_LLM=true and a configured LLM provider",
}, ensure_ascii=False))
return
result = await asyncio.to_thread(
enrich_news_for_symbol,
gateway.storage.market_store,
ticker,
start_date=start_date,
end_date=end_date,
limit=limit,
skip_existing=not force,
only_reanalyze_local=only_local_to_llm,
)
story_status = None
if rebuild_story and story_date:
await asyncio.to_thread(gateway.storage.market_store.delete_story_cache, ticker, as_of_date=story_date)
story_result = await asyncio.to_thread(
news_domain.get_story_payload,
gateway.storage.market_store,
ticker=ticker,
as_of_date=story_date,
)
story_status = {"as_of_date": story_date, "source": story_result.get("source") or "local"}
similar_status = None
if rebuild_similar_days and target_date:
await asyncio.to_thread(gateway.storage.market_store.delete_similar_day_cache, ticker, target_date=target_date)
similar_result = await asyncio.to_thread(
news_domain.get_similar_days_payload,
gateway.storage.market_store,
ticker=ticker,
date=target_date,
n_similar=8,
)
similar_status = {
"target_date": target_date,
"count": len(similar_result.get("items") or []),
"error": similar_result.get("error"),
}
await websocket.send(json.dumps({
"type": "stock_enrich_completed",
"ticker": ticker,
"start_date": start_date,
"end_date": end_date,
"story_date": story_date or None,
"target_date": target_date or None,
"force": force,
"only_local_to_llm": only_local_to_llm,
"stats": result,
"story_status": story_status,
"similar_status": similar_status,
}, ensure_ascii=False, default=str))

View File

@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
"""
Market Data Service
Supports live, mock, and backtest modes
Supports live and backtest modes
"""
import asyncio
import logging
@@ -10,7 +10,7 @@ from typing import Any, Callable, Dict, List, Optional
from zoneinfo import ZoneInfo
import pandas_market_calendars as mcal
from backend.config.data_config import get_data_source
from backend.config.data_config import get_data_sources
from backend.data.provider_utils import normalize_symbol
logger = logging.getLogger(__name__)
@@ -36,7 +36,6 @@ class MarketService:
self,
tickers: List[str],
poll_interval: int = 10,
mock_mode: bool = False,
backtest_mode: bool = False,
api_key: Optional[str] = None,
backtest_start_date: Optional[str] = None,
@@ -44,7 +43,6 @@ class MarketService:
):
self.tickers = [normalize_symbol(ticker) for ticker in tickers]
self.poll_interval = poll_interval
self.mock_mode = mock_mode
self.backtest_mode = backtest_mode
self.api_key = api_key
self.backtest_start_date = backtest_start_date
@@ -69,8 +67,6 @@ class MarketService:
"""Return the active live quote provider for UI/debugging."""
if self.backtest_mode:
return "backtest"
if self.mock_mode:
return "mock"
if self._price_manager and hasattr(self._price_manager, "provider"):
provider = getattr(self._price_manager, "provider", None)
if isinstance(provider, str) and provider.strip():
@@ -81,8 +77,6 @@ class MarketService:
def mode_name(self) -> str:
if self.backtest_mode:
return "BACKTEST"
elif self.mock_mode:
return "MOCK"
return "LIVE"
async def start(self, broadcast_func: Callable):
@@ -96,8 +90,6 @@ class MarketService:
if self.backtest_mode:
self._start_backtest_mode()
elif self.mock_mode:
self._start_mock_mode()
else:
self._start_real_mode()
@@ -125,26 +117,10 @@ class MarketService:
return callback
def _start_mock_mode(self):
from backend.data.mock_price_manager import MockPriceManager
self._price_manager = MockPriceManager(
poll_interval=self.poll_interval,
volatility=0.5,
)
self._price_manager.add_price_callback(self._make_price_callback())
self._price_manager.subscribe(
self.tickers,
base_prices={t: 100.0 for t in self.tickers},
)
self._price_manager.start()
def _start_real_mode(self):
from backend.data.polling_price_manager import PollingPriceManager
provider = get_data_source()
if provider == "local_csv":
provider = "yfinance"
provider = self._resolve_live_quote_provider()
if provider == "finnhub" and not self.api_key:
raise ValueError("API key required for live mode")
@@ -157,6 +133,13 @@ class MarketService:
self._price_manager.subscribe(self.tickers)
self._price_manager.start()
def _resolve_live_quote_provider(self) -> str:
"""Pick the first configured provider that supports live quote polling."""
for provider in get_data_sources():
if provider in {"finnhub", "yfinance"}:
return provider
return "yfinance"
def _start_backtest_mode(self):
from backend.data.historical_price_manager import (
HistoricalPriceManager,
@@ -257,13 +240,7 @@ class MarketService:
if removed:
self._price_manager.unsubscribe(removed)
if added:
if self.mock_mode:
self._price_manager.subscribe(
added,
base_prices={ticker: 100.0 for ticker in added},
)
else:
self._price_manager.subscribe(added)
self._price_manager.subscribe(added)
if self.backtest_mode and self._current_date:
self._price_manager.set_date(self._current_date)

View File

@@ -0,0 +1,754 @@
# -*- coding: utf-8 -*-
"""Thin service wrapper around the OpenClaw CLI."""
from __future__ import annotations
import json
import os
import shlex
import shutil
import subprocess
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Any
from shared.models.openclaw import (
AgentSummary,
AgentsList,
ApprovalRequest,
ApprovalsList,
CronJob,
CronList,
DaemonStatus,
HookStatusEntry,
HookStatusReport,
ModelAliasesList,
ModelFallbacksList,
ModelRow,
ModelsList,
OpenClawStatus,
PairingListResponse,
PluginDiagnostic,
PluginRecord,
PluginsList,
QrCodeResponse,
SecretsAuditReport,
SecurityAuditResponse,
SecurityAuditReport,
SessionEntry,
SessionHistory,
SessionsList,
SkillStatusEntry,
SkillStatusReport,
SkillUpdateResult,
UpdateCheckResult,
UpdateStatusResponse,
normalize_agents,
normalize_approvals,
normalize_cron_jobs,
normalize_daemon_status,
normalize_hooks,
normalize_model_aliases,
normalize_model_fallbacks,
normalize_models,
normalize_pairing,
normalize_plugins,
normalize_qr,
normalize_security_audit,
normalize_secrets_audit,
normalize_session_history,
normalize_sessions,
normalize_skill_update,
normalize_skills,
normalize_status,
normalize_update_status,
)
PROJECT_ROOT = Path(__file__).resolve().parents[2]
REFERENCE_OPENCLAW_ROOT = PROJECT_ROOT / "reference" / "openclaw"
REFERENCE_OPENCLAW_ENTRY = REFERENCE_OPENCLAW_ROOT / "openclaw.mjs"
class OpenClawCliError(RuntimeError):
"""Raised when the OpenClaw CLI invocation fails."""
def __init__(
self,
message: str,
*,
command: list[str],
exit_code: int | None = None,
stdout: str = "",
stderr: str = "",
) -> None:
super().__init__(message)
self.command = command
self.exit_code = exit_code
self.stdout = stdout
self.stderr = stderr
@dataclass(frozen=True)
class OpenClawCliResult:
"""Command execution result."""
command: list[str]
exit_code: int
stdout: str
stderr: str
def resolve_openclaw_base_command() -> list[str]:
"""Resolve the command prefix used to launch OpenClaw."""
explicit = os.getenv("OPENCLAW_CMD", "").strip()
if explicit:
return shlex.split(explicit)
installed = shutil.which("openclaw")
if installed:
return [installed]
if REFERENCE_OPENCLAW_ENTRY.exists():
return [sys.executable if sys.executable.endswith("node") else "node", str(REFERENCE_OPENCLAW_ENTRY)]
return ["openclaw"]
def resolve_openclaw_cwd() -> Path:
"""Resolve the working directory for CLI execution."""
explicit = os.getenv("OPENCLAW_CWD", "").strip()
if explicit:
return Path(explicit).expanduser()
if REFERENCE_OPENCLAW_ROOT.exists():
return REFERENCE_OPENCLAW_ROOT
return PROJECT_ROOT
class OpenClawCliService:
"""OpenClaw CLI integration service."""
def __init__(
self,
*,
base_command: list[str] | None = None,
cwd: Path | None = None,
timeout_seconds: float | None = None,
) -> None:
self.base_command = list(base_command or resolve_openclaw_base_command())
self.cwd = cwd or resolve_openclaw_cwd()
self.timeout_seconds = timeout_seconds or float(
os.getenv("OPENCLAW_TIMEOUT_SECONDS", "15")
)
def health(self) -> dict[str, Any]:
"""Return the current CLI wiring state."""
binary = self.base_command[0] if self.base_command else "openclaw"
resolved = shutil.which(binary) if len(self.base_command) == 1 else binary
return {
"status": "healthy",
"service": "openclaw-service",
"base_command": self.base_command,
"cwd": str(self.cwd),
"binary_resolved": resolved is not None,
"reference_entry_available": REFERENCE_OPENCLAW_ENTRY.exists(),
"timeout_seconds": self.timeout_seconds,
}
def status(self) -> dict[str, Any]:
"""Read `openclaw status --json`."""
return self.run_json(["status", "--json"])
def list_sessions(self) -> dict[str, Any]:
"""Read `openclaw sessions --json`."""
return self.run_json(["sessions", "--json"])
def get_session(self, session_key: str) -> dict[str, Any]:
"""Resolve a single session out of the sessions list."""
payload = self.list_sessions()
sessions = payload.get("sessions") or []
for item in sessions:
if not isinstance(item, dict):
continue
if item.get("key") == session_key or item.get("sessionKey") == session_key:
return item
raise KeyError(session_key)
def get_session_history(self, session_key: str, *, limit: int = 20) -> dict[str, Any]:
"""Read session history with a JSON-first fallback to raw text."""
args = ["sessions", "history", session_key, "--json", "--limit", str(limit)]
try:
return self.run_json(args)
except OpenClawCliError as exc:
raise exc
except json.JSONDecodeError:
result = self.run(args)
return {
"sessionKey": session_key,
"limit": limit,
"rawText": result.stdout,
}
def list_cron_jobs(self) -> dict[str, Any]:
"""Read `openclaw cron list --json`."""
return self.run_json(["cron", "list", "--json"])
def list_approvals(self) -> dict[str, Any]:
"""Read `openclaw approvals get --json`."""
return self.run_json(["approvals", "get", "--json"])
def list_agents(self) -> dict[str, Any]:
"""Read `openclaw agents list --json`."""
return self.run_json(["agents", "list", "--json"])
def list_skills(self) -> dict[str, Any]:
"""Read `openclaw skills list --json`."""
return self.run_json(["skills", "list", "--json"])
def list_models(self) -> dict[str, Any]:
"""Read `openclaw models list --json`."""
return self.run_json(["models", "list", "--json"])
def list_hooks(self) -> dict[str, Any]:
"""Read `openclaw hooks list --json`."""
return self.run_json(["hooks", "list", "--json"])
def list_plugins(self) -> dict[str, Any]:
"""Read `openclaw plugins list --json`."""
return self.run_json(["plugins", "list", "--json"])
def secrets_audit(self) -> dict[str, Any]:
"""Read `openclaw secrets audit --json`."""
return self.run_json(["secrets", "audit", "--json"])
def security_audit(self) -> dict[str, Any]:
"""Read `openclaw security audit --json`."""
return self.run_json(["security", "audit", "--json"])
def daemon_status(self) -> dict[str, Any]:
"""Read `openclaw daemon status --json`."""
return self.run_json(["daemon", "status", "--json"])
def pairing_list(self) -> dict[str, Any]:
"""Read `openclaw pairing list --json`."""
return self.run_json(["pairing", "list", "--json"])
def qr_code(self) -> dict[str, Any]:
"""Read `openclaw qr --json`."""
return self.run_json(["qr", "--json"])
def update_status(self) -> dict[str, Any]:
"""Read `openclaw update status --json`."""
return self.run_json(["update", "status", "--json"])
def list_model_aliases(self) -> dict[str, Any]:
"""Read `openclaw models aliases list --json`."""
return self.run_json(["models", "aliases", "list", "--json"])
def list_model_fallbacks(self) -> dict[str, Any]:
"""Read `openclaw models fallbacks list --json`."""
return self.run_json(["models", "fallbacks", "list", "--json"])
def list_model_image_fallbacks(self) -> dict[str, Any]:
"""Read `openclaw models image-fallbacks list --json`."""
return self.run_json(["models", "image-fallbacks", "list", "--json"])
def skill_update(self, *, slug: str | None = None, all: bool = False) -> dict[str, Any]:
"""Read `openclaw skills update --json`."""
args = ["skills", "update", "--json"]
if slug:
args.append(slug)
if all:
args.append("--all")
return self.run_json(args)
def models_status(self, *, probe: bool = False) -> dict[str, Any]:
"""Read `openclaw models status --json [--probe]`."""
args = ["models", "status", "--json"]
if probe:
args.append("--probe")
return self.run_json(args)
def channels_status(self, *, probe: bool = False) -> dict[str, Any]:
"""Read `openclaw channels status [--probe] --json`."""
args = ["channels", "status", "--json"]
if probe:
args.append("--probe")
return self.run_json(args)
def list_workspace_files(self, workspace_path: str) -> dict[str, Any]:
"""List .md files in an OpenClaw agent workspace with their content.
Reads the workspace directory and returns metadata + content for each .md file.
"""
import json
from pathlib import Path
wp = Path(workspace_path).expanduser().resolve()
if not wp.exists() or not wp.is_dir():
return {"workspace": str(wp), "files": [], "error": "workspace not found"}
md_files = sorted(wp.glob("*.md"))
files = []
for md_file in md_files:
try:
content = md_file.read_text(encoding="utf-8")
# Preview: first 300 chars
preview = content[:300].strip()
files.append({
"name": md_file.name,
"path": str(md_file),
"size": len(content),
"preview": preview,
"previewTruncated": len(content) > 300,
})
except OSError as exc:
files.append({
"name": md_file.name,
"path": str(md_file),
"size": 0,
"preview": "",
"error": str(exc),
})
return {"workspace": str(wp), "files": files}
def channels_list(self) -> dict[str, Any]:
"""Read `openclaw channels list --json`."""
return self.run_json(["channels", "list", "--json"])
def hook_info(self, name: str) -> dict[str, Any]:
"""Read `openclaw hooks info <name> --json`."""
args = ["hooks", "info", name, "--json"]
try:
return self.run_json(args)
except json.JSONDecodeError:
result = self.run(args)
return {"raw": result.stdout}
def hooks_check(self) -> dict[str, Any]:
"""Read `openclaw hooks check --json`."""
return self.run_json(["hooks", "check", "--json"])
def plugins_inspect(self, *, plugin_id: str | None = None, all: bool = False) -> dict[str, Any]:
"""Read `openclaw plugins inspect [--json] [--all]`."""
args = ["plugins", "inspect", "--json"]
if all:
args.append("--all")
elif plugin_id:
args.append(plugin_id)
return self.run_json(args)
# -------------------------------------------------------------------------
# Typed variants — these use Pydantic models and are the preferred path.
# -------------------------------------------------------------------------
def status_model(self) -> OpenClawStatus:
"""Read and parse `openclaw status --json` into a typed model."""
raw = self.status()
return normalize_status(raw)
def list_sessions_model(self) -> SessionsList:
"""Read and parse `openclaw sessions --json` into a typed model."""
raw = self.list_sessions()
return normalize_sessions(raw)
def get_session_model(self, session_key: str) -> SessionEntry:
"""Resolve a single session and return a typed model."""
raw = self.get_session(session_key)
return SessionEntry.model_validate(raw, strict=False)
def get_session_history_model(self, session_key: str, *, limit: int = 20) -> SessionHistory:
"""Read session history and return a typed model."""
raw = self.get_session_history(session_key, limit=limit)
return normalize_session_history(raw, session_key=session_key)
def list_cron_jobs_model(self) -> CronList:
"""Read and parse `openclaw cron list --json` into a typed model."""
raw = self.list_cron_jobs()
return normalize_cron_jobs(raw)
def list_approvals_model(self) -> ApprovalsList:
"""Read and parse `openclaw approvals get --json` into a typed model."""
raw = self.list_approvals()
return normalize_approvals(raw)
# -------------------------------------------------------------------------
# Typed variants
# -------------------------------------------------------------------------
def list_agents_model(self) -> AgentsList:
"""Read and parse `openclaw agents list --json` into a typed model."""
raw = self.list_agents()
if isinstance(raw, list):
return AgentsList(agents=[AgentSummary.model_validate(a, strict=False) for a in raw if isinstance(a, dict)])
return normalize_agents(raw)
def list_skills_model(self) -> SkillStatusReport:
"""Read and parse `openclaw skills list --json` into a typed model."""
raw = self.list_skills()
return normalize_skills(raw)
def list_models_model(self) -> ModelsList:
"""Read and parse `openclaw models list --json` into a typed model."""
raw = self.list_models()
if isinstance(raw, list):
return ModelsList(models=[ModelRow.model_validate(m, strict=False) for m in raw if isinstance(m, dict)])
return normalize_models(raw)
def list_hooks_model(self) -> HookStatusReport:
raw = self.list_hooks()
return normalize_hooks(raw)
def list_plugins_model(self) -> PluginsList:
raw = self.list_plugins()
return normalize_plugins(raw)
def secrets_audit_model(self) -> SecretsAuditReport:
raw = self.secrets_audit()
return normalize_secrets_audit(raw)
def security_audit_model(self) -> SecurityAuditResponse:
raw = self.security_audit()
return normalize_security_audit(raw)
def daemon_status_model(self) -> DaemonStatus:
raw = self.daemon_status()
return normalize_daemon_status(raw)
def pairing_list_model(self) -> PairingListResponse:
raw = self.pairing_list()
return normalize_pairing(raw)
def qr_code_model(self) -> QrCodeResponse:
raw = self.qr_code()
return normalize_qr(raw)
def update_status_model(self) -> UpdateStatusResponse:
raw = self.update_status()
return normalize_update_status(raw)
def list_model_aliases_model(self) -> ModelAliasesList:
raw = self.list_model_aliases()
return normalize_model_aliases(raw)
def list_model_fallbacks_model(self) -> ModelFallbacksList:
raw = self.list_model_fallbacks()
return normalize_model_fallbacks(raw)
def list_model_image_fallbacks_model(self) -> ModelFallbacksList:
raw = self.list_model_image_fallbacks()
return normalize_model_fallbacks(raw)
def skill_update_model(self, *, slug: str | None = None, all: bool = False) -> SkillUpdateResult:
raw = self.skill_update(slug=slug, all=all)
return normalize_skill_update(raw)
def models_status_model(self, *, probe: bool = False) -> dict[str, Any]:
"""Read `openclaw models status --json` and return the raw dict."""
return self.models_status(probe=probe)
def channels_status_model(self, *, probe: bool = False) -> dict[str, Any]:
"""Read `openclaw channels status --json` and return the raw dict."""
return self.channels_status(probe=probe)
def channels_list_model(self) -> dict[str, Any]:
"""Read `openclaw channels list --json` and return the raw dict."""
return self.channels_list()
def hook_info_model(self, name: str) -> dict[str, Any]:
"""Read `openclaw hooks info <name> --json` and return the raw dict."""
return self.hook_info(name)
def hooks_check_model(self) -> dict[str, Any]:
"""Read `openclaw hooks check --json` and return the raw dict."""
return self.hooks_check()
def plugins_inspect_model(self, *, plugin_id: str | None = None, all: bool = False) -> dict[str, Any]:
"""Read `openclaw plugins inspect --json [--all]` and return the raw dict."""
return self.plugins_inspect(plugin_id=plugin_id, all=all)
def agents_bindings(self, *, agent: str | None = None) -> dict[str, Any]:
"""Read `openclaw agents bindings --json [--agent <id>]`."""
args = ["agents", "bindings", "--json"]
if agent:
args.extend(["--agent", agent])
return self.run_json(args)
def agents_bindings_model(self, *, agent: str | None = None) -> dict[str, Any]:
"""Read `openclaw agents bindings --json` and return the raw dict."""
return self.agents_bindings(agent=agent)
def agents_presence(self) -> dict[str, Any]:
"""Read session presence for all agents from runtime session files.
Reads ~/.openclaw/agents/{agentId}/sessions/sessions.json for each agent
and counts sessions in active states within a recency window.
"""
import json
from pathlib import Path
openclaw_home = Path.home() / ".openclaw"
agents_path = openclaw_home / "agents"
if not agents_path.exists():
return {"status": "not_connected", "agents": {}}
ACTIVE_STATES = {
"running", "active", "busy", "blocked", "waiting_approval",
"working", "in_progress", "processing", "thinking", "executing", "streaming",
}
RECENCY_WINDOW_MS = 45 * 60 * 1000 # 45 minutes
result: dict[str, Any] = {"status": "connected", "agents": {}}
try:
for agent_dir in agents_path.iterdir():
if not agent_dir.is_dir():
continue
sessions_file = agent_dir / "sessions" / "sessions.json"
if not sessions_file.exists():
continue
try:
sessions_data = json.loads(sessions_file.read_text())
except (json.JSONDecodeError, OSError):
continue
sessions = sessions_data if isinstance(sessions_data, list) else []
now_ms = 0 # placeholder; we'll skip recency check if no ts field
active_count = 0
for session in sessions:
if not isinstance(session, dict):
continue
state = str(session.get("state") or session.get("status") or "").lower()
if state in ACTIVE_STATES:
active_count += 1
if active_count > 0:
result["agents"][agent_dir.name] = {
"activeSessions": active_count,
"status": "active",
}
else:
result["agents"][agent_dir.name] = {
"activeSessions": 0,
"status": "idle",
}
except OSError:
result["status"] = "partial"
return result
def agents_from_config(self) -> dict[str, Any]:
"""Read agent list directly from openclaw.json config file.
Falls back to scanning ~/.openclaw/agents/ directories when config is absent.
This avoids the CLI timeout from `agents list --json`.
"""
import json
openclaw_home = Path.home() / ".openclaw"
config_path = openclaw_home / "openclaw.json"
if not config_path.exists():
return {"status": "not_connected", "agents": []}
try:
raw = json.loads(config_path.read_text())
except (json.JSONDecodeError, OSError):
return {"status": "partial", "agents": []}
agents_list = raw.get("agents", {}).get("list", [])
if not agents_list:
return {"status": "partial", "agents": [], "detail": "agents.list is empty"}
agents = []
for entry in agents_list:
if not isinstance(entry, dict):
continue
agent_id = entry.get("id", "").strip()
if not agent_id:
continue
agents.append({
"id": agent_id,
"name": entry.get("name", "").strip() or agent_id,
"model": entry.get("model") or "",
"workspace": entry.get("workspace") or "",
"is_default": entry.get("id") == raw.get("agents", {}).get("defaults", {}).get("id"),
})
return {"status": "connected", "agents": agents}
def gateway_status(self, *, url: str | None = None, token: str | None = None) -> dict[str, Any]:
"""Read `openclaw gateway status --json [--url <url>] [--token <token>]`. May fail if gateway is unreachable."""
args = ["gateway", "status", "--json"]
if url:
args.extend(["--url", url])
if token:
args.extend(["--token", token])
return self.run_json(args)
def memory_status(self, *, agent: str | None = None, deep: bool = False) -> dict[str, Any]:
"""Read `openclaw memory status --json [--agent <id>] [--deep]`. Returns array of per-agent status."""
args = ["memory", "status", "--json"]
if agent:
args.extend(["--agent", agent])
if deep:
args.append("--deep")
return self.run_json(args)
# -------------------------------------------------------------------------
# Write agents commands
# -------------------------------------------------------------------------
def agents_add(
self,
name: str,
*,
workspace: str | None = None,
model: str | None = None,
agent_dir: str | None = None,
bind: list[str] | None = None,
non_interactive: bool = False,
) -> dict[str, Any]:
"""Run `openclaw agents add <name> [--workspace <dir>] [--model <id>] [--agent-dir <dir>] [--bind <spec>] [--non-interactive] --json`."""
args = ["agents", "add", name, "--json"]
if workspace:
args.extend(["--workspace", workspace])
if model:
args.extend(["--model", model])
if agent_dir:
args.extend(["--agent-dir", agent_dir])
if bind:
for b in bind:
args.extend(["--bind", b])
if non_interactive:
args.append("--non-interactive")
return self.run_json(args)
def agents_delete(self, id: str, *, force: bool = False) -> dict[str, Any]:
"""Run `openclaw agents delete <id> [--force] --json`."""
args = ["agents", "delete", id, "--json"]
if force:
args.append("--force")
return self.run_json(args)
def agents_bind(
self,
*,
agent: str | None = None,
bind: list[str] | None = None,
) -> dict[str, Any]:
"""Run `openclaw agents bind [--agent <id>] [--bind <spec>] --json`."""
args = ["agents", "bind", "--json"]
if agent:
args.extend(["--agent", agent])
if bind:
for b in bind:
args.extend(["--bind", b])
return self.run_json(args)
def agents_unbind(
self,
*,
agent: str | None = None,
bind: list[str] | None = None,
all: bool = False,
) -> dict[str, Any]:
"""Run `openclaw agents unbind [--agent <id>] [--bind <spec>] [--all] --json`."""
args = ["agents", "unbind", "--json"]
if agent:
args.extend(["--agent", agent])
if bind:
for b in bind:
args.extend(["--bind", b])
if all:
args.append("--all")
return self.run_json(args)
def agents_set_identity(
self,
*,
agent: str | None = None,
workspace: str | None = None,
identity_file: str | None = None,
name: str | None = None,
emoji: str | None = None,
theme: str | None = None,
avatar: str | None = None,
from_identity: bool = False,
) -> dict[str, Any]:
"""Run `openclaw agents set-identity [--agent <id>] [--workspace <dir>] [--identity-file <path>] [--from-identity] [--name <n>] [--emoji <e>] [--theme <t>] [--avatar <a>] --json`."""
args = ["agents", "set-identity", "--json"]
if agent:
args.extend(["--agent", agent])
if workspace:
args.extend(["--workspace", workspace])
if identity_file:
args.extend(["--identity-file", identity_file])
if from_identity:
args.append("--from-identity")
if name:
args.extend(["--name", name])
if emoji:
args.extend(["--emoji", emoji])
if theme:
args.extend(["--theme", theme])
if avatar:
args.extend(["--avatar", avatar])
return self.run_json(args)
def run_json(self, args: list[str]) -> dict[str, Any]:
"""Run the CLI and decode JSON stdout, falling back to stderr."""
result = self.run(args)
text = result.stdout.strip() or result.stderr.strip()
if not text:
return {}
return json.loads(text)
def run(self, args: list[str]) -> OpenClawCliResult:
"""Run the CLI and return stdout/stderr."""
command = [*self.base_command, *args]
env = os.environ.copy()
try:
completed = subprocess.run(
command,
cwd=self.cwd,
env=env,
capture_output=True,
text=True,
timeout=self.timeout_seconds,
check=False,
)
except FileNotFoundError as exc:
raise OpenClawCliError(
"OpenClaw CLI executable was not found.",
command=command,
) from exc
except subprocess.TimeoutExpired as exc:
raise OpenClawCliError(
f"OpenClaw CLI timed out after {self.timeout_seconds:.1f}s.",
command=command,
stdout=exc.stdout or "",
stderr=exc.stderr or "",
) from exc
if completed.returncode != 0:
raise OpenClawCliError(
"OpenClaw CLI command failed.",
command=command,
exit_code=completed.returncode,
stdout=completed.stdout,
stderr=completed.stderr,
)
return OpenClawCliResult(
command=command,
exit_code=completed.returncode,
stdout=completed.stdout,
stderr=completed.stderr,
)

View File

@@ -9,7 +9,7 @@ from datetime import datetime
from pathlib import Path
from typing import Any, Dict, Iterable
from backend.data.schema import CompanyNews
from shared.schema import CompanyNews
SCHEMA = """

View File

@@ -11,7 +11,6 @@ from pathlib import Path
from typing import Any, Dict, List, Optional
from backend.data.market_store import MarketStore
from .research_db import ResearchDb
from .runtime_db import RuntimeDb
logger = logging.getLogger(__name__)
@@ -22,19 +21,25 @@ class StorageService:
Storage service for data persistence
Responsibilities:
1. Load/save dashboard JSON files
1. Export dashboard JSON files
(summary, holdings, stats, trades, leaderboard)
2. Load/save internal state (_internal_state.json)
3. Load/save server state (server_state.json) with feed history
4. Manage portfolio state persistence
5. Support loading from saved state to resume execution
Notes:
- team_dashboard/*.json is treated as an export/compatibility layer
rather than the authoritative runtime source of truth.
- authoritative runtime reads should prefer in-memory state, server_state,
runtime.db, and market_research.db.
"""
def __init__(
self,
dashboard_dir: Path,
initial_cash: float = 100000.0,
config_name: str = "mock",
config_name: str = "live",
):
"""
Initialize storage service
@@ -49,7 +54,7 @@ class StorageService:
self.initial_cash = initial_cash
self.config_name = config_name
# Dashboard file paths
# Dashboard export file paths
self.files = {
"summary": self.dashboard_dir / "summary.json",
"holdings": self.dashboard_dir / "holdings.json",
@@ -66,7 +71,6 @@ class StorageService:
self.state_dir.mkdir(parents=True, exist_ok=True)
self.server_state_file = self.state_dir / "server_state.json"
self.runtime_db = RuntimeDb(self.state_dir / "runtime.db")
self.research_db = ResearchDb(self.state_dir / "research.db")
self.market_store = MarketStore()
# Feed history (for agent messages)
@@ -84,16 +88,8 @@ class StorageService:
logger.info(f"Storage service initialized: {self.dashboard_dir}")
def load_file(self, file_type: str) -> Optional[Any]:
"""
Load dashboard JSON file
Args:
file_type: One of: summary, holdings, stats, trades, leaderboard
Returns:
Loaded data or None if file doesn't exist
"""
def load_export_file(self, file_type: str) -> Optional[Any]:
"""Load dashboard export JSON file."""
file_path = self.files.get(file_type)
if not file_path or not file_path.exists():
return None
@@ -105,14 +101,12 @@ class StorageService:
logger.error(f"Failed to load {file_type}.json: {e}")
return None
def save_file(self, file_type: str, data: Any):
"""
Save dashboard JSON file
def load_file(self, file_type: str) -> Optional[Any]:
"""Backward-compatible alias for export-layer JSON reads."""
return self.load_export_file(file_type)
Args:
file_type: One of: summary, holdings, stats, trades, leaderboard
data: Data to save
"""
def save_export_file(self, file_type: str, data: Any):
"""Save dashboard export JSON file."""
file_path = self.files.get(file_type)
if not file_path:
logger.error(f"Unknown file type: {file_type}")
@@ -129,6 +123,48 @@ class StorageService:
except Exception as e:
logger.error(f"Failed to save {file_type}.json: {e}")
def save_file(self, file_type: str, data: Any):
"""Backward-compatible alias for export-layer JSON writes."""
self.save_export_file(file_type, data)
def build_dashboard_snapshot_from_state(
self,
state: Optional[Dict[str, Any]] = None,
) -> Dict[str, Any]:
"""Build dashboard view data from runtime state instead of JSON exports."""
runtime_state = state or self.load_server_state()
portfolio = dict(runtime_state.get("portfolio") or {})
holdings = list(runtime_state.get("holdings") or [])
stats = runtime_state.get("stats") or self._get_default_stats()
trades = list(runtime_state.get("trades") or [])
leaderboard = list(runtime_state.get("leaderboard") or [])
summary = {
"totalAssetValue": portfolio.get("total_value", self.initial_cash),
"totalReturn": portfolio.get("pnl_percent", 0.0),
"cashPosition": portfolio.get("cash", self.initial_cash),
"tickerWeights": stats.get("tickerWeights", {}),
"totalTrades": len(trades),
"pnlPct": portfolio.get("pnl_percent", 0.0),
"balance": portfolio.get("total_value", self.initial_cash),
"equity": portfolio.get("equity", []),
"baseline": portfolio.get("baseline", []),
"baseline_vw": portfolio.get("baseline_vw", []),
"momentum": portfolio.get("momentum", []),
"equity_return": portfolio.get("equity_return", []),
"baseline_return": portfolio.get("baseline_return", []),
"baseline_vw_return": portfolio.get("baseline_vw_return", []),
"momentum_return": portfolio.get("momentum_return", []),
}
return {
"summary": summary,
"holdings": holdings,
"stats": stats,
"trades": trades,
"leaderboard": leaderboard,
}
def check_file_updates(self) -> Dict[str, bool]:
"""
Check which dashboard files have been updated since last check
@@ -297,7 +333,7 @@ class StorageService:
def initialize_empty_dashboard(self):
"""Initialize empty dashboard files with default values"""
# Summary
self.save_file(
self.save_export_file(
"summary",
{
"totalAssetValue": self.initial_cash,
@@ -315,10 +351,10 @@ class StorageService:
)
# Holdings
self.save_file("holdings", [])
self.save_export_file("holdings", [])
# Stats
self.save_file(
self.save_export_file(
"stats",
{
"totalAssetValue": self.initial_cash,
@@ -335,7 +371,7 @@ class StorageService:
)
# Trades
self.save_file("trades", [])
self.save_export_file("trades", [])
# Leaderboard with model info
self.generate_leaderboard()
@@ -375,7 +411,7 @@ class StorageService:
ranking_entries.append(entry)
leaderboard = team_entries + ranking_entries
self.save_file("leaderboard", leaderboard)
self.save_export_file("leaderboard", leaderboard)
logger.info("Leaderboard generated with model info")
def update_leaderboard_model_info(self):
@@ -398,7 +434,7 @@ class StorageService:
entry["modelName"] = model_name
entry["modelProvider"] = model_provider
self.save_file("leaderboard", existing)
self.save_export_file("leaderboard", existing)
logger.info("Leaderboard model info updated")
def get_current_timestamp_ms(self, date: str = None) -> int:
@@ -653,7 +689,7 @@ class StorageService:
"momentum": state.get("momentum_history", []),
}
self.save_file("summary", summary)
self.save_export_file("summary", summary)
def _generate_holdings(
self,
@@ -715,7 +751,7 @@ class StorageService:
# Sort by weight
holdings.sort(key=lambda x: abs(x["weight"]), reverse=True)
self.save_file("holdings", holdings)
self.save_export_file("holdings", holdings)
def _generate_stats(self, state: Dict[str, Any], net_value: float):
"""Generate stats.json"""
@@ -738,7 +774,7 @@ class StorageService:
},
}
self.save_file("stats", stats)
self.save_export_file("stats", stats)
def _generate_trades(self, state: Dict[str, Any]):
"""Generate trades.json"""
@@ -764,7 +800,7 @@ class StorageService:
},
)
self.save_file("trades", trades)
self.save_export_file("trades", trades)
# Server State Management Methods
@@ -1001,12 +1037,12 @@ class StorageService:
Args:
state: Server state dictionary to update
"""
# Load dashboard data
summary = self.load_file("summary") or {}
holdings = self.load_file("holdings") or []
stats = self.load_file("stats") or self._get_default_stats()
trades = self.load_file("trades") or []
leaderboard = self.load_file("leaderboard") or []
dashboard_snapshot = self.build_dashboard_snapshot_from_state(state)
summary = dashboard_snapshot.get("summary") or {}
holdings = dashboard_snapshot.get("holdings") or []
stats = dashboard_snapshot.get("stats") or self._get_default_stats()
trades = dashboard_snapshot.get("trades") or []
leaderboard = dashboard_snapshot.get("leaderboard") or []
internal_state = self.load_internal_state()
# Update state
@@ -1040,7 +1076,6 @@ class StorageService:
Start tracking live returns for current trading session.
Captures current values as session start baseline.
"""
summary = self.load_file("summary") or {}
state = self.load_internal_state()
# Capture current values as session start
@@ -1052,7 +1087,7 @@ class StorageService:
self._session_start_equity = (
equity_history[-1]["v"]
if equity_history
else summary.get("totalAssetValue", self.initial_cash)
else self.initial_cash
)
self._session_start_baseline = (
baseline_history[-1]["v"]

View File

@@ -0,0 +1,119 @@
# Skill Template (Anthropic + AgentScope Aligned)
> 用于定义可执行、可路由、可评估的技能规范。
> 建议所有 `SKILL.md` 至少覆盖以下 6 个部分。
---
## Frontmatter Spec
All `SKILL.md` files should begin with a YAML frontmatter block:
```yaml
---
name: skill_name # Required. Unique identifier for the skill.
description: ... # Required. One-line description of the skill.
version: "1.0.0" # Optional. Semantic version string.
tools: [...] # Optional. Tools provided or used by this skill.
allowed_tools: [...] # Optional. List of tool names permitted when this skill is active.
denied_tools: [...] # Optional. List of tool names denied when this skill is active.
---
```
### Frontmatter Fields
| Field | Type | Description |
|-------|------|-------------|
| `name` | string | Unique skill identifier (kebab-case recommended). |
| `description` | string | Human-readable one-line description. |
| `version` | string | Semantic version (e.g., `"1.0.0"`). |
| `tools` | list[string] | Tools provided by or associated with this skill. |
| `allowed_tools` | list[string] | Enumerates which tools are **permitted** when this skill is active. If set, only these tools may be used. |
| `denied_tools` | list[string] | Enumerates which tools are **forbidden** when this skill is active. Denied tools take precedence over `allowed_tools`. |
### Tool Restriction Rules
- If **only** `allowed_tools` is set: only those tools are accessible.
- If **only** `denied_tools` is set: all tools except those are accessible.
- If **both** are set: `allowed_tools` defines the initial set, then `denied_tools` removes from it.
- **Denial takes precedence**: a tool in `denied_tools` is always blocked even if also in `allowed_tools`.
---
## 1) When to use
- 明确触发条件(任务类型、关键词、场景)。
- 明确不应使用该技能的边界(避免误触发)。
## 2) Required inputs
- 列出最小必要输入(如 `tickers`、价格、组合状态、风险约束)。
- 声明输入缺失时的处理规则(终止 / 降级 / 请求补充)。
## 3) Decision procedure
- 采用固定步骤,确保可复现。
- 每一步说明目标、判据和产物(例如中间结论)。
- 标明冲突处理逻辑(信号冲突、数据冲突、置信度冲突)。
## 4) Tool call policy
- 说明优先使用哪些工具组与工具。
- 规定何时可以“无工具直接结论”,何时必须工具先证据后结论。
- 规定工具失败、超时、返回异常时的替代动作。
## 5) Output schema
- 定义标准输出字段,便于下游 Agent 消费与评估。
- 推荐包含:`signal``confidence``reasons``risks``invalidation``next_action`
- 若是组合决策技能,必须包含每个 ticker 的 `action``quantity`
## 6) Failure fallback
- 规定在数据不足、信号冲突、风险超限、工具不可用时的降级策略。
- 默认优先“保守 + 可解释 + 可执行”的输出。
## Optional: Evaluation hooks
定义技能的可评估指标,用于后续记忆/反思阶段写入长期经验。
### 支持的指标类型
| 指标类型 | 描述 | 适用技能 |
|---------|------|---------|
| `hit_rate` | 信号命中率 - 决策信号与实际结果的符合程度 | sentiment_review, technical_review |
| `risk_violation` | 风控违例率 - 触发风控规则的次数 | risk_review, portfolio_decisioning |
| `position_deviation` | 仓位偏离率 - 建议仓位与实际执行仓位的偏差 | portfolio_decisioning |
| `pnl_attribution` | P&L 归因一致性 - 收益归因与实际收益的匹配度 | fundamental_review, valuation_review |
| `signal_consistency` | 信号一致性 - 多来源信号的一致程度 | sentiment_review |
| `decision_latency` | 决策延迟 - 从输入到决策的耗时 | portfolio_decisioning |
| `tool_usage` | 工具使用率 - 工具调用次数与成功率的比值 | 所有技能 |
| `custom` | 自定义指标 | 特定业务场景 |
### 使用方式
```python
from backend.agents.base.evaluation_hook import EvaluationHook, MetricType
# 在技能执行开始时
evaluation_hook.start_evaluation(
skill_name="technical_review",
inputs={"tickers": ["AAPL"], "prices": {...}}
)
# 在技能执行过程中添加指标
evaluation_hook.add_metric(
name="signal_confidence",
metric_type=MetricType.HIT_RATE,
value=0.85,
metadata={"method": "rsi", "threshold": 30}
)
# 在技能完成时记录结果
evaluation_hook.record_outputs({"signal": "buy", "confidence": 0.8})
evaluation_hook.complete_evaluation(success=True)
```
### 评估结果存储
评估结果自动保存到 `runs/{run_id}/evaluations/{agent_id}/{skill_name}_{timestamp}.json`

View File

@@ -0,0 +1,104 @@
# -*- coding: utf-8 -*-
"""Tests for the extracted agent service surface."""
from pathlib import Path
from fastapi.testclient import TestClient
from backend.apps.agent_service import create_app
from backend.api import agents as agents_module
def test_agent_service_routes_include_control_plane_endpoints(tmp_path):
app = create_app(project_root=tmp_path)
paths = {route.path for route in app.routes}
assert "/health" in paths
assert "/api/status" in paths
assert "/api/workspaces" in paths
assert "/api/guard/pending" in paths
def test_agent_service_excludes_runtime_routes(tmp_path):
app = create_app(project_root=tmp_path)
paths = {route.path for route in app.routes}
assert "/api/runtime/start" not in paths
assert "/api/runtime/gateway/port" not in paths
def test_agent_service_read_routes(monkeypatch, tmp_path):
class _FakeSkillsManager:
project_root = tmp_path
def get_agent_asset_dir(self, config_name, agent_id):
return tmp_path / "runs" / config_name / "agents" / agent_id
def resolve_agent_skill_names(self, config_name, agent_id, default_skills=None):
return ["demo_skill"]
def list_agent_skill_catalog(self, config_name, agent_id):
return [
type(
"Skill",
(),
{
"skill_name": "demo_skill",
"name": "Demo Skill",
"description": "demo",
"version": "1.0.0",
"source": "builtin",
"tools": [],
},
)()
]
def load_agent_skill_document(self, config_name, agent_id, skill_name):
return {"skill_name": skill_name, "content": "# demo"}
class _FakeWorkspaceManager:
def load_agent_file(self, config_name, agent_id, filename):
return f"{config_name}:{agent_id}:{filename}"
monkeypatch.setattr(agents_module, "load_agent_profiles", lambda: {"portfolio_manager": {"skills": ["demo_skill"]}})
monkeypatch.setattr(agents_module, "get_agent_model_info", lambda agent_id: ("deepseek-v3.2", "DASHSCOPE"))
monkeypatch.setattr(
agents_module,
"load_agent_workspace_config",
lambda path: type(
"Cfg",
(),
{
"active_tool_groups": ["portfolio_ops"],
"disabled_tool_groups": [],
"enabled_skills": [],
"disabled_skills": [],
"prompt_files": ["SOUL.md", "MEMORY.md"],
},
)(),
)
monkeypatch.setattr(
agents_module,
"get_bootstrap_config_for_run",
lambda project_root, config_name: type("Bootstrap", (), {"agent_override": lambda self, agent_id: {}})(),
)
app = create_app(project_root=tmp_path)
app.dependency_overrides[agents_module.get_skills_manager] = lambda: _FakeSkillsManager()
app.dependency_overrides[agents_module.get_workspace_manager] = lambda: _FakeWorkspaceManager()
with TestClient(app) as client:
profile = client.get("/api/workspaces/demo/agents/portfolio_manager/profile")
skills = client.get("/api/workspaces/demo/agents/portfolio_manager/skills")
detail = client.get("/api/workspaces/demo/agents/portfolio_manager/skills/demo_skill")
workspace_file = client.get("/api/workspaces/demo/agents/portfolio_manager/files/MEMORY.md")
assert profile.status_code == 200
assert profile.json()["profile"]["model_name"] == "deepseek-v3.2"
assert skills.status_code == 200
assert skills.json()["skills"][0]["skill_name"] == "demo_skill"
assert detail.status_code == 200
assert detail.json()["skill"]["content"] == "# demo"
assert workspace_file.status_code == 200
assert workspace_file.json()["content"] == "demo:portfolio_manager:MEMORY.md"

View File

@@ -13,7 +13,7 @@ class _DummyToolkit:
return ""
def test_workspace_manager_creates_extended_agent_files(tmp_path):
def test_workspace_manager_creates_core_agent_files(tmp_path):
manager = WorkspaceManager(project_root=tmp_path)
manager.initialize_default_assets(
@@ -27,7 +27,7 @@ def test_workspace_manager_creates_extended_agent_files(tmp_path):
assert (asset_dir / "PROFILE.md").exists()
assert (asset_dir / "AGENTS.md").exists()
assert (asset_dir / "MEMORY.md").exists()
assert (asset_dir / "HEARTBEAT.md").exists()
assert (asset_dir / "POLICY.md").exists()
assert (asset_dir / "agent.yaml").exists()
assert (asset_dir / "skills" / "installed").is_dir()
assert (asset_dir / "skills" / "active").is_dir()
@@ -35,6 +35,22 @@ def test_workspace_manager_creates_extended_agent_files(tmp_path):
assert (asset_dir / "skills" / "local").is_dir()
def test_workspace_manager_seeds_risk_prompt_content(tmp_path):
manager = WorkspaceManager(project_root=tmp_path)
manager.initialize_default_assets(
config_name="demo",
agent_ids=["risk_manager"],
analyst_personas={},
)
asset_dir = tmp_path / "runs" / "demo" / "agents" / "risk_manager"
soul = (asset_dir / "SOUL.md").read_text(encoding="utf-8")
guide = (asset_dir / "AGENTS.md").read_text(encoding="utf-8")
assert "风险管理经理" in soul
assert "优先使用可用的风险工具量化集中度" in guide
def test_agent_workspace_config_controls_prompt_files(tmp_path, monkeypatch):
manager = WorkspaceManager(project_root=tmp_path)
manager.initialize_default_assets(
@@ -72,6 +88,32 @@ def test_agent_workspace_config_controls_prompt_files(tmp_path, monkeypatch):
assert "profile-line" not in prompt
def test_prompt_is_built_from_workspace_defaults_without_system_templates(tmp_path, monkeypatch):
manager = WorkspaceManager(project_root=tmp_path)
manager.initialize_default_assets(
config_name="demo",
agent_ids=["portfolio_manager"],
analyst_personas={},
)
from backend.agents import prompt_factory
monkeypatch.setattr(
prompt_factory,
"SkillsManager",
lambda: SkillsManager(project_root=tmp_path),
)
prompt = build_agent_system_prompt(
agent_id="portfolio_manager",
config_name="demo",
toolkit=_DummyToolkit(),
)
assert "投资组合经理" in prompt
assert "使用 `make_decision` 工具记录每个股票的最终决策" in prompt
def test_skills_manager_applies_agent_level_skill_toggles(tmp_path):
builtin_root = tmp_path / "backend" / "skills" / "builtin"
for skill_name in ("risk_review", "extra_guard"):

View File

@@ -311,6 +311,17 @@ class TestRiskAgent:
class TestStorageService:
def test_storage_service_defaults_to_live_config(self):
from backend.services.storage import StorageService
with tempfile.TemporaryDirectory() as tmpdir:
storage = StorageService(
dashboard_dir=Path(tmpdir),
initial_cash=100000.0,
)
assert storage.config_name == "live"
def test_calculate_portfolio_value_cash_only(self):
from backend.services.storage import StorageService

View File

@@ -34,7 +34,6 @@ def test_live_runs_incremental_market_store_update_before_start(monkeypatch, tmp
monkeypatch.setattr(cli.subprocess, "run", fake_run)
cli.live(
mock=False,
config_name="smoke_fullstack",
host="0.0.0.0",
port=8765,

View File

@@ -0,0 +1,139 @@
# -*- coding: utf-8 -*-
"""Tests for data_tools preferring split services when configured."""
from backend.tools import data_tools
from shared.schema import CompanyNews, FinancialMetrics, InsiderTrade, LineItem, Price
def test_data_tools_prefers_trading_service(monkeypatch):
monkeypatch.setenv("TRADING_SERVICE_URL", "http://localhost:8001")
monkeypatch.setenv("SERVICE_NAME", "agent_service")
monkeypatch.setattr(data_tools._cache, "get_prices", lambda key: None)
monkeypatch.setattr(data_tools._cache, "get_financial_metrics", lambda key: None)
monkeypatch.setattr(data_tools._cache, "get_insider_trades", lambda key: None)
monkeypatch.setattr(data_tools._cache, "get_company_news", lambda key: None)
def fake_service_get_json(base_url, path, *, params):
if path == "/api/prices":
return {
"ticker": "AAPL",
"prices": [
Price(
open=1,
close=2,
high=3,
low=1,
volume=10,
time="2026-03-16",
).model_dump()
],
}
if path == "/api/financials":
return {
"financial_metrics": [
FinancialMetrics(
ticker="AAPL",
report_period="2026-03-16",
period="ttm",
currency="USD",
market_cap=123.0,
enterprise_value=None,
price_to_earnings_ratio=None,
price_to_book_ratio=None,
price_to_sales_ratio=None,
enterprise_value_to_ebitda_ratio=None,
enterprise_value_to_revenue_ratio=None,
free_cash_flow_yield=None,
peg_ratio=None,
gross_margin=None,
operating_margin=None,
net_margin=None,
return_on_equity=None,
return_on_assets=None,
return_on_invested_capital=None,
asset_turnover=None,
inventory_turnover=None,
receivables_turnover=None,
days_sales_outstanding=None,
operating_cycle=None,
working_capital_turnover=None,
current_ratio=None,
quick_ratio=None,
cash_ratio=None,
operating_cash_flow_ratio=None,
debt_to_equity=None,
debt_to_assets=None,
interest_coverage=None,
revenue_growth=None,
earnings_growth=None,
book_value_growth=None,
earnings_per_share_growth=None,
free_cash_flow_growth=None,
operating_income_growth=None,
ebitda_growth=None,
payout_ratio=None,
earnings_per_share=None,
book_value_per_share=None,
free_cash_flow_per_share=None,
).model_dump()
]
}
if path == "/api/insider-trades":
return {
"insider_trades": [
InsiderTrade(ticker="AAPL", filing_date="2026-03-16").model_dump()
]
}
if path == "/api/news":
return {
"news": [
CompanyNews(
ticker="AAPL",
title="Title",
source="polygon",
url="https://example.com",
).model_dump()
]
}
if path == "/api/market-cap":
return {"ticker": "AAPL", "end_date": "2026-03-16", "market_cap": 2.5e12}
if path == "/api/line-items":
return {
"search_results": [
LineItem(
ticker="AAPL",
report_period="2026-03-16",
period="ttm",
currency="USD",
free_cash_flow=321.0,
).model_dump()
]
}
raise AssertionError(path)
monkeypatch.setattr(data_tools, "_service_get_json", fake_service_get_json)
prices = data_tools.get_prices("AAPL", "2026-03-01", "2026-03-16")
metrics = data_tools.get_financial_metrics("AAPL", "2026-03-16")
trades = data_tools.get_insider_trades("AAPL", "2026-03-16")
news = data_tools.get_company_news("AAPL", "2026-03-16")
market_cap = data_tools.get_market_cap("AAPL", "2026-03-16")
line_items = data_tools.search_line_items(
"AAPL",
["free_cash_flow"],
"2026-03-16",
)
assert prices[0].close == 2
assert metrics[0].ticker == "AAPL"
assert trades[0].ticker == "AAPL"
assert news[0].ticker == "AAPL"
assert market_cap == 2.5e12
assert line_items[0].free_cash_flow == 321.0
def test_data_tools_skips_self_recursion_for_trading_service(monkeypatch):
monkeypatch.setenv("TRADING_SERVICE_URL", "http://localhost:8001")
monkeypatch.setenv("SERVICE_NAME", "trading_service")
assert data_tools._trading_service_url() is None

View File

@@ -6,6 +6,7 @@ import pytest
from backend.services.gateway import Gateway
import backend.services.gateway as gateway_module
from shared.schema import InsiderTrade, InsiderTradeResponse, Price, PriceResponse
class DummyWebSocket:
@@ -35,6 +36,10 @@ class FakeMarketStore:
def __init__(self):
self.calls = []
def get_ticker_watermarks(self, symbol):
self.calls.append(("get_ticker_watermarks", symbol))
return {"symbol": symbol, "last_news_fetch": "2026-12-31"}
def get_news_timeline_enriched(self, symbol, *, start_date=None, end_date=None):
self.calls.append(("get_news_timeline_enriched", symbol, start_date, end_date))
return [{"date": end_date, "count": 2, "source_count": 1, "top_title": "Top", "positive_count": 1}]
@@ -123,6 +128,75 @@ def make_gateway(market_store=None):
)
class FakeNewsClient:
def __init__(self, base_url):
self.base_url = base_url
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc_val, exc_tb):
return None
async def get_categories(self, ticker, start_date=None, end_date=None, limit=200):
return {"ticker": ticker, "categories": {"remote": {"count": 2}}}
async def get_enriched_news(self, ticker, start_date=None, end_date=None, limit=None):
return {
"ticker": ticker,
"news": [
{
"id": "remote-news-1",
"ticker": ticker,
"title": "Remote Title",
"date": end_date,
}
],
}
async def get_story(self, ticker, as_of_date):
return {"symbol": ticker, "as_of_date": as_of_date, "story": "remote story", "source": "news_service"}
class FakeTradingClient:
def __init__(self, base_url):
self.base_url = base_url
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc_val, exc_tb):
return None
async def get_insider_trades(self, ticker, end_date=None, start_date=None, limit=None):
return InsiderTradeResponse(
insider_trades=[
InsiderTrade(
ticker=ticker,
name="Remote Insider",
filing_date=end_date or "2026-03-16",
)
]
)
async def get_prices(self, ticker, start_date=None, end_date=None):
prices = [
Price(
open=float(100 + idx),
close=float(101 + idx),
high=float(102 + idx),
low=float(99 + idx),
volume=1000 + idx,
time=f"2026-01-{idx + 1:02d}",
)
for idx in range(30)
]
return PriceResponse(ticker=ticker, prices=prices)
async def get_market_cap(self, ticker, end_date):
return {"ticker": ticker, "end_date": end_date, "market_cap": 2.5e12}
@pytest.mark.asyncio
async def test_handle_get_stock_news_timeline_uses_market_store_symbol_argument():
market_store = FakeMarketStore()
@@ -135,6 +209,7 @@ async def test_handle_get_stock_news_timeline_uses_market_store_symbol_argument(
)
assert market_store.calls == [
("get_ticker_watermarks", "AAPL"),
("get_news_timeline_enriched", "AAPL", "2026-02-14", "2026-03-16")
]
assert websocket.messages[-1]["type"] == "stock_news_timeline_loaded"
@@ -153,6 +228,7 @@ async def test_handle_get_stock_news_categories_uses_market_store_symbol_argumen
)
assert market_store.calls == [
("get_ticker_watermarks", "AAPL"),
("get_news_items_enriched", "AAPL", "2026-02-14", "2026-03-16", None, 200),
("get_news_categories_enriched", "AAPL", "2026-02-14", "2026-03-16", 200)
]
@@ -175,7 +251,7 @@ async def test_handle_get_stock_range_explain_uses_market_store_rows(monkeypatch
}
monkeypatch.setattr(
gateway_module,
gateway_module.news_domain,
"build_range_explanation",
fake_build_range_explanation,
)
@@ -186,6 +262,7 @@ async def test_handle_get_stock_range_explain_uses_market_store_rows(monkeypatch
)
assert market_store.calls == [
("get_ticker_watermarks", "AAPL"),
("get_news_items_enriched", "AAPL", "2026-03-10", "2026-03-16", None, 100)
]
assert websocket.messages[-1] == {
@@ -207,7 +284,7 @@ async def test_handle_get_stock_range_explain_uses_article_ids_path(monkeypatch)
websocket = DummyWebSocket()
monkeypatch.setattr(
gateway_module,
gateway_module.news_domain,
"build_range_explanation",
lambda **kwargs: {"news_count": len(kwargs["news_rows"])},
)
@@ -222,7 +299,10 @@ async def test_handle_get_stock_range_explain_uses_article_ids_path(monkeypatch)
},
)
assert market_store.calls == [("get_news_by_ids_enriched", "AAPL", ["news-99"])]
assert market_store.calls == [
("get_ticker_watermarks", "AAPL"),
("get_news_by_ids_enriched", "AAPL", ["news-99"])
]
assert websocket.messages[-1]["result"]["news_count"] == 1
@@ -238,6 +318,7 @@ async def test_handle_get_stock_news_for_date_uses_trade_date_lookup():
)
assert market_store.calls == [
("get_ticker_watermarks", "AAPL"),
("get_news_items_enriched", "AAPL", None, None, "2026-03-16", 10)
]
assert websocket.messages[-1]["type"] == "stock_news_for_date_loaded"
@@ -251,7 +332,7 @@ async def test_handle_get_stock_story_returns_story_payload(monkeypatch):
websocket = DummyWebSocket()
monkeypatch.setattr(
gateway_module,
gateway_module.news_domain,
"enrich_news_for_symbol",
lambda *args, **kwargs: {"symbol": "AAPL", "analyzed": 3},
)
@@ -266,6 +347,132 @@ async def test_handle_get_stock_story_returns_story_payload(monkeypatch):
assert "AAPL Story" in websocket.messages[-1]["story"]
@pytest.mark.asyncio
async def test_handle_get_stock_news_categories_uses_news_service_client_when_configured(monkeypatch):
market_store = FakeMarketStore()
gateway = make_gateway(market_store)
websocket = DummyWebSocket()
monkeypatch.setenv("NEWS_SERVICE_URL", "http://news-service.local")
monkeypatch.setattr(gateway_module, "NewsServiceClient", FakeNewsClient)
await gateway._handle_get_stock_news_categories(
websocket,
{"ticker": "AAPL", "lookback_days": 30},
)
assert market_store.calls == []
assert websocket.messages[-1]["type"] == "stock_news_categories_loaded"
assert websocket.messages[-1]["categories"]["remote"]["count"] == 2
@pytest.mark.asyncio
async def test_handle_get_stock_story_uses_news_service_client_when_configured(monkeypatch):
market_store = FakeMarketStore()
gateway = make_gateway(market_store)
websocket = DummyWebSocket()
monkeypatch.setenv("NEWS_SERVICE_URL", "http://news-service.local")
monkeypatch.setattr(gateway_module, "NewsServiceClient", FakeNewsClient)
await gateway._handle_get_stock_story(
websocket,
{"ticker": "AAPL", "as_of_date": "2026-03-16"},
)
assert market_store.calls == []
assert websocket.messages[-1]["type"] == "stock_story_loaded"
assert websocket.messages[-1]["story"] == "remote story"
@pytest.mark.asyncio
async def test_handle_get_stock_news_uses_news_service_client_when_configured(monkeypatch):
market_store = FakeMarketStore()
gateway = make_gateway(market_store)
websocket = DummyWebSocket()
monkeypatch.setenv("NEWS_SERVICE_URL", "http://news-service.local")
monkeypatch.setattr(gateway_module, "NewsServiceClient", FakeNewsClient)
await gateway._handle_get_stock_news(
websocket,
{"ticker": "AAPL", "lookback_days": 30, "limit": 5},
)
assert market_store.calls == []
assert websocket.messages[-1]["type"] == "stock_news_loaded"
assert websocket.messages[-1]["source"] == "news_service"
assert websocket.messages[-1]["news"][0]["title"] == "Remote Title"
@pytest.mark.asyncio
async def test_handle_get_stock_insider_trades_uses_trading_service_client_when_configured(monkeypatch):
market_store = FakeMarketStore()
gateway = make_gateway(market_store)
websocket = DummyWebSocket()
monkeypatch.setenv("TRADING_SERVICE_URL", "http://trading-service.local")
monkeypatch.setattr(gateway_module, "TradingServiceClient", FakeTradingClient)
await gateway._handle_get_stock_insider_trades(
websocket,
{"ticker": "AAPL", "end_date": "2026-03-16", "limit": 10},
)
assert websocket.messages[-1]["type"] == "stock_insider_trades_loaded"
assert websocket.messages[-1]["trades"][0]["name"] == "Remote Insider"
@pytest.mark.asyncio
async def test_handle_get_stock_history_uses_trading_service_client_when_configured(monkeypatch):
market_store = FakeMarketStore()
gateway = make_gateway(market_store)
websocket = DummyWebSocket()
monkeypatch.setenv("TRADING_SERVICE_URL", "http://trading-service.local")
monkeypatch.setattr(gateway_module, "TradingServiceClient", FakeTradingClient)
await gateway._handle_get_stock_history(
websocket,
{"ticker": "AAPL", "lookback_days": 30},
)
assert market_store.calls == []
assert websocket.messages[-1]["type"] == "stock_history_loaded"
assert websocket.messages[-1]["source"] == "trading_service"
assert len(websocket.messages[-1]["prices"]) == 30
@pytest.mark.asyncio
async def test_handle_get_stock_technical_indicators_uses_trading_service_client_when_configured(monkeypatch):
gateway = make_gateway(FakeMarketStore())
websocket = DummyWebSocket()
monkeypatch.setenv("TRADING_SERVICE_URL", "http://trading-service.local")
monkeypatch.setattr(gateway_module, "TradingServiceClient", FakeTradingClient)
await gateway._handle_get_stock_technical_indicators(
websocket,
{"ticker": "AAPL"},
)
assert websocket.messages[-1]["type"] == "stock_technical_indicators_loaded"
assert websocket.messages[-1]["ticker"] == "AAPL"
assert websocket.messages[-1]["indicators"] is not None
@pytest.mark.asyncio
async def test_get_market_caps_uses_trading_service_client_when_configured(monkeypatch):
gateway = make_gateway(FakeMarketStore())
monkeypatch.setenv("TRADING_SERVICE_URL", "http://trading-service.local")
monkeypatch.setattr(gateway_module, "TradingServiceClient", FakeTradingClient)
market_caps = await gateway._get_market_caps(["AAPL", "MSFT"], "2026-03-16")
assert market_caps == {"AAPL": 2.5e12, "MSFT": 2.5e12}
@pytest.mark.asyncio
async def test_handle_get_stock_similar_days_returns_items(monkeypatch):
market_store = FakeMarketStore()
@@ -273,7 +480,7 @@ async def test_handle_get_stock_similar_days_returns_items(monkeypatch):
websocket = DummyWebSocket()
monkeypatch.setattr(
gateway_module,
gateway_module.news_domain,
"enrich_news_for_symbol",
lambda *args, **kwargs: {"symbol": "AAPL", "analyzed": 3},
)
@@ -295,7 +502,12 @@ async def test_handle_run_stock_enrich_rebuilds_caches(monkeypatch):
websocket = DummyWebSocket()
monkeypatch.setattr(
gateway_module,
gateway_module.gateway_stock_handlers,
"enrich_news_for_symbol",
lambda *args, **kwargs: {"symbol": "AAPL", "analyzed": 2, "queued_count": 2},
)
monkeypatch.setattr(
gateway_module.news_domain,
"enrich_news_for_symbol",
lambda *args, **kwargs: {"symbol": "AAPL", "analyzed": 2, "queued_count": 2},
)
@@ -325,7 +537,7 @@ async def test_handle_run_stock_enrich_rejects_local_to_llm_without_llm(monkeypa
gateway = make_gateway(FakeMarketStore())
websocket = DummyWebSocket()
monkeypatch.setattr(gateway_module, "llm_enrichment_enabled", lambda: False)
monkeypatch.setattr(gateway_module.gateway_stock_handlers, "llm_enrichment_enabled", lambda: False)
await gateway._handle_run_stock_enrich(
websocket,
@@ -361,7 +573,7 @@ def test_schedule_watchlist_market_store_refresh_creates_task(monkeypatch):
gateway._schedule_watchlist_market_store_refresh(["AAPL", "MSFT"])
assert captured["coro_name"] == "_refresh_market_store_for_watchlist"
assert captured["coro_name"] == "refresh_market_store_for_watchlist"
@pytest.mark.asyncio
@@ -369,7 +581,7 @@ async def test_refresh_market_store_for_watchlist_emits_system_messages(monkeypa
gateway = make_gateway()
monkeypatch.setattr(
gateway_module,
gateway_module.gateway_cycle_support,
"ingest_symbols",
lambda symbols, mode="incremental": [
{"symbol": symbol, "prices": 3, "news": 4, "aligned": 4}
@@ -445,12 +657,12 @@ async def test_handle_get_agent_profile_returns_model_and_tool_groups(monkeypatc
websocket = DummyWebSocket()
monkeypatch.setattr(
gateway_module,
gateway_module.gateway_admin_handlers,
"load_agent_profiles",
lambda: {"risk_manager": {"skills": ["risk_review"], "active_tool_groups": ["risk_ops", "legacy_group"]}},
)
monkeypatch.setattr(
gateway_module,
gateway_module.gateway_admin_handlers,
"get_agent_model_info",
lambda agent_id: ("gpt-4o-mini", "OPENAI"),
)
@@ -461,7 +673,7 @@ async def test_handle_get_agent_profile_returns_model_and_tool_groups(monkeypatc
return {}
monkeypatch.setattr(
gateway_module,
gateway_module.gateway_admin_handlers,
"get_bootstrap_config_for_run",
lambda project_root, config_name: _Bootstrap(),
)

View File

@@ -0,0 +1,201 @@
# -*- coding: utf-8 -*-
"""Direct tests for Gateway support modules."""
from types import SimpleNamespace
import pytest
from backend.services import gateway_cycle_support, gateway_runtime_support
class _DummyScheduler:
def __init__(self):
self.calls = []
def reconfigure(self, **kwargs):
self.calls.append(kwargs)
class _DummyStateSync:
def __init__(self):
self.updated = []
self.saved = False
self.system_messages = []
self.backtest_dates = []
self.state = {}
def update_state(self, key, value):
self.updated.append((key, value))
self.state[key] = value
def save_state(self):
self.saved = True
async def on_system_message(self, message):
self.system_messages.append(message)
def set_backtest_dates(self, dates):
self.backtest_dates = list(dates)
class _DummyStorage:
def __init__(self):
self.initial_cash = 100000.0
self.is_live_session_active = False
self.server_state_updates = []
def can_apply_initial_cash(self):
return True
def apply_initial_cash(self, value):
self.initial_cash = value
return True
def update_server_state_from_dashboard(self, state):
self.server_state_updates.append(state)
def load_file(self, name):
if name == "summary":
return {"totalAssetValue": self.initial_cash}
return []
def build_dashboard_snapshot_from_state(self, state):
return {
"summary": {"totalAssetValue": self.initial_cash},
"holdings": [],
"stats": {},
"trades": [],
"leaderboard": [],
}
class _DummyPM:
def __init__(self):
self.portfolio = {"margin_requirement": 0.0}
def apply_runtime_portfolio_config(self, margin_requirement=None, initial_cash=None):
if margin_requirement is not None:
self.portfolio["margin_requirement"] = margin_requirement
return {"margin_requirement": True}
def can_apply_initial_cash(self):
return True
class _DummyMarketService:
def __init__(self):
self.updated = None
self.stopped = False
def update_tickers(self, tickers):
self.updated = list(tickers)
return {"active": list(tickers), "added": list(tickers), "removed": []}
def stop(self):
self.stopped = True
def make_gateway_stub():
pipeline = SimpleNamespace(max_comm_cycles=0, pm=_DummyPM())
gateway = SimpleNamespace(
market_service=_DummyMarketService(),
pipeline=pipeline,
scheduler=_DummyScheduler(),
config={
"tickers": ["AAPL"],
"schedule_mode": "daily",
"interval_minutes": 60,
"trigger_time": "09:30",
"enable_memory": False,
},
storage=_DummyStorage(),
state_sync=_DummyStateSync(),
_watchlist_ingest_task=None,
_market_status_task=None,
_backtest_task=None,
_backtest_start_date=None,
_backtest_end_date=None,
_manual_cycle_task=None,
)
return gateway
def test_normalize_watchlist_filters_invalid_and_dedupes():
assert gateway_runtime_support.normalize_watchlist(["aapl", " AAPL ", "", "msft"]) == ["AAPL", "MSFT"]
assert gateway_runtime_support.normalize_watchlist("aapl,msft") == ["AAPL", "MSFT"]
def test_normalize_agent_workspace_filename_obeys_allowlist():
allowlist = {"SOUL.md", "PROFILE.md"}
assert gateway_runtime_support.normalize_agent_workspace_filename("SOUL.md", allowlist=allowlist) == "SOUL.md"
assert gateway_runtime_support.normalize_agent_workspace_filename("README.md", allowlist=allowlist) is None
def test_apply_runtime_config_updates_gateway_state():
gateway = make_gateway_stub()
result = gateway_runtime_support.apply_runtime_config(
gateway,
{
"tickers": ["MSFT", "NVDA"],
"schedule_mode": "intraday",
"interval_minutes": 30,
"trigger_time": "10:30",
"initial_cash": 150000.0,
"margin_requirement": 0.5,
"max_comm_cycles": 4,
"enable_memory": False,
},
)
assert gateway.config["tickers"] == ["MSFT", "NVDA"]
assert gateway.config["schedule_mode"] == "intraday"
assert gateway.storage.initial_cash == 150000.0
assert result["runtime_config_applied"]["max_comm_cycles"] == 4
assert gateway.scheduler.calls[-1] == {
"mode": "intraday",
"trigger_time": "10:30",
"interval_minutes": 30,
}
def test_schedule_watchlist_market_store_refresh_creates_task(monkeypatch):
gateway = make_gateway_stub()
captured = {}
class DummyTask:
def done(self):
return False
def cancel(self):
captured["cancelled"] = True
def fake_create_task(coro):
captured["name"] = coro.cr_code.co_name
coro.close()
return DummyTask()
monkeypatch.setattr(gateway_cycle_support.asyncio, "create_task", fake_create_task)
gateway_cycle_support.schedule_watchlist_market_store_refresh(gateway, ["AAPL", "MSFT"])
assert captured["name"] == "refresh_market_store_for_watchlist"
@pytest.mark.asyncio
async def test_refresh_market_store_for_watchlist_emits_system_messages(monkeypatch):
gateway = make_gateway_stub()
monkeypatch.setattr(
gateway_cycle_support,
"ingest_symbols",
lambda symbols, mode="incremental": [
{"symbol": symbol, "prices": 3, "news": 4}
for symbol in symbols
],
)
await gateway_cycle_support.refresh_market_store_for_watchlist(gateway, ["AAPL", "MSFT"])
assert gateway.state_sync.system_messages[0] == "正在同步自选股市场数据: AAPL, MSFT"
assert "自选股市场数据已同步:" in gateway.state_sync.system_messages[1]

View File

@@ -0,0 +1,81 @@
# -*- coding: utf-8 -*-
"""Tests for market ingest watermark handling."""
from backend.data import market_ingest
class _FakeStore:
def __init__(self, *, last_news_fetch=None, latest_news_date=None):
self._watermarks = {
"symbol": "AAPL",
"last_price_fetch": None,
"last_news_fetch": last_news_fetch,
}
self._latest_news_date = latest_news_date
self.updated = []
def get_ticker_watermarks(self, symbol):
return dict(self._watermarks)
def get_latest_news_date(self, symbol):
return self._latest_news_date
def upsert_ticker(self, **kwargs):
return None
def upsert_ohlc(self, symbol, rows, source="polygon"):
return len(rows)
def upsert_news(self, symbol, rows, source="polygon"):
return len(rows)
def update_fetch_watermark(self, **kwargs):
self.updated.append(kwargs)
def test_refresh_news_incremental_does_not_advance_watermark_without_news(monkeypatch):
store = _FakeStore(last_news_fetch="2026-03-28", latest_news_date="2026-03-28")
monkeypatch.setattr(market_ingest, "fetch_ticker_details", lambda ticker: {"name": ticker, "sic_description": None, "active": True})
class _Router:
def get_company_news(self, **kwargs):
return [], "polygon"
monkeypatch.setattr(market_ingest, "DataProviderRouter", lambda: _Router())
monkeypatch.setattr(market_ingest, "align_news_for_symbol", lambda store, ticker: 0)
result = market_ingest.refresh_news_incremental(
"AAPL",
end_date="2026-03-29",
store=store,
)
assert result["start_news_date"] == "2026-03-29"
assert result["news"] == 0
assert store.updated[-1]["news_date"] is None
def test_refresh_news_incremental_clamps_future_watermark_to_latest_stored_date(monkeypatch):
store = _FakeStore(last_news_fetch="2026-03-30", latest_news_date="2026-03-28")
captured = {}
monkeypatch.setattr(market_ingest, "fetch_ticker_details", lambda ticker: {"name": ticker, "sic_description": None, "active": True})
class _Router:
def get_company_news(self, **kwargs):
captured.update(kwargs)
return [], "polygon"
monkeypatch.setattr(market_ingest, "DataProviderRouter", lambda: _Router())
monkeypatch.setattr(market_ingest, "align_news_for_symbol", lambda store, ticker: 0)
result = market_ingest.refresh_news_incremental(
"AAPL",
end_date="2026-03-29",
store=store,
)
assert result["start_news_date"] == "2026-03-29"
assert captured["start_date"] == "2026-03-29"
assert captured["end_date"] == "2026-03-29"

View File

@@ -2,153 +2,12 @@
# pylint: disable=W0212
import asyncio
import time
import logging
from unittest.mock import MagicMock, AsyncMock, patch
import pytest
from backend.services.market import MarketService
from backend.data.mock_price_manager import MockPriceManager
from backend.data.polling_price_manager import PollingPriceManager
class TestMockPriceManager:
def test_init_default(self):
manager = MockPriceManager()
assert manager.poll_interval == 10
assert manager.volatility == 0.5
assert manager.running is False
assert len(manager.subscribed_symbols) == 0
def test_init_custom(self):
manager = MockPriceManager(poll_interval=5, volatility=1.0)
assert manager.poll_interval == 5
assert manager.volatility == 1.0
def test_subscribe(self):
manager = MockPriceManager()
manager.subscribe(["AAPL", "MSFT"])
assert "AAPL" in manager.subscribed_symbols
assert "MSFT" in manager.subscribed_symbols
assert manager.base_prices["AAPL"] == 237.50 # default price
assert manager.base_prices["MSFT"] == 425.30 # default price
def test_subscribe_with_base_prices(self):
manager = MockPriceManager()
manager.subscribe(["AAPL"], base_prices={"AAPL": 100.0})
assert manager.base_prices["AAPL"] == 100.0
assert manager.open_prices["AAPL"] == 100.0
assert manager.latest_prices["AAPL"] == 100.0
def test_subscribe_unknown_symbol(self):
manager = MockPriceManager()
manager.subscribe(["UNKNOWN"])
assert "UNKNOWN" in manager.subscribed_symbols
assert manager.base_prices["UNKNOWN"] > 0 # random price generated
def test_unsubscribe(self):
manager = MockPriceManager()
manager.subscribe(["AAPL", "MSFT"])
manager.unsubscribe(["AAPL"])
assert "AAPL" not in manager.subscribed_symbols
assert "MSFT" in manager.subscribed_symbols
def test_add_price_callback(self):
manager = MockPriceManager()
callback = MagicMock()
manager.add_price_callback(callback)
assert callback in manager.price_callbacks
def test_generate_price_update_within_bounds(self):
manager = MockPriceManager(volatility=0.5)
manager.subscribe(["AAPL"], base_prices={"AAPL": 100.0})
for _ in range(100):
new_price = manager._generate_price_update("AAPL")
# Should be within +/-10% of open
assert 90.0 <= new_price <= 110.0
def test_update_prices_triggers_callback(self):
manager = MockPriceManager()
manager.subscribe(["AAPL"], base_prices={"AAPL": 100.0})
callback = MagicMock()
manager.add_price_callback(callback)
manager._update_prices()
callback.assert_called_once()
call_args = callback.call_args[0][0]
assert call_args["symbol"] == "AAPL"
assert "price" in call_args
assert "timestamp" in call_args
def test_start_stop(self):
manager = MockPriceManager(poll_interval=1)
manager.subscribe(["AAPL"], base_prices={"AAPL": 100.0})
manager.start()
assert manager.running is True
time.sleep(0.1) # let thread start
manager.stop()
assert manager.running is False
def test_start_without_subscription(self):
manager = MockPriceManager()
manager.start()
assert (
manager.running is False
) # should not start without subscriptions
def test_get_latest_price(self):
manager = MockPriceManager()
manager.subscribe(["AAPL"], base_prices={"AAPL": 100.0})
price = manager.get_latest_price("AAPL")
assert price == 100.0
def test_get_latest_price_unknown(self):
manager = MockPriceManager()
price = manager.get_latest_price("UNKNOWN")
assert price is None
def test_get_all_latest_prices(self):
manager = MockPriceManager()
manager.subscribe(
["AAPL", "MSFT"],
base_prices={"AAPL": 100.0, "MSFT": 200.0},
)
prices = manager.get_all_latest_prices()
assert prices["AAPL"] == 100.0
assert prices["MSFT"] == 200.0
def test_reset_open_prices(self):
manager = MockPriceManager()
manager.subscribe(["AAPL"], base_prices={"AAPL": 100.0})
manager.latest_prices["AAPL"] = 105.0
manager.reset_open_prices()
# Open price should change (based on latest with small gap)
assert manager.open_prices["AAPL"] != 100.0
def test_set_base_price(self):
manager = MockPriceManager()
manager.subscribe(["AAPL"], base_prices={"AAPL": 100.0})
manager.set_base_price("AAPL", 150.0)
assert manager.base_prices["AAPL"] == 150.0
assert manager.open_prices["AAPL"] == 150.0
assert manager.latest_prices["AAPL"] == 150.0
from backend.llm.models import RetryChatModel
class TestPollingPriceManager:
@@ -231,37 +90,67 @@ class TestPollingPriceManager:
assert len(manager.open_prices) == 0
def test_fetch_prices_suppresses_repeated_failures(self, caplog):
manager = PollingPriceManager(provider="yfinance", poll_interval=10)
manager.subscribe(["AAPL"])
with patch.object(manager, "_fetch_quote", side_effect=ValueError("empty quote")):
with caplog.at_level(logging.DEBUG):
for _ in range(3):
manager._fetch_prices()
assert manager._failure_counts["AAPL"] == 3
warning_messages = [record.message for record in caplog.records if record.levelno >= logging.WARNING]
assert any("Failed to fetch AAPL price: empty quote" in message for message in warning_messages)
def test_fetch_prices_logs_recovery_after_failure(self, caplog):
manager = PollingPriceManager(provider="yfinance", poll_interval=10)
manager.subscribe(["AAPL"])
with patch.object(
manager,
"_fetch_quote",
side_effect=[
ValueError("temporary outage"),
{"c": 100.0, "o": 99.0, "h": 101.0, "l": 98.0, "pc": 99.5, "d": 0.5, "dp": 0.5, "t": 1},
],
):
with caplog.at_level(logging.INFO):
manager._fetch_prices()
manager._fetch_prices()
assert "AAPL" not in manager._failure_counts
assert any("recovered after 1 consecutive failures" in record.message for record in caplog.records)
class TestRetryChatModel:
@pytest.mark.asyncio
async def test_async_retry_recovers_from_disconnect(self):
attempts = {"count": 0}
class FakeAsyncModel:
model_name = "fake-async-model"
async def __call__(self, *args, **kwargs):
attempts["count"] += 1
if attempts["count"] < 2:
raise RuntimeError("Server disconnected")
return {"ok": True}
wrapped = RetryChatModel(FakeAsyncModel(), max_retries=2, initial_delay=0.01)
result = await wrapped("hello")
assert result == {"ok": True}
assert attempts["count"] == 2
class TestMarketService:
def test_init_mock_mode(self):
service = MarketService(
tickers=["AAPL", "MSFT"],
poll_interval=10,
mock_mode=True,
)
assert service.tickers == ["AAPL", "MSFT"]
assert service.poll_interval == 10
assert service.mock_mode is True
assert service.running is False
def test_init_real_mode(self):
service = MarketService(
tickers=["AAPL"],
mock_mode=False,
api_key="test_key",
)
assert service.mock_mode is False
assert service.api_key == "test_key"
@patch("backend.services.market.get_data_source", return_value="yfinance")
@patch("backend.services.market.get_data_sources", return_value=["yfinance", "local_csv"])
@patch.object(PollingPriceManager, "start")
def test_start_real_mode_with_yfinance(self, _mock_start, _mock_source):
def test_start_real_mode_with_yfinance(self, _mock_start, _mock_sources):
service = MarketService(
tickers=["AAPL"],
poll_interval=10,
mock_mode=False,
)
service._start_real_mode()
@@ -269,30 +158,24 @@ class TestMarketService:
assert isinstance(service._price_manager, PollingPriceManager)
assert service._price_manager.provider == "yfinance"
@pytest.mark.asyncio
async def test_start_mock_mode(self):
@patch("backend.services.market.get_data_sources", return_value=["financial_datasets", "yfinance", "local_csv"])
@patch.object(PollingPriceManager, "start")
def test_start_real_mode_uses_first_supported_live_provider(self, _mock_start, _mock_sources):
service = MarketService(
tickers=["AAPL"],
poll_interval=10,
mock_mode=True,
)
broadcast_func = AsyncMock()
service._start_real_mode()
await service.start(broadcast_func)
assert isinstance(service._price_manager, PollingPriceManager)
assert service._price_manager.provider == "yfinance"
assert service.running is True
assert service._price_manager is not None
assert isinstance(service._price_manager, MockPriceManager)
service.stop()
@patch("backend.services.market.get_data_source", return_value="finnhub")
@patch("backend.services.market.get_data_sources", return_value=["finnhub", "yfinance"])
@pytest.mark.asyncio
async def test_start_real_mode_without_api_key(self, _mock_source):
async def test_start_real_mode_without_api_key(self, _mock_sources):
service = MarketService(
tickers=["AAPL"],
mock_mode=False,
api_key=None,
)
@@ -307,11 +190,12 @@ class TestMarketService:
async def test_start_already_running(self):
service = MarketService(
tickers=["AAPL"],
mock_mode=True,
backtest_mode=True,
)
broadcast_func = AsyncMock()
# First start with backtest mode
await service.start(broadcast_func)
assert service.running is True
@@ -323,7 +207,7 @@ class TestMarketService:
def test_stop(self):
service = MarketService(
tickers=["AAPL"],
mock_mode=True,
backtest_mode=True,
)
service.running = True
service._price_manager = MagicMock()
@@ -336,7 +220,7 @@ class TestMarketService:
def test_stop_when_not_running(self):
service = MarketService(
tickers=["AAPL"],
mock_mode=True,
backtest_mode=True,
)
# Should not raise
@@ -344,20 +228,20 @@ class TestMarketService:
assert service.running is False
def test_get_price_sync(self):
service = MarketService(tickers=["AAPL"], mock_mode=True)
service = MarketService(tickers=["AAPL"], backtest_mode=True)
service.cache["AAPL"] = {"price": 150.0, "open": 148.0}
price = service.get_price_sync("AAPL")
assert price == 150.0
def test_get_price_sync_not_found(self):
service = MarketService(tickers=["AAPL"], mock_mode=True)
service = MarketService(tickers=["AAPL"], backtest_mode=True)
price = service.get_price_sync("MSFT")
assert price is None
def test_get_all_prices(self):
service = MarketService(tickers=["AAPL", "MSFT"], mock_mode=True)
service = MarketService(tickers=["AAPL", "MSFT"], backtest_mode=True)
service.cache["AAPL"] = {"price": 150.0}
service.cache["MSFT"] = {"price": 400.0}
@@ -368,7 +252,7 @@ class TestMarketService:
@pytest.mark.asyncio
async def test_broadcast_price_update(self):
service = MarketService(tickers=["AAPL"], mock_mode=True)
service = MarketService(tickers=["AAPL"], backtest_mode=True)
service._broadcast_func = AsyncMock()
price_data = {
@@ -388,7 +272,7 @@ class TestMarketService:
@pytest.mark.asyncio
async def test_broadcast_price_update_no_func(self):
service = MarketService(tickers=["AAPL"], mock_mode=True)
service = MarketService(tickers=["AAPL"], backtest_mode=True)
service._broadcast_func = None
price_data = {"symbol": "AAPL", "price": 150.0, "open": 148.0}
@@ -396,67 +280,6 @@ class TestMarketService:
# Should not raise
await service._broadcast_price_update(price_data)
@pytest.mark.asyncio
async def test_price_callback_thread_safety(self):
service = MarketService(
tickers=["AAPL"],
poll_interval=1,
mock_mode=True,
)
received_prices = []
async def capture_broadcast(msg):
received_prices.append(msg)
await service.start(capture_broadcast)
# Wait for at least one price update
await asyncio.sleep(1.5)
service.stop()
# Should have received at least one price update
assert len(received_prices) >= 1
assert received_prices[0]["type"] == "price_update"
class TestMarketServiceIntegration:
@pytest.mark.asyncio
async def test_full_mock_cycle(self):
service = MarketService(
tickers=["AAPL", "MSFT"],
poll_interval=1,
mock_mode=True,
)
messages = []
async def collect_messages(msg):
messages.append(msg)
await service.start(collect_messages)
# Wait for price updates
await asyncio.sleep(2.5)
service.stop()
# Should have received multiple price updates
assert len(messages) >= 2
# Check message structure
symbols_seen = set()
for msg in messages:
assert msg["type"] == "price_update"
assert "symbol" in msg
assert "price" in msg
assert "ret" in msg
symbols_seen.add(msg["symbol"])
# Should have prices for both tickers
assert "AAPL" in symbols_seen or "MSFT" in symbols_seen
if __name__ == "__main__":
pytest.main([__file__, "-v"])

View File

@@ -0,0 +1,197 @@
# -*- coding: utf-8 -*-
"""Unit tests for the news domain helpers."""
from backend.domains import news as news_domain
class _FakeStore:
def __init__(self):
self.calls = []
def get_ticker_watermarks(self, symbol):
self.calls.append(("get_ticker_watermarks", symbol))
return {"symbol": symbol, "last_news_fetch": "2026-03-10"}
def get_news_items_enriched(self, ticker, start_date=None, end_date=None, trade_date=None, limit=100):
self.calls.append(("get_news_items_enriched", ticker, start_date, end_date, trade_date, limit))
target = trade_date or end_date
return [{"id": "n1", "ticker": ticker, "date": target, "trade_date": target}]
def get_news_timeline_enriched(self, ticker, start_date=None, end_date=None):
self.calls.append(("get_news_timeline_enriched", ticker, start_date, end_date))
return [{"date": end_date, "count": 1}]
def get_news_categories_enriched(self, ticker, start_date=None, end_date=None, limit=200):
self.calls.append(("get_news_categories_enriched", ticker, start_date, end_date, limit))
return {"macro": {"count": 1}}
def get_news_by_ids_enriched(self, ticker, article_ids):
self.calls.append(("get_news_by_ids_enriched", ticker, list(article_ids)))
return [{"id": article_ids[0], "ticker": ticker, "date": "2026-03-16"}]
def test_news_rows_need_enrichment_detects_missing_fields():
assert news_domain.news_rows_need_enrichment([]) is True
assert news_domain.news_rows_need_enrichment([{"sentiment": "", "relevance": "", "key_discussion": ""}]) is True
assert news_domain.news_rows_need_enrichment([{"sentiment": "positive"}]) is False
def test_ensure_news_fresh_triggers_incremental_refresh_when_watermark_is_stale(monkeypatch):
store = _FakeStore()
calls = []
monkeypatch.setattr(
news_domain,
"update_ticker_incremental",
lambda symbol, end_date=None, store=None: calls.append((symbol, end_date)),
)
payload = news_domain.ensure_news_fresh(store, ticker="AAPL", target_date="2026-03-16")
assert calls == [("AAPL", "2026-03-16")]
assert payload["target_date"] == "2026-03-16"
assert payload["refreshed"] is True
def test_ensure_news_fresh_skips_refresh_when_watermark_is_current(monkeypatch):
store = _FakeStore()
calls = []
monkeypatch.setattr(
store,
"get_ticker_watermarks",
lambda symbol: {"symbol": symbol, "last_news_fetch": "2026-03-16"},
)
monkeypatch.setattr(
news_domain,
"update_ticker_incremental",
lambda symbol, end_date=None, store=None: calls.append((symbol, end_date)),
)
payload = news_domain.ensure_news_fresh(store, ticker="AAPL", target_date="2026-03-16")
assert calls == []
assert payload["refreshed"] is False
def test_get_enriched_news_returns_rows_without_enrichment_when_present(monkeypatch):
store = _FakeStore()
monkeypatch.setattr(news_domain, "news_rows_need_enrichment", lambda rows: False)
monkeypatch.setattr(
news_domain,
"ensure_news_fresh",
lambda store, ticker, target_date=None, refresh_if_stale=False: {
"ticker": ticker,
"target_date": target_date,
"last_news_fetch": target_date,
"refreshed": False,
},
)
payload = news_domain.get_enriched_news(
store,
ticker="AAPL",
start_date="2026-03-01",
end_date="2026-03-16",
limit=20,
)
assert payload["ticker"] == "AAPL"
assert payload["news"][0]["ticker"] == "AAPL"
assert payload["freshness"]["target_date"] is None or payload["freshness"]["target_date"] == "2026-03-16"
assert store.calls == [
("get_news_items_enriched", "AAPL", "2026-03-01", "2026-03-16", None, 20)
]
def test_get_story_and_similar_days_delegate(monkeypatch):
store = _FakeStore()
monkeypatch.setattr(
news_domain,
"ensure_news_fresh",
lambda store, ticker, target_date=None, refresh_if_stale=False: {
"ticker": ticker,
"target_date": target_date,
"last_news_fetch": target_date,
"refreshed": False,
},
)
monkeypatch.setattr(news_domain, "enrich_news_for_symbol", lambda *args, **kwargs: {"analyzed": 1})
monkeypatch.setattr(
news_domain,
"get_or_create_stock_story",
lambda store, symbol, as_of_date: {"symbol": symbol, "as_of_date": as_of_date, "story": "story"},
)
monkeypatch.setattr(
news_domain,
"find_similar_days",
lambda store, symbol, target_date, top_k: {"symbol": symbol, "target_date": target_date, "items": [{"score": 0.9}]},
)
story = news_domain.get_story_payload(store, ticker="AAPL", as_of_date="2026-03-16")
similar = news_domain.get_similar_days_payload(store, ticker="AAPL", date="2026-03-16", n_similar=8)
assert story["story"] == "story"
assert "freshness" in story
assert similar["items"][0]["score"] == 0.9
assert "freshness" in similar
def test_get_enriched_news_defaults_to_read_only_freshness(monkeypatch):
store = _FakeStore()
ensure_calls = []
def fake_ensure(store, ticker, target_date=None, refresh_if_stale=False):
ensure_calls.append(refresh_if_stale)
return {
"ticker": ticker,
"target_date": target_date,
"last_news_fetch": target_date,
"refreshed": False,
}
monkeypatch.setattr(news_domain, "ensure_news_fresh", fake_ensure)
monkeypatch.setattr(news_domain, "news_rows_need_enrichment", lambda rows: False)
payload = news_domain.get_enriched_news(
store,
ticker="AAPL",
end_date="2026-03-16",
)
assert payload["ticker"] == "AAPL"
assert ensure_calls == [False]
def test_get_range_explain_payload_uses_article_ids(monkeypatch):
store = _FakeStore()
monkeypatch.setattr(
news_domain,
"ensure_news_fresh",
lambda store, ticker, target_date=None, refresh_if_stale=False: {
"ticker": ticker,
"target_date": target_date,
"last_news_fetch": target_date,
"refreshed": False,
},
)
monkeypatch.setattr(news_domain, "news_rows_need_enrichment", lambda rows: False)
monkeypatch.setattr(
news_domain,
"build_range_explanation",
lambda ticker, start_date, end_date, news_rows: {"ticker": ticker, "count": len(news_rows)},
)
payload = news_domain.get_range_explain_payload(
store,
ticker="AAPL",
start_date="2026-03-10",
end_date="2026-03-16",
article_ids=["news-9"],
limit=50,
)
assert payload["ticker"] == "AAPL"
assert payload["result"] == {"ticker": "AAPL", "count": 1}
assert "freshness" in payload
assert store.calls == [("get_news_by_ids_enriched", "AAPL", ["news-9"])]

View File

@@ -0,0 +1,180 @@
# -*- coding: utf-8 -*-
"""Tests for the extracted news service app surface."""
from fastapi.testclient import TestClient
from backend.apps.news_service import create_app
class _FakeStore:
def get_ticker_watermarks(self, symbol):
return {"symbol": symbol, "last_news_fetch": "2026-12-31"}
def get_news_timeline_enriched(self, symbol, start_date=None, end_date=None):
return [{"date": end_date, "count": 1}]
def get_news_items(self, symbol, start_date=None, end_date=None, limit=100):
return [{"id": "news-raw-1", "ticker": symbol, "title": "Raw Title", "date": end_date}]
def get_news_items_enriched(self, symbol, start_date=None, end_date=None, trade_date=None, limit=100):
return [{"id": "news-1", "ticker": symbol, "title": "Title", "date": trade_date or end_date}]
def upsert_news_analysis(self, symbol, rows):
return len(rows)
def get_analyzed_news_ids(self, symbol, start_date=None, end_date=None):
return set()
def get_news_categories_enriched(self, symbol, start_date=None, end_date=None, limit=200):
return {"market": {"label": "market", "count": 1, "article_ids": ["news-1"]}}
def get_news_by_ids_enriched(self, symbol, article_ids):
return [{"id": article_ids[0], "ticker": symbol, "title": "Picked"}]
def test_news_service_routes_are_exposed():
app = create_app()
paths = {route.path for route in app.routes}
assert "/health" in paths
assert "/api/enriched-news" in paths
assert "/api/news-for-date" in paths
assert "/api/news-timeline" in paths
assert "/api/categories" in paths
assert "/api/similar-days" in paths
assert "/api/stories/{ticker}" in paths
assert "/api/range-explain" in paths
def test_news_service_enriched_news_and_categories(monkeypatch):
app = create_app()
app.dependency_overrides.clear()
from backend.apps import news_service as news_service_module
app.dependency_overrides[news_service_module.get_market_store] = lambda: _FakeStore()
monkeypatch.setattr(
"backend.domains.news.enrich_news_for_symbol",
lambda *args, **kwargs: {"symbol": "AAPL", "analyzed": 1},
)
with TestClient(app) as client:
news_response = client.get(
"/api/enriched-news",
params={"ticker": "AAPL", "end_date": "2026-03-23"},
)
categories_response = client.get(
"/api/categories",
params={"ticker": "AAPL", "end_date": "2026-03-23"},
)
assert news_response.status_code == 200
assert news_response.json()["news"][0]["ticker"] == "AAPL"
assert categories_response.status_code == 200
assert categories_response.json()["categories"]["market"]["count"] == 1
def test_news_service_news_for_date_and_timeline(monkeypatch):
app = create_app()
from backend.apps import news_service as news_service_module
app.dependency_overrides[news_service_module.get_market_store] = lambda: _FakeStore()
monkeypatch.setattr(
"backend.domains.news.enrich_news_for_symbol",
lambda *args, **kwargs: {"symbol": "AAPL", "analyzed": 1},
)
with TestClient(app) as client:
date_response = client.get(
"/api/news-for-date",
params={"ticker": "AAPL", "date": "2026-03-23"},
)
timeline_response = client.get(
"/api/news-timeline",
params={
"ticker": "AAPL",
"start_date": "2026-03-01",
"end_date": "2026-03-23",
},
)
assert date_response.status_code == 200
assert date_response.json()["date"] == "2026-03-23"
assert timeline_response.status_code == 200
assert timeline_response.json()["timeline"][0]["count"] == 1
def test_news_service_similar_days_and_story(monkeypatch):
app = create_app()
from backend.apps import news_service as news_service_module
app.dependency_overrides[news_service_module.get_market_store] = lambda: _FakeStore()
monkeypatch.setattr(
"backend.domains.news.enrich_news_for_symbol",
lambda *args, **kwargs: {"symbol": "AAPL", "analyzed": 1},
)
monkeypatch.setattr(
"backend.domains.news.find_similar_days",
lambda store, symbol, target_date, top_k: {
"symbol": symbol,
"target_date": target_date,
"items": [{"date": "2026-03-20", "score": 0.9}],
},
)
monkeypatch.setattr(
"backend.domains.news.get_or_create_stock_story",
lambda store, symbol, as_of_date: {
"symbol": symbol,
"as_of_date": as_of_date,
"story": "story body",
"source": "local",
},
)
with TestClient(app) as client:
similar_response = client.get(
"/api/similar-days",
params={"ticker": "AAPL", "date": "2026-03-23", "n_similar": 3},
)
story_response = client.get(
"/api/stories/AAPL",
params={"as_of_date": "2026-03-23"},
)
assert similar_response.status_code == 200
assert similar_response.json()["items"][0]["score"] == 0.9
assert story_response.status_code == 200
assert story_response.json()["story"] == "story body"
def test_news_service_range_explain(monkeypatch):
app = create_app()
from backend.apps import news_service as news_service_module
app.dependency_overrides[news_service_module.get_market_store] = lambda: _FakeStore()
monkeypatch.setattr(
"backend.domains.news.enrich_news_for_symbol",
lambda *args, **kwargs: {"symbol": "AAPL", "analyzed": 1},
)
monkeypatch.setattr(
"backend.domains.news.build_range_explanation",
lambda ticker, start_date, end_date, news_rows: {
"symbol": ticker,
"news_count": len(news_rows),
"start_date": start_date,
"end_date": end_date,
},
)
with TestClient(app) as client:
response = client.get(
"/api/range-explain",
params={
"ticker": "AAPL",
"start_date": "2026-03-01",
"end_date": "2026-03-23",
"article_ids": ["news-7"],
},
)
assert response.status_code == 200
assert response.json()["result"]["news_count"] == 1

View File

@@ -0,0 +1,60 @@
# -*- coding: utf-8 -*-
"""Tests for the OpenClaw CLI service wrapper."""
from pathlib import Path
import pytest
from backend.services.openclaw_cli import OpenClawCliError, OpenClawCliService
class _Completed:
def __init__(self, *, returncode=0, stdout="", stderr=""):
self.returncode = returncode
self.stdout = stdout
self.stderr = stderr
def test_openclaw_cli_service_runs_json_command(monkeypatch, tmp_path):
captured = {}
def _fake_run(command, **kwargs):
captured["command"] = command
captured["cwd"] = kwargs["cwd"]
return _Completed(stdout='{"sessions":[{"key":"main/session-1"}]}')
monkeypatch.setattr("backend.services.openclaw_cli.subprocess.run", _fake_run)
service = OpenClawCliService(base_command=["openclaw"], cwd=tmp_path, timeout_seconds=3)
payload = service.list_sessions()
assert payload["sessions"][0]["key"] == "main/session-1"
assert captured["command"] == ["openclaw", "sessions", "--json"]
assert captured["cwd"] == tmp_path
def test_openclaw_cli_service_raises_on_failure(monkeypatch, tmp_path):
def _fake_run(command, **kwargs):
return _Completed(returncode=7, stdout="", stderr="boom")
monkeypatch.setattr("backend.services.openclaw_cli.subprocess.run", _fake_run)
service = OpenClawCliService(base_command=["openclaw"], cwd=tmp_path, timeout_seconds=3)
with pytest.raises(OpenClawCliError) as exc_info:
service.list_cron_jobs()
assert exc_info.value.exit_code == 7
assert exc_info.value.stderr == "boom"
def test_openclaw_cli_service_can_extract_single_session(monkeypatch, tmp_path):
def _fake_run(command, **kwargs):
return _Completed(stdout='{"sessions":[{"key":"main/session-1","agentId":"main"}]}')
monkeypatch.setattr("backend.services.openclaw_cli.subprocess.run", _fake_run)
service = OpenClawCliService(base_command=["openclaw"], cwd=tmp_path, timeout_seconds=3)
session = service.get_session("main/session-1")
assert session["agentId"] == "main"

View File

@@ -0,0 +1,110 @@
# -*- coding: utf-8 -*-
"""Tests for the extracted OpenClaw service app surface."""
from fastapi.testclient import TestClient
from backend.apps.openclaw_service import create_app
from backend.api import openclaw as openclaw_module
class _FakeOpenClawCliService:
def health(self):
return {
"status": "healthy",
"service": "openclaw-service",
"base_command": ["openclaw"],
"cwd": "/tmp/openclaw",
"binary_resolved": True,
"reference_entry_available": True,
"timeout_seconds": 15.0,
}
def status(self):
return {"runtimeVersion": "2026.3.24"}
def list_sessions(self):
return {
"sessions": [
{"key": "main/session-1", "agentId": "main"},
{"key": "analyst/session-2", "agentId": "analyst"},
]
}
def get_session(self, session_key: str):
for session in self.list_sessions()["sessions"]:
if session["key"] == session_key:
return session
raise KeyError(session_key)
def get_session_history(self, session_key: str, *, limit: int = 20):
return {
"sessionKey": session_key,
"limit": limit,
"items": [{"role": "assistant", "text": "hello"}],
}
def list_cron_jobs(self):
return {"jobs": [{"id": "job-1", "name": "Daily sync"}]}
def list_approvals(self):
return {"approvals": [{"id": "ap-1", "status": "pending"}]}
def test_openclaw_service_routes_are_exposed():
app = create_app()
paths = {route.path for route in app.routes}
assert "/health" in paths
assert "/api/status" in paths
assert "/api/openclaw/status" in paths
assert "/api/openclaw/sessions" in paths
assert "/api/openclaw/sessions/{session_key:path}" in paths
assert "/api/openclaw/sessions/{session_key:path}/history" in paths
assert "/api/openclaw/cron" in paths
assert "/api/openclaw/approvals" in paths
def test_openclaw_service_read_routes():
app = create_app()
app.dependency_overrides[openclaw_module.get_openclaw_cli_service] = (
lambda: _FakeOpenClawCliService()
)
with TestClient(app) as client:
health = client.get("/health")
status = client.get("/api/status")
openclaw_status = client.get("/api/openclaw/status")
sessions = client.get("/api/openclaw/sessions")
session = client.get("/api/openclaw/sessions/main/session-1")
history = client.get("/api/openclaw/sessions/main/session-1/history", params={"limit": 5})
cron = client.get("/api/openclaw/cron")
approvals = client.get("/api/openclaw/approvals")
assert health.status_code == 200
assert health.json()["service"] == "openclaw-service"
assert status.status_code == 200
assert status.json()["status"] == "operational"
assert openclaw_status.status_code == 200
assert openclaw_status.json()["runtimeVersion"] == "2026.3.24"
assert sessions.status_code == 200
assert len(sessions.json()["sessions"]) == 2
assert session.status_code == 200
assert session.json()["session"]["agentId"] == "main"
assert history.status_code == 200
assert history.json()["limit"] == 5
assert cron.status_code == 200
assert cron.json()["jobs"][0]["id"] == "job-1"
assert approvals.status_code == 200
assert approvals.json()["approvals"][0]["id"] == "ap-1"
def test_openclaw_service_session_404():
app = create_app()
app.dependency_overrides[openclaw_module.get_openclaw_cli_service] = (
lambda: _FakeOpenClawCliService()
)
with TestClient(app) as client:
response = client.get("/api/openclaw/sessions/missing")
assert response.status_code == 404

View File

@@ -0,0 +1,74 @@
# -*- coding: utf-8 -*-
"""Tests for the OpenClaw WebSocket client session helpers."""
import pytest
from shared.client.openclaw_websocket_client import OpenClawWebSocketClient
@pytest.mark.asyncio
async def test_resolve_session_parses_gateway_key_response():
client = OpenClawWebSocketClient(gateway_token="test-token")
async def fake_send_request(method, params=None, _allow_handshake=False):
assert method == "sessions.resolve"
assert params["agentId"] == "main"
return {"ok": True, "key": "agent:main:main"}
client._send_request = fake_send_request # type: ignore[method-assign]
resolved = await client.resolve_session(agent_id="main")
assert resolved == "agent:main:main"
@pytest.mark.asyncio
async def test_send_message_uses_session_send_payload():
client = OpenClawWebSocketClient(gateway_token="test-token")
async def fake_send_request(method, params=None, _allow_handshake=False):
assert method == "sessions.send"
assert params == {
"key": "agent:main:main",
"message": "hello",
"thinking": "medium",
}
return {"ok": True, "runId": "run-1"}
client._send_request = fake_send_request # type: ignore[method-assign]
result = await client.send_message("agent:main:main", "hello", thinking="medium")
assert result["runId"] == "run-1"
@pytest.mark.asyncio
async def test_get_session_history_uses_sessions_preview():
client = OpenClawWebSocketClient(gateway_token="test-token")
async def fake_send_request(method, params=None, _allow_handshake=False):
assert method == "sessions.preview"
assert params == {"keys": ["agent:main:main"], "limit": 12}
return {"previews": []}
client._send_request = fake_send_request # type: ignore[method-assign]
result = await client.get_session_history("agent:main:main", limit=12)
assert result == {"previews": []}
@pytest.mark.asyncio
async def test_unsubscribe_uses_session_messages_unsubscribe():
client = OpenClawWebSocketClient(gateway_token="test-token")
async def fake_send_request(method, params=None, _allow_handshake=False):
assert method == "sessions.messages.unsubscribe"
assert params == {"key": "agent:main:main"}
return {"subscribed": False}
client._send_request = fake_send_request # type: ignore[method-assign]
result = await client.unsubscribe("agent:main:main")
assert result == {"subscribed": False}

View File

@@ -9,6 +9,7 @@ def test_router_includes_local_csv_fallback(monkeypatch):
monkeypatch.delenv("FINNHUB_API_KEY", raising=False)
monkeypatch.delenv("FINANCIAL_DATASETS_API_KEY", raising=False)
monkeypatch.delenv("FIN_DATA_SOURCE", raising=False)
monkeypatch.delenv("ENABLED_DATA_SOURCES", raising=False)
reset_config()
router = DataProviderRouter()

View File

@@ -0,0 +1,364 @@
# -*- coding: utf-8 -*-
"""Tests for the extracted runtime service app surface."""
import json
from pathlib import Path
from fastapi.testclient import TestClient
from backend.api import runtime as runtime_module
from backend.apps.runtime_service import create_app
def test_runtime_service_routes_are_exposed():
app = create_app()
paths = {route.path for route in app.routes}
assert "/health" in paths
assert "/api/status" in paths
assert "/api/runtime/start" in paths
assert "/api/runtime/stop" in paths
assert "/api/runtime/cleanup" in paths
assert "/api/runtime/history" in paths
assert "/api/runtime/current" in paths
assert "/api/runtime/gateway/port" in paths
def test_runtime_service_health_and_status(monkeypatch):
runtime_state = runtime_module.get_runtime_state()
runtime_state.gateway_process = None
runtime_state.gateway_port = 9876
runtime_state.runtime_manager = object()
with TestClient(create_app()) as client:
health_response = client.get("/health")
status_response = client.get("/api/status")
assert health_response.status_code == 200
assert health_response.json() == {
"status": "healthy",
"service": "runtime-service",
"gateway_running": False,
"gateway_port": 9876,
}
assert status_response.status_code == 200
assert status_response.json() == {
"status": "operational",
"service": "runtime-service",
"runtime": {
"gateway_running": False,
"gateway_port": 9876,
"has_runtime_manager": True,
},
}
def test_runtime_service_gateway_port_endpoint_uses_runtime_router(monkeypatch):
runtime_module.get_runtime_state().gateway_port = 9345
monkeypatch.setattr(runtime_module, "_is_gateway_running", lambda: True)
with TestClient(create_app()) as client:
response = client.get(
"/api/runtime/gateway/port",
headers={"host": "runtime.example:8003", "x-forwarded-proto": "https"},
)
assert response.status_code == 200
assert response.json() == {
"port": 9345,
"is_running": True,
"ws_url": "wss://runtime.example:9345",
}
def test_runtime_service_get_runtime_config(monkeypatch, tmp_path):
run_dir = tmp_path / "runs" / "demo"
state_dir = run_dir / "state"
state_dir.mkdir(parents=True)
(run_dir / "BOOTSTRAP.md").write_text(
"---\n"
"tickers:\n"
" - AAPL\n"
"schedule_mode: intraday\n"
"interval_minutes: 30\n"
"trigger_time: '10:00'\n"
"max_comm_cycles: 3\n"
"enable_memory: true\n"
"---\n",
encoding="utf-8",
)
(state_dir / "runtime_state.json").write_text(
json.dumps(
{
"context": {
"config_name": "demo",
"run_dir": str(run_dir),
"bootstrap_values": {
"tickers": ["AAPL"],
"schedule_mode": "intraday",
"interval_minutes": 30,
"trigger_time": "10:00",
"max_comm_cycles": 3,
"enable_memory": True,
},
}
}
),
encoding="utf-8",
)
monkeypatch.setattr(runtime_module, "PROJECT_ROOT", tmp_path)
monkeypatch.setattr(runtime_module, "_is_gateway_running", lambda: True)
runtime_module.get_runtime_state().gateway_port = 8765
with TestClient(create_app()) as client:
response = client.get("/api/runtime/config")
assert response.status_code == 200
payload = response.json()
assert payload["run_id"] == "demo"
assert payload["bootstrap"]["schedule_mode"] == "intraday"
assert payload["resolved"]["interval_minutes"] == 30
assert payload["resolved"]["enable_memory"] is True
def test_runtime_service_update_runtime_config_persists_bootstrap(monkeypatch, tmp_path):
run_dir = tmp_path / "runs" / "demo"
state_dir = run_dir / "state"
state_dir.mkdir(parents=True)
(run_dir / "BOOTSTRAP.md").write_text(
"---\n"
"tickers:\n"
" - AAPL\n"
"schedule_mode: daily\n"
"interval_minutes: 60\n"
"trigger_time: '09:30'\n"
"max_comm_cycles: 2\n"
"---\n",
encoding="utf-8",
)
(state_dir / "runtime_state.json").write_text(
json.dumps(
{
"context": {
"config_name": "demo",
"run_dir": str(run_dir),
"bootstrap_values": {
"tickers": ["AAPL"],
"schedule_mode": "daily",
"interval_minutes": 60,
"trigger_time": "09:30",
"max_comm_cycles": 2,
},
}
}
),
encoding="utf-8",
)
class _DummyContext:
def __init__(self):
self.bootstrap_values = {
"tickers": ["AAPL"],
"schedule_mode": "daily",
"interval_minutes": 60,
"trigger_time": "09:30",
"max_comm_cycles": 2,
}
class _DummyManager:
def __init__(self):
self.config_name = "demo"
self.bootstrap = dict(_DummyContext().bootstrap_values)
self.context = _DummyContext()
def _persist_snapshot(self):
return None
monkeypatch.setattr(runtime_module, "PROJECT_ROOT", tmp_path)
monkeypatch.setattr(runtime_module, "_is_gateway_running", lambda: True)
runtime_module.get_runtime_state().runtime_manager = _DummyManager()
runtime_module.get_runtime_state().gateway_port = 8765
with TestClient(create_app()) as client:
response = client.put(
"/api/runtime/config",
json={
"schedule_mode": "intraday",
"interval_minutes": 15,
"trigger_time": "10:15",
"max_comm_cycles": 4,
},
)
assert response.status_code == 200
payload = response.json()
assert payload["bootstrap"]["schedule_mode"] == "intraday"
assert payload["resolved"]["interval_minutes"] == 15
assert "interval_minutes: 15" in (run_dir / "BOOTSTRAP.md").read_text(encoding="utf-8")
def test_prune_old_timestamped_runs_keeps_named_runs(monkeypatch, tmp_path):
runs_dir = tmp_path / "runs"
runs_dir.mkdir()
keep_dirs = ["20260324_110000", "20260324_120000"]
prune_dir = "20260324_100000"
named_dir = "smoke_fullstack"
for name in [*keep_dirs, prune_dir, named_dir]:
(runs_dir / name).mkdir(parents=True)
monkeypatch.setattr(runtime_module, "PROJECT_ROOT", tmp_path)
pruned = runtime_module._prune_old_timestamped_runs(keep=1, exclude_run_ids={"20260324_120000"})
assert prune_dir in pruned
assert (runs_dir / named_dir).exists()
assert (runs_dir / "20260324_120000").exists()
assert (runs_dir / "20260324_110000").exists()
def test_runtime_cleanup_endpoint_prunes_old_runs(monkeypatch, tmp_path):
runs_dir = tmp_path / "runs"
runs_dir.mkdir()
for name in ["20260324_090000", "20260324_100000", "20260324_110000", "smoke_fullstack"]:
(runs_dir / name).mkdir(parents=True)
monkeypatch.setattr(runtime_module, "PROJECT_ROOT", tmp_path)
monkeypatch.setattr(runtime_module, "_is_gateway_running", lambda: False)
with TestClient(create_app()) as client:
response = client.post("/api/runtime/cleanup?keep=1")
assert response.status_code == 200
payload = response.json()
assert payload["status"] == "ok"
assert sorted(payload["pruned_run_ids"]) == ["20260324_090000", "20260324_100000"]
assert (runs_dir / "20260324_110000").exists()
assert (runs_dir / "smoke_fullstack").exists()
def test_runtime_history_lists_recent_runs(monkeypatch, tmp_path):
run_dir = tmp_path / "runs" / "20260324_120000"
(run_dir / "state").mkdir(parents=True)
(run_dir / "team_dashboard").mkdir(parents=True)
(run_dir / "state" / "runtime_state.json").write_text(
json.dumps(
{
"context": {
"config_name": "20260324_120000",
"run_dir": str(run_dir),
"bootstrap_values": {"tickers": ["AAPL"]},
},
"events": [],
}
),
encoding="utf-8",
)
(run_dir / "team_dashboard" / "summary.json").write_text(
json.dumps({"totalTrades": 3, "totalAssetValue": 123456.0}),
encoding="utf-8",
)
monkeypatch.setattr(runtime_module, "PROJECT_ROOT", tmp_path)
with TestClient(create_app()) as client:
response = client.get("/api/runtime/history?limit=5")
assert response.status_code == 200
payload = response.json()
assert payload["runs"][0]["run_id"] == "20260324_120000"
assert payload["runs"][0]["total_trades"] == 3
def test_restore_run_assets_copies_state(monkeypatch, tmp_path):
source_run = tmp_path / "runs" / "20260324_100000"
(source_run / "team_dashboard").mkdir(parents=True)
(source_run / "state").mkdir(parents=True)
(source_run / "agents").mkdir(parents=True)
(source_run / "team_dashboard" / "_internal_state.json").write_text("{}", encoding="utf-8")
(source_run / "state" / "server_state.json").write_text("{}", encoding="utf-8")
target_run = tmp_path / "runs" / "20260324_130000"
monkeypatch.setattr(runtime_module, "PROJECT_ROOT", tmp_path)
runtime_module._restore_run_assets("20260324_100000", target_run)
assert (target_run / "team_dashboard" / "_internal_state.json").exists()
assert (target_run / "state" / "server_state.json").exists()
def test_start_runtime_restore_reuses_historical_run_id(monkeypatch, tmp_path):
run_dir = tmp_path / "runs" / "20260324_100000"
(run_dir / "state").mkdir(parents=True)
(run_dir / "state" / "runtime_state.json").write_text(
json.dumps(
{
"context": {
"config_name": "20260324_100000",
"run_dir": str(run_dir),
"bootstrap_values": {
"tickers": ["AAPL"],
"schedule_mode": "intraday",
"interval_minutes": 30,
"trigger_time": "now",
"max_comm_cycles": 2,
"initial_cash": 100000.0,
"margin_requirement": 0.0,
"enable_memory": False,
"mode": "live",
"poll_interval": 10,
},
}
}
),
encoding="utf-8",
)
class _DummyManager:
def __init__(self, config_name, run_dir, bootstrap):
self.config_name = config_name
self.run_dir = Path(run_dir)
self.bootstrap = bootstrap
self.context = None
def prepare_run(self):
self.context = type(
"Ctx",
(),
{
"config_name": self.config_name,
"run_dir": self.run_dir,
"bootstrap_values": self.bootstrap,
},
)()
return self.context
class _DummyProcess:
def poll(self):
return None
monkeypatch.setattr(runtime_module, "PROJECT_ROOT", tmp_path)
monkeypatch.setattr(runtime_module, "_find_available_port", lambda start_port=8765, max_port=9000: 8765)
monkeypatch.setattr(runtime_module, "_start_gateway_process", lambda **kwargs: _DummyProcess())
monkeypatch.setattr(runtime_module, "_stop_gateway", lambda: True)
monkeypatch.setattr("backend.runtime.manager.TradingRuntimeManager", _DummyManager)
runtime_state = runtime_module.get_runtime_state()
runtime_state.gateway_process = None
with TestClient(create_app()) as client:
response = client.post(
"/api/runtime/start",
json={
"launch_mode": "restore",
"restore_run_id": "20260324_100000",
"tickers": [],
},
)
assert response.status_code == 200
payload = response.json()
assert payload["run_id"] == "20260324_100000"
assert payload["run_dir"] == str(run_dir)

View File

@@ -0,0 +1,130 @@
# -*- coding: utf-8 -*-
"""Tests for split-aware shared service clients."""
import pytest
from shared.client.control_client import ControlPlaneClient
from shared.client.openclaw_client import OpenClawServiceClient
from shared.client.runtime_client import RuntimeServiceClient
class _DummyResponse:
def __init__(self, payload):
self._payload = payload
def raise_for_status(self):
return None
def json(self):
return self._payload
class _DummyAsyncClient:
def __init__(self):
self.calls = []
async def get(self, path, params=None):
self.calls.append(("get", path, params))
return _DummyResponse({"path": path, "params": params})
async def post(self, path, json=None):
self.calls.append(("post", path, json))
return _DummyResponse({"path": path, "json": json})
async def put(self, path, json=None):
self.calls.append(("put", path, json))
return _DummyResponse({"path": path, "json": json})
async def aclose(self):
return None
@pytest.mark.asyncio
async def test_control_plane_client_hits_current_workspace_and_guard_routes():
client = ControlPlaneClient()
client._client = _DummyAsyncClient()
await client.list_workspaces()
await client.get_workspace("demo")
await client.list_agents("demo")
await client.get_agent("demo", "risk_manager")
await client.fetch_pending_approvals()
await client.approve_pending_approval("ap-1")
await client.deny_pending_approval("ap-2", reason="nope")
assert client._client.calls == [
("get", "/workspaces", None),
("get", "/workspaces/demo", None),
("get", "/workspaces/demo/agents", None),
("get", "/workspaces/demo/agents/risk_manager", None),
("get", "/guard/pending", None),
(
"post",
"/guard/approve",
{
"approval_id": "ap-1",
"one_time": True,
"expires_in_minutes": 30,
},
),
(
"post",
"/guard/deny",
{
"approval_id": "ap-2",
"reason": "nope",
},
),
]
@pytest.mark.asyncio
async def test_runtime_service_client_hits_current_runtime_routes():
client = RuntimeServiceClient()
client._client = _DummyAsyncClient()
await client.fetch_context()
await client.fetch_agents()
await client.fetch_events()
await client.fetch_gateway_port()
await client.start_runtime({"tickers": ["AAPL"]})
await client.stop_runtime(force=True)
await client.restart_runtime({"tickers": ["MSFT"]})
await client.fetch_current_runtime()
await client.get_runtime_config()
await client.update_runtime_config({"schedule_mode": "intraday"})
assert client._client.calls == [
("get", "/context", None),
("get", "/agents", None),
("get", "/events", None),
("get", "/gateway/port", None),
("post", "/start", {"tickers": ["AAPL"]}),
("post", "/stop?force=true", None),
("post", "/restart", {"tickers": ["MSFT"]}),
("get", "/current", None),
("get", "/config", None),
("put", "/config", {"schedule_mode": "intraday"}),
]
@pytest.mark.asyncio
async def test_openclaw_service_client_hits_current_openclaw_routes():
client = OpenClawServiceClient()
client._client = _DummyAsyncClient()
await client.fetch_status()
await client.list_sessions()
await client.get_session("main/session-1")
await client.get_session_history("main/session-1", limit=5)
await client.list_cron_jobs()
await client.list_approvals()
assert client._client.calls == [
("get", "/status", None),
("get", "/sessions", None),
("get", "/sessions/main/session-1", None),
("get", "/sessions/main/session-1/history", {"limit": 5}),
("get", "/cron", None),
("get", "/approvals", None),
]

View File

@@ -0,0 +1,32 @@
# -*- coding: utf-8 -*-
"""Regression coverage for the shared schema bridge."""
from backend.data import schema as legacy_schema
from shared import schema as shared_schema
def test_backend_data_schema_reexports_shared_contracts():
assert legacy_schema.Price is shared_schema.Price
assert legacy_schema.PriceResponse is shared_schema.PriceResponse
assert legacy_schema.FinancialMetrics is shared_schema.FinancialMetrics
assert legacy_schema.FinancialMetricsResponse is (
shared_schema.FinancialMetricsResponse
)
assert legacy_schema.LineItem is shared_schema.LineItem
assert legacy_schema.LineItemResponse is shared_schema.LineItemResponse
assert legacy_schema.InsiderTrade is shared_schema.InsiderTrade
assert legacy_schema.InsiderTradeResponse is (
shared_schema.InsiderTradeResponse
)
assert legacy_schema.CompanyNews is shared_schema.CompanyNews
assert legacy_schema.CompanyNewsResponse is shared_schema.CompanyNewsResponse
assert legacy_schema.CompanyFacts is shared_schema.CompanyFacts
assert legacy_schema.CompanyFactsResponse is (
shared_schema.CompanyFactsResponse
)
assert legacy_schema.Position is shared_schema.Position
assert legacy_schema.Portfolio is shared_schema.Portfolio
assert legacy_schema.AnalystSignal is shared_schema.AnalystSignal
assert legacy_schema.TickerAnalysis is shared_schema.TickerAnalysis
assert legacy_schema.AgentStateData is shared_schema.AgentStateData
assert legacy_schema.AgentStateMetadata is shared_schema.AgentStateMetadata

View File

@@ -0,0 +1,47 @@
# -*- coding: utf-8 -*-
"""Unit tests for the trading domain helpers."""
from backend.domains import trading as trading_domain
def test_trading_domain_payload_wrappers(monkeypatch):
monkeypatch.setattr(trading_domain, "get_prices", lambda ticker, start_date, end_date: [{"close": 1}])
monkeypatch.setattr(trading_domain, "get_financial_metrics", lambda ticker, end_date, period, limit: [{"ticker": ticker}])
monkeypatch.setattr(trading_domain, "get_company_news", lambda ticker, end_date, start_date=None, limit=1000: [{"ticker": ticker}])
monkeypatch.setattr(trading_domain, "get_insider_trades", lambda ticker, end_date, start_date=None, limit=1000: [{"ticker": ticker}])
monkeypatch.setattr(trading_domain, "get_market_cap", lambda ticker, end_date: 2.5e12)
assert trading_domain.get_prices_payload(ticker="AAPL", start_date="2026-03-01", end_date="2026-03-16") == {
"ticker": "AAPL",
"prices": [{"close": 1}],
}
assert trading_domain.get_financials_payload(ticker="AAPL", end_date="2026-03-16") == {
"financial_metrics": [{"ticker": "AAPL"}],
}
assert trading_domain.get_news_payload(ticker="AAPL", end_date="2026-03-16") == {
"news": [{"ticker": "AAPL"}],
}
assert trading_domain.get_insider_trades_payload(ticker="AAPL", end_date="2026-03-16") == {
"insider_trades": [{"ticker": "AAPL"}],
}
assert trading_domain.get_market_cap_payload(ticker="AAPL", end_date="2026-03-16") == {
"ticker": "AAPL",
"end_date": "2026-03-16",
"market_cap": 2.5e12,
}
def test_get_market_status_payload_uses_market_service(monkeypatch):
class _FakeMarketService:
def __init__(self, tickers):
self.tickers = tickers
def get_market_status(self):
return {"status": "open", "status_text": "Open"}
monkeypatch.setattr(trading_domain, "MarketService", _FakeMarketService)
assert trading_domain.get_market_status_payload() == {
"status": "open",
"status_text": "Open",
}

View File

@@ -0,0 +1,231 @@
# -*- coding: utf-8 -*-
"""Tests for the extracted trading service app surface."""
from fastapi.testclient import TestClient
from backend.apps.trading_service import create_app
from shared.schema import CompanyNews, FinancialMetrics, InsiderTrade, LineItem, Price
def test_trading_service_routes_are_exposed():
app = create_app()
paths = {route.path for route in app.routes}
assert "/health" in paths
assert "/api/prices" in paths
assert "/api/financials" in paths
assert "/api/news" in paths
assert "/api/insider-trades" in paths
assert "/api/market/status" in paths
assert "/api/market-cap" in paths
assert "/api/line-items" in paths
def test_trading_service_prices_endpoint(monkeypatch):
monkeypatch.setattr(
"backend.domains.trading.get_prices_payload",
lambda ticker, start_date, end_date: {
"ticker": ticker,
"prices": [
Price(
open=1.0,
close=2.0,
high=2.5,
low=0.5,
volume=100,
time="2026-03-20",
)
],
},
)
with TestClient(create_app()) as client:
response = client.get(
"/api/prices",
params={
"ticker": "AAPL",
"start_date": "2026-03-01",
"end_date": "2026-03-20",
},
)
assert response.status_code == 200
assert response.json()["ticker"] == "AAPL"
assert response.json()["prices"][0]["close"] == 2.0
def test_trading_service_financials_endpoint(monkeypatch):
monkeypatch.setattr(
"backend.domains.trading.get_financials_payload",
lambda ticker, end_date, period, limit: {
"financial_metrics": [
FinancialMetrics(
ticker=ticker,
report_period=end_date,
period=period,
currency="USD",
market_cap=123.0,
enterprise_value=None,
price_to_earnings_ratio=None,
price_to_book_ratio=None,
price_to_sales_ratio=None,
enterprise_value_to_ebitda_ratio=None,
enterprise_value_to_revenue_ratio=None,
free_cash_flow_yield=None,
peg_ratio=None,
gross_margin=None,
operating_margin=None,
net_margin=None,
return_on_equity=None,
return_on_assets=None,
return_on_invested_capital=None,
asset_turnover=None,
inventory_turnover=None,
receivables_turnover=None,
days_sales_outstanding=None,
operating_cycle=None,
working_capital_turnover=None,
current_ratio=None,
quick_ratio=None,
cash_ratio=None,
operating_cash_flow_ratio=None,
debt_to_equity=None,
debt_to_assets=None,
interest_coverage=None,
revenue_growth=None,
earnings_growth=None,
book_value_growth=None,
earnings_per_share_growth=None,
free_cash_flow_growth=None,
operating_income_growth=None,
ebitda_growth=None,
payout_ratio=None,
earnings_per_share=None,
book_value_per_share=None,
free_cash_flow_per_share=None,
)
]
},
)
with TestClient(create_app()) as client:
response = client.get(
"/api/financials",
params={"ticker": "AAPL", "end_date": "2026-03-20"},
)
assert response.status_code == 200
assert response.json()["financial_metrics"][0]["ticker"] == "AAPL"
def test_trading_service_news_and_insider_endpoints(monkeypatch):
monkeypatch.setattr(
"backend.domains.trading.get_news_payload",
lambda ticker, end_date, start_date=None, limit=1000: {
"news": [
CompanyNews(
ticker=ticker,
title="News title",
source="polygon",
url="https://example.com/news",
date=end_date,
)
]
},
)
monkeypatch.setattr(
"backend.domains.trading.get_insider_trades_payload",
lambda ticker, end_date, start_date=None, limit=1000: {
"insider_trades": [
InsiderTrade(ticker=ticker, filing_date=end_date)
]
},
)
with TestClient(create_app()) as client:
news_response = client.get(
"/api/news",
params={"ticker": "AAPL", "end_date": "2026-03-20"},
)
insider_response = client.get(
"/api/insider-trades",
params={"ticker": "AAPL", "end_date": "2026-03-20"},
)
assert news_response.status_code == 200
assert news_response.json()["news"][0]["title"] == "News title"
assert insider_response.status_code == 200
assert insider_response.json()["insider_trades"][0]["ticker"] == "AAPL"
def test_trading_service_market_status_endpoint(monkeypatch):
class _FakeMarketService:
def get_market_status(self):
return {"status": "open", "status_text": "Open"}
monkeypatch.setattr(
"backend.domains.trading.get_market_status_payload",
lambda: _FakeMarketService().get_market_status(),
)
with TestClient(create_app()) as client:
response = client.get("/api/market/status")
assert response.status_code == 200
assert response.json() == {"status": "open", "status_text": "Open"}
def test_trading_service_market_cap_endpoint(monkeypatch):
monkeypatch.setattr(
"backend.domains.trading.get_market_cap_payload",
lambda ticker, end_date: {
"ticker": ticker,
"end_date": end_date,
"market_cap": 3.5e12,
},
)
with TestClient(create_app()) as client:
response = client.get(
"/api/market-cap",
params={"ticker": "AAPL", "end_date": "2026-03-20"},
)
assert response.status_code == 200
assert response.json() == {
"ticker": "AAPL",
"end_date": "2026-03-20",
"market_cap": 3.5e12,
}
def test_trading_service_line_items_endpoint(monkeypatch):
monkeypatch.setattr(
"backend.domains.trading.get_line_items_payload",
lambda ticker, line_items, end_date, period, limit: {
"search_results": [
LineItem(
ticker=ticker,
report_period=end_date,
period=period,
currency="USD",
free_cash_flow=123.0,
)
]
},
)
with TestClient(create_app()) as client:
response = client.get(
"/api/line-items",
params=[
("ticker", "AAPL"),
("line_items", "free_cash_flow"),
("end_date", "2026-03-20"),
],
)
assert response.status_code == 200
assert response.json()["search_results"][0]["ticker"] == "AAPL"
assert response.json()["search_results"][0]["free_cash_flow"] == 123.0

View File

@@ -3,13 +3,16 @@
# pylint: disable=C0301
"""Data fetching tools backed by the unified provider router."""
import datetime
import os
import httpx
import pandas as pd
import pandas_market_calendars as mcal
from backend.data.provider_utils import normalize_symbol
from backend.data.cache import get_cache
from backend.data.provider_router import get_provider_router
from backend.data.schema import (
from shared.schema import (
CompanyNews,
FinancialMetrics,
InsiderTrade,
@@ -23,6 +26,31 @@ _cache = get_cache()
_router = get_provider_router()
def _service_name() -> str:
return str(os.getenv("SERVICE_NAME", "")).strip().lower()
def _trading_service_url() -> str | None:
value = str(os.getenv("TRADING_SERVICE_URL", "")).strip().rstrip("/")
if not value or _service_name() == "trading_service":
return None
return value
def _news_service_url() -> str | None:
value = str(os.getenv("NEWS_SERVICE_URL", "")).strip().rstrip("/")
if not value or _service_name() == "news_service":
return None
return value
def _service_get_json(base_url: str, path: str, *, params: dict[str, object]) -> dict:
with httpx.Client(base_url=base_url, timeout=30.0) as client:
response = client.get(path, params=params)
response.raise_for_status()
return response.json()
def get_last_tradeday(date: str) -> str:
"""
Get the previous trading day for the specified date
@@ -104,6 +132,24 @@ def get_prices(
if cached_data := _cache.get_prices(cache_key):
return [Price(**price) for price in cached_data]
service_url = _trading_service_url()
if service_url:
try:
payload = _service_get_json(
service_url,
"/api/prices",
params={
"ticker": ticker,
"start_date": start_date,
"end_date": end_date,
},
)
prices = [Price(**price) for price in payload.get("prices", [])]
if prices:
return prices
except Exception as exc:
logger.info("Trading service price lookup failed for %s: %s", ticker, exc)
try:
prices, data_source = _router.get_prices(ticker, start_date, end_date)
except Exception as exc:
@@ -146,6 +192,28 @@ def get_financial_metrics(
if cached_data := _cache.get_financial_metrics(cache_key):
return [FinancialMetrics(**metric) for metric in cached_data]
service_url = _trading_service_url()
if service_url:
try:
payload = _service_get_json(
service_url,
"/api/financials",
params={
"ticker": ticker,
"end_date": end_date,
"period": period,
"limit": limit,
},
)
metrics = [
FinancialMetrics(**metric)
for metric in payload.get("financial_metrics", [])
]
if metrics:
return metrics
except Exception as exc:
logger.info("Trading service financial lookup failed for %s: %s", ticker, exc)
try:
financial_metrics, data_source = _router.get_financial_metrics(
ticker=ticker,
@@ -183,6 +251,22 @@ def search_line_items(
ticker = normalize_symbol(ticker)
if not ticker:
return []
service_url = _trading_service_url()
if service_url:
payload = _service_get_json(
service_url,
"/api/line-items",
params={
"ticker": ticker,
"line_items": line_items,
"end_date": end_date,
"period": period,
"limit": limit,
},
)
return [LineItem(**item) for item in payload.get("search_results", [])]
return _router.search_line_items(
ticker=ticker,
line_items=line_items,
@@ -213,6 +297,26 @@ def get_insider_trades(
if cached_data := _cache.get_insider_trades(cache_key):
return [InsiderTrade(**trade) for trade in cached_data]
service_url = _trading_service_url()
if service_url:
try:
params = {"ticker": ticker, "end_date": end_date, "limit": limit}
if start_date:
params["start_date"] = start_date
payload = _service_get_json(
service_url,
"/api/insider-trades",
params=params,
)
trades = [
InsiderTrade(**trade)
for trade in payload.get("insider_trades", [])
]
if trades:
return trades
except Exception as exc:
logger.info("Trading service insider lookup failed for %s: %s", ticker, exc)
try:
all_trades, data_source = _router.get_insider_trades(
ticker=ticker,
@@ -248,6 +352,40 @@ def get_company_news(
if cached_data := _cache.get_company_news(cache_key):
return [CompanyNews(**news) for news in cached_data]
trading_service_url = _trading_service_url()
if trading_service_url:
try:
params = {"ticker": ticker, "end_date": end_date, "limit": limit}
if start_date:
params["start_date"] = start_date
payload = _service_get_json(
trading_service_url,
"/api/news",
params=params,
)
news = [CompanyNews(**item) for item in payload.get("news", [])]
if news:
return news
except Exception as exc:
logger.info("Trading service news lookup failed for %s: %s", ticker, exc)
news_service_url = _news_service_url()
if news_service_url:
try:
params = {"ticker": ticker, "end_date": end_date, "limit": limit}
if start_date:
params["start_date"] = start_date
payload = _service_get_json(
news_service_url,
"/api/enriched-news",
params=params,
)
news = [CompanyNews(**item) for item in payload.get("news", [])]
if news:
return news
except Exception as exc:
logger.info("News service lookup failed for %s: %s", ticker, exc)
try:
all_news, data_source = _router.get_company_news(
ticker=ticker,
@@ -272,6 +410,19 @@ def get_market_cap(ticker: str, end_date: str) -> float | None:
if not ticker:
return None
service_url = _trading_service_url()
if service_url:
try:
payload = _service_get_json(
service_url,
"/api/market-cap",
params={"ticker": ticker, "end_date": end_date},
)
value = payload.get("market_cap")
return float(value) if value is not None else None
except Exception as exc:
logger.info("Trading service market-cap lookup failed for %s: %s", ticker, exc)
def _metrics_lookup(symbol: str, date: str):
for source in _router.api_sources():
cache_key = f"{symbol}_ttm_{date}_10_{source}"

Some files were not shown because too many files have changed in this diff Show More