cillin/evotraders
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

init

Browse Source
This commit is contained in:
raykkk
2025-10-17 21:40:45 +08:00
commit 7d0451131f
155 changed files with 14873 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

0
functionality/__init__.py Normal file
View File

325
functionality/long_term_memory_mem0/README.md Normal file
View File

@@ -0,0 +1,325 @@
# Mem0LongTermMemory
## Overview
**Note**: We are working on merging Mem0LongTermMemory into the main AgentScope repository.
Mem0LongTermMemory is a long-term memory implementation built on top of the mem0 library, designed to provide persistent, semantic memory storage for AgentScope agents. It enables agents to record, store, and retrieve conversation history, reasoning processes, and contextual information across sessions, supporting advanced memory management and knowledge retention.
This example demonstrates how to use Mem0LongTermMemory to create persistent memory systems that can store and retrieve information based on semantic similarity, enabling agents to maintain context and learn from past interactions.
## Core Features
### Persistent Memory Storage
- **Vector-based Storage**: Uses Qdrant vector database for efficient semantic search and retrieval
- **Configurable Backends**: Support for multiple embedding models (OpenAI, DashScope) and vector stores
- **Async Operations**: Full async support for non-blocking memory operations
### Semantic Memory Management
- **Content Recording**: Store conversation messages, tool usage, and reasoning processes
- **Thinking Integration**: Record agent thinking processes alongside content for better context
- **Flexible Input Formats**: Support for strings, Msg objects, and dictionaries
### Agent Integration
- **Direct AgentScope Integration**: Seamless integration with AgentScope's ReActAgent
- **Memory Modes**: Support for agent_control, dev_control, and both modes
- **Tool Response Format**: Returns structured ToolResponse objects for easy integration
## File Structure
```
memory_by_mem0/
├── README.md # This documentation file
├── long_term_memory_by_mem0.py # Core Mem0LongTermMemory implementation
├── memory_example.py # Standalone examples demonstrating memory operations
├── conversation_agent_with_longterm_mem.py # Interactive conversation example with ReActAgent
└── utils.py # AgentScope integration utilities for mem0
```
## Prerequisites
### Clone the AgentScope Repository
This example depends on AgentScope. Please clone the full repository.
### Install Dependencies
**Recommended**: Python 3.10+
Install the following dependencies:
```bash
pip install mem0ai
```
### API Keys
By default, the example uses DashScope/OpenAI for embedding and LLM. Set your API key:
```bash
export DASHSCOPE_API_KEY='YOUR_API_KEY'
export DASHSCOPE_API_BASE_URL='YOUR_API_BASE_URL'
export DASHSCOPE_MODEL_4_MEMORY='USED_MODEL_NAME'
export DASHSCOPE_EMBEDDING_MODEL='text-embedding-v2'
```
## How It Works
### 1. Configuration
The memory system uses a `MemoryConfig` that specifies:
- **Embedder**: Configuration for embedding models (OpenAI, DashScope)
- **LLM**: Configuration for language models used in memory processing
- **Vector Store**: Configuration for vector database (Qdrant with on-disk storage as default)
### 2. Memory Structure
- **Mem0LongTermMemory**: Inherits from `LongTermMemoryBase` and maintains an async memory server
- **Single AsyncMemory Instance**: Uses one AsyncMemory instance for all storage and retrieval operations
- **Agent/User Context**: Maintains separate memory spaces for different agent-user combinations
### 3. Memory Recording Flow
1. **Input Processing**: Formats various input types (strings, Msg objects, dictionaries) into standardized format
2. **Content Combination**: Merges thinking processes with content for comprehensive memory storage
3. **Vector Storage**: Stores processed content with metadata in the vector database
4. **Response Formatting**: Returns structured ToolResponse objects for easy integration
### 4. Memory Retrieval Flow
1. **Semantic Search**: Performs vector similarity search in the memory database
2. **Response Formatting**: Returns retrieved memories in structured format
## Usage Examples
### Basic Usage
Run the standalone memory examples to see the complete memory operations:
```bash
python ./memory_example.py
```
### Example Scenarios
The example demonstrates several typical use cases:
1. **Basic Conversation Recording**: Store simple user-agent conversations
2. **Tool Usage and Results**: Record tool usage with thinking processes
3. **Multi-step Reasoning**: Store complex reasoning processes step by step
4. **Error Handling**: Record error scenarios and recovery strategies
5. **User Preferences**: Store user preferences and contextual information
## API Reference
### Mem0LongTermMemory Class
#### Main Methods
**`__init__(agent_name=None, user_name=None, run_name=None, model=None, embedding_model=None, vector_store_config=None, mem0_config=None, default_memory_type=None, **kwargs)`**
- Initialize the memory instance with agent, user, and run context
- `agent_name` (str, optional): The name of the agent
- `user_name` (str, optional): The name of the user
- `run_name` (str, optional): The name of the run/session
- `model` (ChatModelBase, optional): The model to use for the long-term memory
- `embedding_model` (EmbeddingModelBase, optional): The embedding model to use
- `vector_store_config` (VectorStoreConfig, optional): Vector store configuration
- `mem0_config` (MemoryConfig, optional): Complete mem0 configuration
- `default_memory_type` (str, optional): Default memory type for storage
**Note**:
1. At least one of `agent_name`, `user_name`, or `run_name` is required.
2. During memory recording, these parameters become metadata for the stored memories.
3. During memory retrieval, only memories with matching metadata values will be returned.
**`record_to_memory(thinking, content, memory_type=None, **kwargs)`**
- Record content with thinking process
- `thinking` (str): Your thinking and reasoning about what to record
- `content` (list[str]): The content to remember, which is a list of strings
- `memory_type` (str, optional): The type of memory to use. Default is None, to create a semantic memory. "procedural_memory" is explicitly used for procedural memories
- Returns: ToolResponse with success/error status
**`retrieve_from_memory(keywords, **kwargs)`**
- Retrieve memories based on keywords
- `keywords` (list[str]): Keywords to search for in the memory, which should be specific and concise, e.g. the person's name, the date, the location, etc.
- `limit_per_search` (int): Number of memories to retrieve per search (default: 5)
- Returns: ToolResponse with retrieved memories
#### Internal Methods
**`record(msgs, **kwargs)`**
- Record message sequences to memory
- `msgs` (Sequence[Msg | None]): Messages to record
**`_record_all(content, thinking=None, memory_type=None, infer=True, **kwargs)`**
- Record content with comprehensive processing
- `content` (list[str] | list[Msg] | list[dict]): The content to remember, which is a list of strings or Msg objects or dict objects
- `thinking` (str, optional): Your thinking and reasoning about what to record, if not provided, the content will be used as the thinking
- `memory_type` (str, optional): The type of memory to use. Default is None, to create a semantic memory. "procedural_memory" is explicitly used for procedural memories
- `infer` (bool): Whether to infer memory type (default: True)
- Handles various input formats and thinking integration
**`retrieve(msg, **kwargs)`**
- Retrieve memories based on message content
- `msg` (Msg | list[Msg] | None): The message to search for in the memory, which should be specific and concise, e.g. the person's name, the date, the location, etc.
- `limit_per_search` (int): Number of results per search (default: 5)
- Returns: list[str] - A list of retrieved memory strings
### Configuration
#### Direct Model Configuration
```python
# Initialize with AgentScope models directly
long_term_memory = Mem0LongTermMemory(
agent_name="Friday",
user_name="user_123",
model=OpenAIChatModel(
model_name="gpt-4",
api_key="your_api_key",
base_url="your_base_url"
),
embedding_model=OpenAITextEmbedding(
model_name="text-embedding-3-small",
api_key="your_api_key",
base_url="your_base_url"
)
)
```
## Customization & Extension
### Backend Replacement
Easily customize embedding, LLM, or vector store by modifying the configuration:
```python
# Example: Using different embedding model
embedder=EmbedderConfig(
provider="dashscope",
config={
"model": "text-embedding-v1",
"api_key": "your_dashscope_key"
}
)
```
### Memory Config Replacement
Mem0LongTermMemory supports directly receiving memory configurations defined in mem0, allowing users to easily adopt various memory configurations and backends supported by mem0. This provides flexibility to use different embedding models, LLMs, and vector stores without modifying the core implementation.
```python
# Example: Using a complete mem0 MemoryConfig
from mem0.configs.base import MemoryConfig
from mem0.embeddings.configs import EmbedderConfig
from mem0.llms.configs import LlmConfig
from mem0.vector_stores.configs import VectorStoreConfig
# Create a custom mem0 configuration
mem0_config = MemoryConfig(
embedder=EmbedderConfig(
provider="openai",
config={
"model": "text-embedding-3-small",
"api_key": "your_openai_key"
}
),
llm=LlmConfig(
provider="openai",
config={
"model": "gpt-4",
"api_key": "your_openai_key"
}
),
vector_store=VectorStoreConfig(
provider="qdrant",
config={
"on_disk": True,
"path": "./memory_data"
}
)
)
# Initialize with the custom mem0 configuration
long_term_memory = Mem0LongTermMemory(
agent_name="Friday",
user_name="user_123",
mem0_config=mem0_config
)
```
**Note**: In Mem0LongTermMemory, if the `model`, `embedding_model`, or `vector_store_config` parameters are not None, they will override the corresponding configurations in `mem0_config`. This allows for flexible configuration where you can use a base mem0 configuration and selectively override specific components.
### AgentScope Integration
The implementation includes custom AgentScope providers for mem0:
- **AgentScopeLLM**: Integrates AgentScope ChatModelBase with mem0
- **AgentScopeEmbedding**: Integrates AgentScope EmbeddingModelBase with mem0
These providers handle the conversion between mem0's expected format and AgentScope's message/response formats.
### Memory Type Customization
Add custom memory types for different use cases:
```python
# Example: Procedural memory
await memory.record_to_memory(
content=["Step 1: Analyze input", "Step 2: Process data"],
thinking="This is a procedural workflow for data processing",
memory_type="procedural_memory"
)
```
## Best Practices
### Memory Recording
1. **Be Specific**: Record specific, actionable information rather than general statements
2. **Include Context**: Always include relevant context and reasoning when recording
3. **Use Thinking**: Leverage the thinking parameter to explain why information is important
4. **Structured Content**: Use structured formats for complex information
### Memory Retrieval
1. **Specific Keywords**: Use specific, relevant keywords for better search results
2. **Appropriate Limits**: Set reasonable limits based on your use case
3. **Context Awareness**: Consider the current context when retrieving memories
4. **Error Handling**: Always handle potential retrieval errors gracefully
### Performance Optimization
1. **Batch Operations**: Group related memory operations when possible
2. **Efficient Queries**: Use specific keywords to reduce search scope
3. **Memory Cleanup**: Periodically clean up irrelevant or outdated memories
4. **Configuration Tuning**: Optimize vector store and embedding configurations
## Troubleshooting
### Common Issues
**Memory Not Found**
- Check if the memory was properly recorded
- Verify agent_id and user_id consistency
- Ensure vector store is properly configured
**Poor Search Results**
- Use more specific keywords
- Check embedding model configuration
- Verify content was properly formatted during recording
**Performance Issues**
- Optimize vector store configuration
- Reduce search limits
- Consider using on-disk storage for large datasets
**AgentScope Integration Issues**
- Ensure AgentScope models are properly configured
- Check that the custom providers are registered correctly
- Verify message format compatibility
### Debug Mode
Enable debug logging to troubleshoot issues:
```python
import logging
logging.basicConfig(level=logging.DEBUG)
```
## Reference
- [mem0 Documentation](https://github.com/mem0ai/mem0)
- [AgentScope Documentation](https://github.com/agentscope-ai/agentscope)
- [Qdrant Vector Database](https://qdrant.tech/)
For further customization or integration, please refer to the full implementation in the `long_term_memory_by_mem0.py` file and the mem0 official documentation.

121
functionality/long_term_memory_mem0/memory_example.py Normal file
View File

@@ -0,0 +1,121 @@
# -*- coding: utf-8 -*-
"""Memory example demonstrating long-term memory functionality with mem0.
This module provides examples of how to use the Mem0LongTermMemory class
for recording and retrieving persistent memories.
"""
import asyncio
import os
from agentscope.agent import ReActAgent
from agentscope.embedding import DashScopeTextEmbedding
from agentscope.formatter import DashScopeChatFormatter
from agentscope.memory import InMemoryMemory, Mem0LongTermMemory
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.tool import Toolkit
from dotenv import load_dotenv
load_dotenv()
async def main() -> None:
"""Run the memory examples."""
# Initialize long term memory
long_term_memory = Mem0LongTermMemory(
agent_name="Friday",
user_name="user_123",
model=DashScopeChatModel(
model_name="qwen-max-latest",
api_key=os.environ.get("DASHSCOPE_API_KEY"),
stream=False,
),
embedding_model=DashScopeTextEmbedding(
model_name="text-embedding-v2",
api_key=os.environ.get("DASHSCOPE_API_KEY"),
),
on_disk=False,
)
print("=== Long Term Memory Examples with mem0 ===\n")
# Example 1: Basic conversation recording
print("1. Basic Conversation Recording")
print("-" * 40)
results = await long_term_memory.record(
msgs=[
Msg(
role="user",
content="Please help me book a hotel, preferably homestay",
name="user",
),
],
)
print(f"Recorded conversation: {results}\n")
# Example 2: Retrieving memories
print("2. Retrieving Memories")
print("-" * 40)
print("Searching for weather-related memories...")
weather_memories = await long_term_memory.retrieve(
msg=[
Msg(
role="user",
content="What's the weather like today?",
name="user",
),
],
)
print(f"Retrieved weather memories: {weather_memories}\n")
print("Searching for user preference memories...")
preference_memories = await long_term_memory.retrieve(
msg=[
Msg(
role="user",
content=(
"I prefer temperatures in Celsius and wind speed in km/h"
),
name="user",
),
],
)
print(f"Retrieved preference memories: {preference_memories}\n")
# Example 3: ReActAgent with long term memory
print("3. ReActAgent with long term memory")
print("-" * 40)
toolkit = Toolkit()
agent = ReActAgent(
name="Friday",
sys_prompt="You are a helpful assistant named Friday.",
model=DashScopeChatModel(
model_name="qwen-max-latest",
api_key=os.environ.get("DASHSCOPE_API_KEY"),
stream=False,
),
formatter=DashScopeChatFormatter(),
toolkit=toolkit,
memory=InMemoryMemory(),
long_term_memory=long_term_memory,
long_term_memory_mode="both",
)
await agent.memory.clear()
msg = Msg(
role="user",
content="When I travel to Hangzhou, I prefer to stay in a homestay",
name="user",
)
msg = await agent(msg)
print(f"ReActAgent response: {msg.get_text_content()}\n")
msg = Msg(role="user", content="what preference do I have?", name="user")
msg = await agent(msg)
print(f"ReActAgent response: {msg.get_text_content()}\n")
if __name__ == "__main__":
asyncio.run(main())

3
functionality/long_term_memory_mem0/requirements.txt Normal file
View File

@@ -0,0 +1,3 @@
agentscope[full]>=1.0.5
packaging>=25.0
mem0ai>=1.0.0

61
functionality/mcp/README.md Normal file
View File

@@ -0,0 +1,61 @@
# MCP in AgentScope
This example demonstrates how to
- create MCP client with different transports (SSE and Streamable HTTP) and type (Stateless and Stateful),
- register MCP tool functions and use them in a ReAct agent, and
- get MCP tool function as a local callable object from the MCP client.
## Prerequisites
- Python 3.10 or higher
- DashScope API key from Alibaba Cloud
## Installation
### Install from PyPI (Recommended)
### Install AgentScope
```bash
# Install from source
cd {PATH_TO_AGENTSCOPE}
pip install -e .
```
## QuickStart
Install agentscope and ensure you have a valid DashScope API key in your environment variables.
> Note: The example is built with DashScope chat model. If you want to change the model in this example, don't forget
> to change the formatter at the same time! The corresponding relationship between built-in models and formatters are
> list in [our tutorial](https://doc.agentscope.io/tutorial/task_prompt.html#id1)
```bash
pip install agentscope
```
Start the MCP servers by the following commands in two separate terminals:
```bash
# In one terminal, run:
python mcp_add.py
# In another terminal, run:
python mcp_multiply.py
```
Two MCP servers will be started on `http://127.0.0.1:8001` (SSE server) and `http://127.0.0.1:8002` (streamable
HTTP server).
After starting the MCP servers, you can run the agent example:
```bash
python main.py
```
The agent will:
1. Register the MCP tools from the servers
2. Use a ReAct agent to solve a calculation problem (multiplying two numbers and then adding another number)
3. Return structured output with the final result

109
functionality/mcp/main.py Normal file
View File

@@ -0,0 +1,109 @@
# -*- coding: utf-8 -*-
"""
Demo showcasing ReAct agent with MCP tools using different transports.
This example demonstrates:
- Registering MCP tools with different transports (sse and streamable_http)
- Using a ReAct agent with registered MCP tools
- Getting structured output from the agent
Before running this demo, please execute:
python mcp_servers.py
"""
import asyncio
import json
import os
from agentscope.agent import ReActAgent
from agentscope.formatter import DashScopeChatFormatter
from agentscope.mcp import HttpStatefulClient, HttpStatelessClient
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.tool import Toolkit
from pydantic import BaseModel, Field
class NumberResult(BaseModel):
"""A simple number result model for structured output."""
result: int = Field(description="The result of the calculation")
async def main() -> None:
"""The main entry of the MCP example."""
toolkit = Toolkit()
# Create a stateful MCP client to connect to the SSE MCP server
# note you can also use the stateless client
add_mcp_client = HttpStatefulClient(
name="add_mcp",
transport="sse",
url="http://127.0.0.1:8001/sse",
)
# Create a stateless MCP client to connect to the StreamableHTTP MCP server
# note you can also use the stateful client
multiply_mcp_client = HttpStatelessClient(
name="multiply_mcp",
transport="streamable_http",
url="http://127.0.0.1:8002/mcp",
)
# The stateful client must be connected before using
await add_mcp_client.connect()
# Register the MCP clients to the toolkit
await toolkit.register_mcp_client(add_mcp_client)
await toolkit.register_mcp_client(multiply_mcp_client)
# Initialize the agent
agent = ReActAgent(
name="Jarvis",
sys_prompt="You're a helpful assistant named Jarvis.",
model=DashScopeChatModel(
model_name="qwen-max",
api_key=os.environ["DASHSCOPE_API_KEY"],
),
formatter=DashScopeChatFormatter(),
toolkit=toolkit,
)
# Run the agent with a calculation task
res = await agent(
Msg(
"user",
"Calculate 2345 multiplied by 3456, then add 4567 to the result,"
" what is the final outcome?",
"user",
),
structured_model=NumberResult,
)
print(
"Structured Output:\n"
"```\n"
f"{json.dumps(res.metadata, indent=4, ensure_ascii=False)}\n"
"```",
)
# AgentScope also allows developers to obtain the MCP tool as a local
# callable object, and use it directly.
add_tool_function = await add_mcp_client.get_callable_function(
"add",
# If wrap the MCP tool result into the ToolResponse object in
# AgentScope
wrap_tool_result=True,
)
# Call it manually
manual_res = await add_tool_function(a=5, b=10)
print("When manually calling the MCP tool function:")
print(manual_res)
# The stateful client should be disconnected manually!
await add_mcp_client.close()
asyncio.run(main())

15
functionality/mcp/mcp_add.py Normal file
View File

@@ -0,0 +1,15 @@
# -*- coding: utf-8 -*-
"""An SSE MCP server with a simple add tool function."""
from mcp.server import FastMCP
mcp = FastMCP("Add", port=8001)
@mcp.tool()
def add(a: int, b: int) -> int:
"""Add two numbers."""
return a + b
mcp.run(transport="sse")

15
functionality/mcp/mcp_multiply.py Normal file
View File

@@ -0,0 +1,15 @@
# -*- coding: utf-8 -*-
"""An SSE MCP server with a simple multiply tool function."""
from mcp.server import FastMCP
mcp = FastMCP("Multiply", port=8002)
@mcp.tool()
def multiply(c: int, d: int) -> int:
"""Multiply two numbers."""
return c * d
mcp.run(transport="streamable-http")

1
functionality/mcp/requirements.txt Normal file
View File

@@ -0,0 +1 @@
agentscope[full]>=1.0.5

155
functionality/meta_planner_agent/README.md Normal file
View File

@@ -0,0 +1,155 @@
# Meta Planner Agent Example
An advanced AI agent example that demonstrates sophisticated task planning and execution capabilities using AgentScope. The Meta Planner breaks down complex tasks into manageable subtasks and orchestrates specialized worker agents to complete them efficiently.
## Overview
The Meta Planner agent is designed to handle complex, multi-step tasks that would be difficult for a simple agent to manage directly. It uses a planning-execution pattern where:
1. **Complex tasks are decomposed** into smaller, manageable subtasks
2. **Worker agents can be dynamically created** with appropriate tools for each subtask
3. **Progress is tracked and managed** through a roadmap system
4. **Results are coordinated** to achieve the overall goal
This approach enables handling sophisticated workflows like data analysis, research projects, content creation, and multi-step problem solving.
## Key Features
- **Intelligent Task Decomposition**: Automatically breaks down complex requests into executable subtasks
- **Progress Tracking**: Maintains a structured roadmap with status tracking for all subtasks
- **Dynamic Worker Management**: Creates and manages specialized worker agents with relevant toolkits
- **State Persistence**: Saves and restores agent state for long-running tasks
- **Flexible Modes**: Can operate in simple ReAct mode or advanced planning mode based on task complexity
## Architecture
### Core Components
1. **MetaPlanner** (`_meta_planner.py`): The main agent class that extends ReActAgent with planning capabilities
2. **Planning Tools** (`_planning_tools/`):
- `PlannerNoteBook`: Manages session context and user inputs
- `RoadmapManager`: Handles task decomposition and progress tracking
- `WorkerManager`: Creates and manages worker agents
3. **System Prompts** (`_built_in_long_sys_prompt/`): Detailed instructions for (worker) agent behavior
4. **Demo Entry Point** (`main.py`): The main function to start the application with meta planner agent.
## Prerequisites for Running This Example
### Required Environment Variables
```bash
# Anthropic API key for the Claude model
export ANTHROPIC_API_KEY="your_anthropic_api_key"
# Tavily API key for search functionality
export TAVILY_API_KEY="your_tavily_api_key"
```
### Optional Environment Variables
```bash
# Custom working directory for agent operations (default: ./meta_agent_demo_env)
export AGENT_OPERATION_DIR="/path/to/custom/working/directory"
```
## Usage
### Basic Usage
Run the agent interactively:
```bash
cd examples/agent_meta_planner
python main.py
```
The agent will start in chat mode where you can provide complex tasks. For example:
- "Create a comprehensive analysis of Meta's stock performance in Q1 2025"
- "Research and write a 7-day exercise plan with detailed instructions"
- "Analyze the latest AI trends and create a summary report"
### Example Interactions
1. **Data Analysis Task**:
```
User: "Analyze the files in my directory and create a summary report"
```
2. **Research Task**:
```
User: "Research Alibaba's latest quarterly results and competitive position"
```
## Configuration
### Agent Modes
The Meta Planner supports three operation modes:
- **`dynamic`** (default): Automatically switches between simple ReAct and planning mode based on task complexity
- **`enforced`**: Always uses planning mode for all tasks
- **`disable`**: Only uses simple ReAct mode (no planning capabilities)
### Tool Configuration
The agent uses two main toolkits:
1. **Planner Toolkit**: Planning-specific tools for task decomposition and worker management
2. **Worker Toolkit**: Comprehensive tools including:
- Shell command execution
- File operations
- Web search (via Tavily)
- Filesystem access (via MCP)
### State Management
Agent states are automatically saved during execution:
- **Location**: `./agent-states/run-YYYYMMDDHHMMSS/`
- **Types**:
- `state-post_reasoning-*.json`: After reasoning steps
- `state-post-action-{tool_name}-*.json`: After tool executions
### State Recovery
If an agent gets stuck or fails:
1. Check the latest state file in `./agent-states/`
2. Resume from the last successful state:
```bash
python main.py --load_state path/to/state/file.json
```
## Advanced Customization
### Adding New Tools
1. Create tool functions following AgentScope patterns
2. Register tools in the appropriate toolkit:
```python
worker_toolkit.register_tool_function(your_custom_tool)
```
### Custom MCP Clients
Add additional MCP clients in `main.py`:
```python
mcp_clients.append(
StdIOStatefulClient(
name="custom_mcp",
command="npx",
args=["-y", "your-mcp-server"],
env={"API_KEY": "your_key"},
)
)
```
### System Prompt Modifications
Modify prompts in `_built_in_long_sys_prompt/` to customize agent behavior.

0
functionality/meta_planner_agent/__init__.py Normal file
View File

11
functionality/meta_planner_agent/_built_in_long_sys_prompt/_tool_usage_rules.md Normal file
View File

@@ -0,0 +1,11 @@
### Tool usage rules
1. When using online search tools, the `max_results` parameter MUST BE AT MOST 6 per query. Try to avoid include raw content when call the search.
2. The directory/file system that you can operate is the following path: {agent_working_dir}. DO NOT try to save/read/modify file in other directories.
3. Try to use the local resource before going to online search. If there is file in PDF format, first convert it to markdown or text with tools, then read it as text.
4. NEVER use `read_file` tool to read PDF file directly.
5. DO NOT targeting at generating PDF file unless the user specifies.
6. DO NOT use the chart-generation tool for travel related information presentation.
7. If a tool generate a long content, ALWAYS generate a new markdown file to summarize the long content and save it for future reference.
8. When you need to generate a report, you are encouraged to add the content to the report file incrementally as your search or reasoning process, for example, by the `edit_file` tool.
9. When you use the `write_file` tool, you **MUST ALWAYS** remember to provide the both the `path` and `content` parameters. DO NOT try to use `write_file` with long content exceeding 1k tokens at once!!!

22
functionality/meta_planner_agent/_built_in_long_sys_prompt/_worker_additional_sys_prompt.md Normal file
View File

@@ -0,0 +1,22 @@
## Additional Operation Notice
### Checklist Management
1. You will receive a markdown-style checklist (i.e., "Expected Output" checklist) in your input instruction. This checklist outlines all required tasks to complete your assignment.
2. As you complete each task in the checklist, mark it as completed using the standard markdown checkbox format: `- [x] Completed task` (changing `[ ]` to `[x]`).
3. Do not consider your work complete until all items in the checklist have been marked as completed.
### Process Flow
1. Work through the checklist methodically, addressing each item in a logical sequence.
2. For each item, document your reasoning and actions taken to complete it.
3. If you cannot complete an item due to insufficient information, clearly note what additional information you need.
### Completion and Output
1. Once all checklist items are completed (or you've determined that additional information is required), use the `generate_response` tool to submit your work to the meta planner.
### Technical Constraints
1. If you need to generate a long report with a long content, generate it step by step: first use `write_file` with BOTH `path` and `content` (the structure or skeleton of the report in string) and later use the `edit_file` tool to gradually fill in content. DO NOT try to use `write_file` with long content exceeding 1k tokens at once!!!
### Progress Tracking
1. Regularly review the checklist to confirm your progress.
2. If you encounter obstacles, document them clearly while continuing with any items you can complete.

56
functionality/meta_planner_agent/_built_in_long_sys_prompt/meta_planner_sys_prompt.md Normal file
View File

@@ -0,0 +1,56 @@
## Identity
You are ASAgent, a multifunctional agent that can help people solving different complex tasks. You act like a meta planner to solve complicated tasks by decomposing the task and building/orchestrating different worker agents to finish the sub-tasks.
## Core Mission
Your primary purpose is to break down complicated tasks into manageable subtasks, build appropriate worker agents for each subtask, and coordinate their execution to achieve the user's goal efficiently.
### Operation Paradigm
You are provided some tools/functions that can be considered operations in solving tasks requiring multi-stage to solve. The key functionalities include clarifying task ambiguities, decomposing tasks into executable subtasks, building worker agents, and orchestrating them to solve the subtasks one by one.
1. **Task Decomposition**: With a well-defined and no ambiguous task:
- You need to build a structured roadmap by calling `decompose_task_and_build_roadmap` before proceeding to the following steps. DO NOT break the task into too detailed subtasks. It is acceptable that each subtask can be done 10-15 steps of tool calling and reasoning.
- Once you have the roadmap, you must consider how to finish the subtask following the roadmap.
- After a subtask is done, you can use `get_next_unfinished_subtask_from_roadmap` to obtain a reminder about what is the next unfinished subtask.
2. **Worker Agent Selection/Creation**: For each subtask, determine if an existing worker can handle it:
- You can use `show_current_worker_pool` to check whether there are appropriate worker that have already been created in the worker pool.
- If no suitable worker exists, create new one with `create_worker` tool.
3. **Subtask Execution**: With the decomposed sub-tasks, you need to execute the worker agent by `execute_worker`.
4. **Progress Tracking**: After you execute a worker agent and receive ANY response from the worker:
- You MUST USE `revise_roadmap` to revise the progress, update the roadmap for solving the following subtask (for example, update the input and output
- make sure the plan can still solve the original given task.
5. When all the sub-tasks are solved (marked as `Done`), call `generate_response`.
### Important Constraints
1. You MUST provide a reason to explain why to you call a function / use a tool.
2. DO NOT TRY TO SOLVE THE SUBTASKS DIRECTLY yourself.
3. ONLY do reasoning and select functions to coordinate.
4. DO NOT synthesize function return results.
5. Always follow the roadmap sequence.
6. DO NOT finish until all subtasks are marked with \"Done\" after revising the roadmap.
7. DO NOT read user's provided file directly. Instead, create a worker to do so for you.
### Error Handling
In case you encounter any error when you use tools (building/orchestrating workers):
1. If a worker marks its subtask as unfinished or in-process, pay attention to the `progress_summary` information in their response:
- If the worker requests more information to finish the subtask, and you have enough information, call `revise_roadmap` to improve the input with the exact information to the worker, and `execute_worker` again.
- If the worker fails with errors, then try to creat new worker agent to solve the task.
## Example Flow
Task: "Create a data visualization from my sales spreadsheet"
1. Clarify specifics (visualization type, data points of interest)
2. Build roadmap (data loading, cleaning, analysis, visualization, export)
3. Create/select appropriate workers for the i-th subtask (e.g., data searcher or processor)
4. Execute worker for the i-th subtask, revising roadmap after the worker finishes
5. Repeat step 3 and 4 until all subtasks are mark as "Done"
5. Generate final response with visualization results
## Auxiliary Information Usage
You will be provided with a "session environment" with information that may be useful. The auxiliary information includes:
* **Time**: the current operation time that you need to consider, especially for those tasks requiring the latest information;
* **User input**: a list of strings including the user's initial input and follow-up requirements and adjustments;
* **Detail_analysis_for_plan**: a detailed analysis of the given task and a plan to solve it in natural language;
* **Roadmap**: a plan with subtasks status tracking to solve the task in JSON format;
* **Files**: available files that may fall into the following categories 1) provided by the user as part of the task, 2) generated by some worker agent in the process of solving subtasks, 3) subtasks finish report;
* **User preferences**: a set of records of the user's personal preferences, which may contain information such as the preferred format output, usual location, etc.
## Available Tools for workers
{tool_list}

517
functionality/meta_planner_agent/_meta_planner.py Normal file
View File

@@ -0,0 +1,517 @@
# -*- coding: utf-8 -*-
"""
Meta Planner agent class that can handle complicated tasks with
planning-execution pattern.
"""
import json
import os
import uuid
from datetime import datetime
from pathlib import Path
from typing import Any, Literal, Optional
from _planning_tools import (
PlannerNoteBook, # pylint: disable=C0411
RoadmapManager,
WorkerManager,
share_tools,
)
from agentscope.agent import ReActAgent
from agentscope.formatter import FormatterBase
from agentscope.memory import MemoryBase
from agentscope.message import Msg, TextBlock, ToolResultBlock, ToolUseBlock
from agentscope.model import ChatModelBase
from agentscope.tool import Toolkit, ToolResponse
PlannerStage = Literal["post_reasoning", "post_action", "pre_reasoning"]
def _infer_planner_stage_with_msg(
cur_msg: Msg,
) -> tuple[PlannerStage, list[str]]:
"""
Infer the planner stage and extract tool names from a message.
Analyzes a message to determine the current stage of the planner workflow
and extracts any tool names if tool calls are present in the message.
Args:
cur_msg (Msg): The message to analyze for stage inference.
Returns:
tuple[PlannerStage, list[str]]: A tuple containing:
- PlannerStage: One of "pre_reasoning", "post_reasoning", or
"post_action"
- list[str]: List of tool names found in tool_use or
tool_result blocks
Note:
- "pre_reasoning": System role messages with string content
- "post_reasoning": Messages with tool_use blocks or plain text content
- "post_action": Messages with tool_result blocks
- Tool names are extracted from both tool_use and tool_result blocks
"""
blocks = cur_msg.content
if isinstance(blocks, str) and cur_msg.role in ["system", "user"]:
return "pre_reasoning", []
cur_tool_names = [
str(b.get("name", "no_name_tool"))
for b in blocks
if b["type"] in ["tool_use", "tool_result"]
]
if cur_msg.has_content_blocks("tool_result"):
return "post_action", cur_tool_names
elif cur_msg.has_content_blocks("tool_use"):
return "post_reasoning", cur_tool_names
else:
return "post_reasoning", cur_tool_names
def update_user_input_pre_reply_hook(
self: "MetaPlanner",
kwargs: dict[str, Any],
) -> None:
"""Hook for loading user input to planner notebook"""
msg = kwargs.get("msg", None)
if isinstance(msg, Msg):
msg = [msg]
if isinstance(msg, list):
for m in msg:
self.planner_notebook.user_input.append(m.content)
def planner_save_post_reasoning_state(
self: "MetaPlanner",
reasoning_input: dict[str, Any], # pylint: disable=W0613
reasoning_output: Msg,
) -> None:
"""Hook func for save state after reasoning step"""
if self.state_saving_dir:
os.makedirs(self.state_saving_dir, exist_ok=True)
cur_stage, _ = _infer_planner_stage_with_msg(reasoning_output)
time_str = datetime.now().strftime("%Y%m%d%H%M%S")
file_path = os.path.join(
self.state_saving_dir,
f"state-{cur_stage}-{time_str}.json",
)
with open(file_path, "w", encoding="utf-8") as f:
json.dump(self.state_dict(), f, ensure_ascii=False, indent=4)
async def planner_load_state_pre_reasoning_hook(
self: "MetaPlanner", # pylint: disable=W0613
*args: Any,
**kwargs: Any,
) -> None:
"""Hook func for loading saved state after reasoning step"""
mem_msgs = await self.memory.get_memory()
if len(mem_msgs) > 0:
stage, _ = _infer_planner_stage_with_msg(mem_msgs[-1])
if stage == "post_reasoning":
self.state_loading_reasoning_msg = mem_msgs[-1]
# delete the last reasoning message to avoid error when
# calling model in reasoning step
await self.memory.delete(len(mem_msgs) - 1)
async def planner_load_state_post_reasoning_hook(
self: "MetaPlanner", # pylint: disable=W0613
*args: Any,
**kwargs: Any,
) -> Msg:
"""Hook func for loading saved state after reasoning step"""
if self.state_loading_reasoning_msg is not None:
num_msgs = await self.memory.size()
# replace the newly generated reasoning message with the loaded one
await self.memory.delete(num_msgs - 1)
old_reasoning_msg = self.state_loading_reasoning_msg
await self.memory.add(old_reasoning_msg)
self.state_loading_reasoning_msg = None
return old_reasoning_msg
async def planner_compose_reasoning_msg_pre_reasoning_hook(
self: "MetaPlanner", # pylint: disable=W0613
*args: Any,
**kwargs: Any,
) -> None:
"""Hook func for composing msg for reasoning step"""
reasoning_info = (
"## All User Input\n{all_user_input}\n\n"
"## Session Context\n"
"```json\n{notebook_string}\n```\n\n"
).format_map(
{
"notebook_string": self.planner_notebook.model_dump_json(
exclude={"user_input", "full_tool_list"},
indent=2,
),
"all_user_input": self.planner_notebook.user_input,
},
)
reasoning_msg = Msg(
"user",
content=reasoning_info,
role="user",
)
await self.memory.add(reasoning_msg)
async def planner_remove_reasoning_msg_post_reasoning_hook(
self: "MetaPlanner", # pylint: disable=W0613
*args: Any,
**kwargs: Any,
) -> None:
"""Hook func for removing msg for reasoning step"""
num_msgs = await self.memory.size()
if num_msgs > 1:
# remove the msg added by planner_compose_reasoning_pre_reasoning_hook
await self.memory.delete(num_msgs - 2)
def planner_save_post_action_state(
self: "MetaPlanner",
action_input: dict[str, Any],
tool_output: Optional[Msg], # pylint: disable=W0613
) -> None:
"""Hook func for save state after action step"""
if self.state_saving_dir:
os.makedirs(self.state_saving_dir, exist_ok=True)
time_str = datetime.now().strftime("%Y%m%d%H%M%S")
file_path = os.path.join(
self.state_saving_dir,
"state-post-action-"
f"{action_input.get('tool_call').get('name')}-{time_str}.json",
)
with open(file_path, "w", encoding="utf-8") as f:
json.dump(self.state_dict(), f, ensure_ascii=False, indent=4)
class MetaPlanner(ReActAgent):
"""
A meta-planning agent that extends ReActAgent with enhanced planning
capabilities. The MetaPlanner is designed to handle complex multistep
planning tasks by leveraging a combination of reasoning and action
capabilities. The subtasks will be solved by dynamically create ReAct
worker agent and provide it with necessary tools.
"""
def __init__(
self,
name: str,
model: ChatModelBase,
worker_full_toolkit: Toolkit,
formatter: FormatterBase,
memory: MemoryBase,
toolkit: Toolkit,
agent_working_dir: str,
sys_prompt: Optional[str] = None,
max_iters: int = 10,
state_saving_dir: Optional[str] = None,
planner_mode: Literal["disable", "dynamic", "enforced"] = "dynamic",
) -> None:
"""
Initialize the MetaPlanner with the given parameters.
Args:
name (str):
The name identifier for this agent instance.
model (ChatModelBase):
The primary chat model used for reasoning and response
generation.
worker_full_toolkit (Toolkit):
Complete set of tools available to the worker agent.
formatter (FormatterBase):
Formatter for formatting messages to the model API provider's
format.
memory (MemoryBase):
Memory system for storing conversation history and context.
toolkit (Toolkit):
Toolkit for managing tools available to the agent.
agent_working_dir (str):
Directory for agent's file operations.
sys_prompt (str, optional):
Meta planner's system prompt
max_iters (int, optional):
Maximum number of planning iterations. Defaults to 10.
state_saving_dir (Optional[str], optional):
Directory to save the agent's state. Defaults to None.
planner_mode (bool, optional):
Enable planner mode for solving tasks. Defaults to True.
"""
name = "Task-Meta-Planner" if name is None else name
if sys_prompt is None:
sys_prompt = (
"You are a helpful assistant named Task-Meta-Planner."
"If a given task can not be done easily, then you may need "
"to use the tool `enter_solving_complicated_task_mode` to "
"change yourself to a more long-term planning mode."
)
# Call super().__init__() early to initialize StateModule attributes
super().__init__(
name=name,
sys_prompt=sys_prompt,
model=model,
formatter=formatter,
memory=memory,
toolkit=toolkit,
max_iters=max_iters,
)
self.agent_working_dir_root = agent_working_dir
self.task_dir = self.agent_working_dir_root
self.worker_full_toolkit = worker_full_toolkit
self.state_saving_dir = state_saving_dir
# if we load a trajectory and the last step was reasoning,
# then we need a buffer to store the reasoning message and replace
# with this message after reasoning
self.state_loading_reasoning_msg: Optional[Msg] = None
# for debugging and state resume, we need a flag to indicate
self.planner_mode = planner_mode
self.in_planner_mode = False
self.register_state("planner_mode")
self.register_state("in_planner_mode")
self.planner_notebook = None
self.roadmap_manager, self.worker_manager = None, None
if planner_mode in ["dynamic", "enforced"]:
self.planner_notebook = PlannerNoteBook()
self.prepare_planner_tools(planner_mode)
self.register_state(
"planner_notebook",
lambda x: x.model_dump(),
lambda x: PlannerNoteBook(**x),
)
# pre-reply hook
self.register_instance_hook(
"pre_reply",
"update_user_input_to_notebook_pre_reply_hook",
update_user_input_pre_reply_hook,
)
# pre-reasoning hook
self.register_instance_hook(
"pre_reasoning",
"planner_load_state_pre_reasoning_hook",
planner_load_state_pre_reasoning_hook,
)
self.register_instance_hook(
"pre_reasoning",
"planner_compose_reasoning_msg_pre_reasoning_hook",
planner_compose_reasoning_msg_pre_reasoning_hook,
)
# post_reasoning hook
self.register_instance_hook(
"post_reasoning",
"planner_load_state_post_reasoning_hook",
planner_load_state_post_reasoning_hook,
)
self.register_instance_hook(
"post_reasoning",
"planner_remove_reasoning_msg_post_reasoning_hook",
planner_remove_reasoning_msg_post_reasoning_hook,
)
self.register_instance_hook(
"post_reasoning",
"save_state_post_reasoning_hook",
planner_save_post_reasoning_state,
)
# post_action_hook
self.register_instance_hook(
"post_acting",
"save_state_post_action_hook",
planner_save_post_action_state,
)
def prepare_planner_tools(
self,
planner_mode: Literal["disable", "enforced", "dynamic"],
) -> None:
"""
Prepare tool to planning depending on the selected mode.
"""
self.roadmap_manager = RoadmapManager(
planner_notebook=self.planner_notebook,
)
self.worker_manager = WorkerManager(
worker_model=self.model,
worker_formatter=self.formatter,
planner_notebook=self.planner_notebook,
agent_working_dir=self.task_dir,
worker_full_toolkit=self.worker_full_toolkit,
)
# clean
self.toolkit.remove_tool_groups("planning")
self.toolkit.create_tool_group(
"planning",
"Tool group for planning capability",
)
# re-register planning tool to enable loading the correct info
self.toolkit.register_tool_function(
self.roadmap_manager.decompose_task_and_build_roadmap,
group_name="planning",
)
self.toolkit.register_tool_function(
self.roadmap_manager.revise_roadmap,
group_name="planning",
)
self.toolkit.register_tool_function(
self.roadmap_manager.get_next_unfinished_subtask_from_roadmap,
group_name="planning",
)
self.toolkit.register_tool_function(
self.worker_manager.show_current_worker_pool,
group_name="planning",
)
self.toolkit.register_tool_function(
self.worker_manager.create_worker,
group_name="planning",
)
self.toolkit.register_tool_function(
self.worker_manager.execute_worker,
group_name="planning",
)
if planner_mode == "dynamic":
if "enter_solving_complicated_task_mode" not in self.toolkit.tools:
self.toolkit.register_tool_function(
self.enter_solving_complicated_task_mode,
)
# Only activate after agent decides to enter the
# planning-execution mode
self.toolkit.update_tool_groups(["planning"], False)
elif planner_mode == "enforced":
self.toolkit.update_tool_groups(["planning"], True)
# use the self.agent_working_dir as working dir
self._update_toolkit_and_sys_prompt()
def _ensure_file_system_functions(self) -> None:
required_tool_list = [
"read_file",
"write_file",
"edit_file",
"create_directory",
"list_directory",
"directory_tree",
"list_allowed_directories",
]
for tool_name in required_tool_list:
if tool_name not in self.worker_full_toolkit.tools:
raise ValueError(
f"{tool_name} must be in the worker toolkit and "
"its tool group must be active for complicated.",
)
share_tools(self.worker_full_toolkit, self.toolkit, required_tool_list)
async def enter_solving_complicated_task_mode(
self,
task_name: str,
) -> ToolResponse:
"""
When the user task meets any of the following conditions, enter the
solving complicated task mode by using this tool.
1. the task cannot be done within 5 reasoning-acting iterations;
2. the task cannot be done by the current tools you can see;
3. the task is related to comprehensive research or information
gathering
Args:
task_name (`str`):
Given a name to the current task as an indicator. Because
this name will be used to create a directory, so try to
use "_" instead of space between words, e.g. "A_NEW_TASK".
"""
# build directory for the task
self._ensure_file_system_functions()
self.task_dir = os.path.join(
self.agent_working_dir_root,
task_name,
)
self.worker_manager.agent_working_dir = self.task_dir
create_task_dir = ToolUseBlock(
type="tool_use",
id=str(uuid.uuid4()),
name="create_directory",
input={
"path": self.task_dir,
},
)
tool_res = await self.toolkit.call_tool_function(create_task_dir)
tool_res_msg = Msg(
"system",
content=[
ToolResultBlock(
type="tool_result",
output=[],
name="create_directory",
id=create_task_dir["id"],
),
],
role="system",
)
async for chunk in tool_res:
# Turn into a tool result block
tool_res_msg.content[0]["output"] = chunk.content
await self.print(tool_res_msg)
self._update_toolkit_and_sys_prompt()
return ToolResponse(
metadata={"success": True},
content=[
TextBlock(
type="text",
text=(
"Successfully enter the planning-execution mode to "
"solve complicated task. "
"All the file operations, including"
"read/write/modification, should be done in directory "
f"{self.task_dir}"
),
),
],
)
def _update_toolkit_and_sys_prompt(self) -> None:
# change agent settings for solving complicated task
full_worker_tool_list = [
{
"tool_name": func_dict.get("function", {}).get("name", ""),
"description": func_dict.get("function", {}).get(
"description",
"",
),
}
for func_dict in self.worker_full_toolkit.get_json_schemas()
]
self.planner_notebook.full_tool_list = full_worker_tool_list
with open(
Path(__file__).parent
/ "_built_in_long_sys_prompt"
/ "meta_planner_sys_prompt.md",
"r",
encoding="utf-8",
) as f:
sys_prompt = f.read()
sys_prompt = sys_prompt.format_map(
{
"tool_list": json.dumps(
full_worker_tool_list,
ensure_ascii=False,
),
},
)
self._sys_prompt = sys_prompt # pylint: disable=W0201
self.toolkit.update_tool_groups(["planning"], True)
self.in_planner_mode = True
def resume_planner_tools(self) -> None:
"""Resume the planner notebook for tools"""
self.prepare_planner_tools(self.planner_mode)
if self.in_planner_mode:
self._update_toolkit_and_sys_prompt()

24
functionality/meta_planner_agent/_planning_tools/__init__.py Normal file
View File

@@ -0,0 +1,24 @@
# -*- coding: utf-8 -*-
"""planning tools"""
from ._planning_notebook import (
PlannerNoteBook,
RoadMap,
SubTaskStatus,
Update,
WorkerInfo,
WorkerResponse,
)
from ._roadmap_manager import RoadmapManager
from ._worker_manager import WorkerManager, share_tools
__all__ = [
"PlannerNoteBook",
"RoadmapManager",
"WorkerManager",
"WorkerResponse",
"RoadMap",
"SubTaskStatus",
"WorkerInfo",
"Update",
"share_tools",
]

325
functionality/meta_planner_agent/_planning_tools/_planning_notebook.py Normal file
View File

@@ -0,0 +1,325 @@
# -*- coding: utf-8 -*-
# pylint: disable=E0213
"""
Data structures about the roadmap for complicated tasks
"""
from datetime import datetime
from typing import Any, Dict, List, Literal, Optional, Tuple
from pydantic import BaseModel, Field, field_validator
def get_current_time_message() -> str:
"""
Returns the current time as a formatted string.
Returns:
str: The current time formatted as 'YYYY-MM-DD HH:MM:SS'.
"""
return f"Current time is {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}"
WORKER_PROGRESS_SUMMARY = (
"## Instruction\n"
"Review the execution trace above and generate a comprehensive summary "
"report in Markdown format that addresses the original task/query. "
"Your report must include:\n\n"
"1. **Task Overview**\n"
" - Include the original query/task verbatim;\n"
" - Briefly state the main objective.\n"
"2. **Comprehensive Analysis**"
" - Provide a detailed, structured answer to the original query/task;\n"
" - Include all relevant information requested in the original task;\n"
" - Support your findings with specific references from your execution "
"trace;\n"
" - Organize content into logical sections with appropriate headings;\n"
" - Include data visualizations, tables, or formatted lists when "
"applicable.\n\n"
"3. **Completion Checklist**\n"
" - Reproduce the original 'Expected Output' checklist of required "
"tasks/information; **NEVER** makeup additional expected output items "
"in the checklist\n"
" - Mark each item as [x] Completed or [ ] Incomplete;\n"
" - For each completed item, reference where in your report this "
"information appears;\n"
" - For incomplete items, explain briefly why they remain unaddressed;\n"
"4. **Conclusion**\n"
" - If the task is fully complete, provide a brief conclusion "
"summarizing key findings;\n"
" - If the task remains incomplete, outline a specific meta_planner_agent to "
"address remaining items, including:\n"
" - Which tools would be used;\n"
" - What information is still needed;\n"
" - Sequence of planned actions.\n\n"
"Format your report professionally with consistent heading levels, "
"proper spacing, and appropriate emphasis for key information."
)
WORKER_NEXT_STEP_INSTRUCTION = """
If the subtask remains incomplete, outline a specific meta_planner_agent to address remaining
items, including:
- Which tools would be used
- What information is still needed
- Sequence of planned actions
Leave it as an empty string is the subtask has been done successfully.
"""
WORKER_FILE_COLLECTION_INSTRUCTION = (
"Collect all files generated in the execution process, "
"such as the files generated by `write_file` and `edit_file`."
"This field MUST be in dictionary, where"
"the keys are the paths of generated files "
"(e.g. '/FULL/PATH/OF/FILE_1.md') and the values are short "
"descriptions about the generated files."
)
class WorkerResponse(BaseModel):
"""
Represents the response structure from a worker agent after task execution.
This class defines the expected format for worker responses, including
progress summaries, next steps, tool usage information, and task
completion status.
Attributes:
subtask_progress_summary (str):
Comprehensive summary report of task execution.
next_step (str):
Description of planned next actions if task is incomplete.
generated_files (dict):
Dictionary mapping file paths to descriptions of generated files.
task_done (bool):
Flag indicating whether the task has been completed.
"""
subtask_progress_summary: str = Field(
...,
description=WORKER_PROGRESS_SUMMARY,
)
next_step: str = Field(
...,
description=WORKER_NEXT_STEP_INSTRUCTION,
)
generated_files: dict = Field(
...,
description=WORKER_FILE_COLLECTION_INSTRUCTION,
)
task_done: bool = Field(
...,
description="Whether task is done or it require addition effort",
)
class Update(BaseModel):
"""Represents an update record from a worker during task execution.
This class tracks progress updates from workers as they work on subtasks,
including status changes, progress summaries, and execution details.
Attributes:
reason_for_status (str): Explanation for the current status.
task_done (bool): Whether the task has been completed.
subtask_progress_summary (str): Summary of progress made.
next_step (str): Description of planned next actions.
worker (str): Identifier of the worker providing the update.
attempt_idx (int): Index of the current attempt.
"""
reason_for_status: str
task_done: bool
subtask_progress_summary: str
next_step: str
worker: str
attempt_idx: int
@field_validator(
"subtask_progress_summary",
"reason_for_status",
"next_step",
"worker",
mode="before",
)
def _stringify(cls, v: Any) -> str:
"""ensure the attributes are string"""
if v is None:
return ""
return str(v)
class WorkerInfo(BaseModel):
"""Contains information about a worker agent assigned to a subtask.
This class stores metadata about worker agents, including their
capabilities, creation type, and configuration details.
Attributes:
worker_name (str):
Name identifier of the worker.
status (str):
Current status of the worker.
create_type (Literal["built-in", "dynamic-built"]):
How the worker was created.
description (str):
Description of the worker's purpose and capabilities.
tool_lists (List[str]):
List of tools available to this worker.
sys_prompt (str):
System prompt used to configure the worker.
"""
worker_name: str = ""
status: str = ""
create_type: Literal["built-in", "dynamic-built"] = "dynamic-built"
description: str = ""
# for dynamically create worker agents
tool_lists: List[str] = Field(default_factory=list)
sys_prompt: str = ""
@field_validator(
"worker_name",
"status",
mode="before",
)
def _stringify(cls, v: Any) -> str:
if v is None:
return ""
return str(v)
class SubTaskSpecification(BaseModel):
"""
Details of a subtask within a larger task decomposition.
"""
subtask_description: str = Field(description="Description of the subtask.")
input_intro: str = Field(
...,
description="Introduction or context for the subtask input.",
)
exact_input: str = Field(
...,
description="The exact input data or parameters for the subtask.",
)
expected_output: str = Field(
...,
description="The expected output data or parameters for the subtask.",
)
desired_auxiliary_tools: str = Field(
...,
description="Tools that would be helpful for this subtask.",
)
@field_validator(
"subtask_description",
"input_intro",
"exact_input",
"expected_output",
"desired_auxiliary_tools",
mode="before",
)
def _stringify(cls, v: Any) -> str:
if v is None:
return ""
return str(v)
class SubTaskStatus(BaseModel):
"""
Represents the status and details of a subtask within a
larger task decomposition.
This class tracks individual subtasks, their execution status,
assigned workers, and progress updates throughout the execution lifecycle.
Attributes:
status (Literal["Planned", "In-process", "Done"]):
Current execution status.
updates (List[Update]):
List of progress updates from workers.
attempt (int):
Number of execution attempts for this subtask.
workers (List[WorkerInfo]):
List of workers assigned to this subtask.
"""
subtask_specification: SubTaskSpecification = Field(
default_factory=SubTaskSpecification,
)
status: Literal["Planned", "In-process", "Done"] = "Planned"
updates: List[Update] = Field(
default_factory=list,
description=(
"List of updates from workers. "
"MUST be empty list when initialized."
),
)
attempt: int = 0
workers: List[WorkerInfo] = Field(
default_factory=list,
description=(
"List of workers that have been assigned to this subtask."
"MUST be EMPTY when initialize the subtask."
),
)
class RoadMap(BaseModel):
"""Represents a roadmap for task decomposition and execution tracking.
This class manages the overall task breakdown, containing the original task
description and a list of decomposed subtasks with their execution status.
Attributes:
original_task (str):
The original task description before decomposition.
decomposed_tasks (List[SubTaskStatus]):
List of subtasks created from the original task.
"""
original_task: str = ""
decomposed_tasks: List[SubTaskStatus] = Field(default_factory=list)
def next_unfinished_subtask(
self,
) -> Tuple[Optional[int], Optional[SubTaskStatus]]:
"""Find the next subtask that is not yet completed.
Iterates through the decomposed tasks to find the first subtask
with status "Planned" or "In-process".
Returns:
Tuple[Optional[int], Optional[SubTaskStatus]]: A tuple containing:
- The index of the next unfinished subtask
(None if all tasks are done)
- The SubTaskStatus object of the next unfinished subtask
(None if all tasks are done)
"""
for i, subtask in enumerate(self.decomposed_tasks):
if subtask.status in ["Planned", "In-process"]:
return i, subtask
return None, None
class PlannerNoteBook(BaseModel):
"""
Represents a planner notebook.
Attributes:
time (str): The current time message.
user_input (List[str]): List of user inputs.
detail_analysis_for_plan (str): Detailed analysis for the meta_planner_agent.
roadmap (RoadMap): The roadmap associated with the planner.
files (Dict[str, str]): Dictionary of files related to the planner.
full_tool_list (dict[str, dict]): Full schema of tools.
"""
time: str = Field(default_factory=get_current_time_message)
user_input: List[str] = Field(default_factory=list)
detail_analysis_for_plan: str = (
"Unknown. Please call `build_roadmap_and_decompose_task` to analyze."
)
roadmap: RoadMap = Field(default_factory=RoadMap)
files: Dict[str, str] = Field(default_factory=dict)
full_tool_list: list[dict] = Field(default_factory=list)

279
functionality/meta_planner_agent/_planning_tools/_roadmap_manager.py Normal file
View File

@@ -0,0 +1,279 @@
# -*- coding: utf-8 -*-
"""
Planning handler module for meta planner
"""
from typing import Literal, Optional
from agentscope.message import TextBlock
from agentscope.module import StateModule
from agentscope.tool import ToolResponse
from ._planning_notebook import (
PlannerNoteBook,
SubTaskSpecification,
SubTaskStatus,
Update,
)
class RoadmapManager(StateModule):
"""Handles planning operations for meta planner agent.
This class provides functionality for task decomposition, roadmap creation,
and roadmap revision.
"""
def __init__(
self,
planner_notebook: PlannerNoteBook,
):
"""Initialize the PlanningHandler.
Args:
planner_notebook (PlannerNoteBook):
Data structure containing planning state.
"""
super().__init__()
self.planner_notebook = planner_notebook
async def decompose_task_and_build_roadmap(
self,
user_latest_input: str,
given_task_conclusion: str,
detail_analysis_for_plan: str,
decomposed_subtasks: list[SubTaskSpecification],
) -> ToolResponse:
"""
Analysis the user subtask, generate a comprehensive reasoning for how
to decompose the task into multiple subtasks.
Args:
user_latest_input (str):
The latest user input. If there are multiple rounds
of user input, faithfully record the latest user input.
given_task_conclusion (str):
The user's task to decompose. If there are multiple rounds
of user input, analysis and give the key idea of the task that
the user really you to solve.
detail_analysis_for_plan (str):
A detailed analysis of how a task should be decomposed.
decomposed_subtasks (list[SubTaskSpecification]):
List of subtasks that was decomposed.
"""
self.planner_notebook.detail_analysis_for_plan = (
detail_analysis_for_plan
)
self.planner_notebook.roadmap.original_task = given_task_conclusion
for subtask in decomposed_subtasks:
if isinstance(subtask, dict):
subtask_status = SubTaskStatus(
subtask_specification=SubTaskSpecification(
**subtask,
),
)
elif isinstance(subtask, SubTaskSpecification):
subtask_status = SubTaskStatus(
subtask_specification=subtask,
)
else:
raise TypeError(
"Unexpected type of `decomposed_subtasks`,"
"which is expected to strictly follow List of "
"SubTaskSpecification.",
)
self.planner_notebook.roadmap.decomposed_tasks.append(
subtask_status,
)
self.planner_notebook.user_input.append(user_latest_input)
return ToolResponse(
metadata={"success": True},
content=[
TextBlock(
type="text",
text="Successfully decomposed the task into subtasks",
),
],
)
async def get_next_unfinished_subtask_from_roadmap(self) -> ToolResponse:
"""
Obtains the next unfinished subtask from the roadmap.
"""
idx, subtask = self.planner_notebook.roadmap.next_unfinished_subtask()
if idx is None or subtask is None:
return ToolResponse(
metadata={"success": False},
content=[
TextBlock(
type="text",
text=(
"No unfinished subtask was found. "
"Either all subtasks have been done, or the task"
" has not been decomposed."
),
),
],
)
return ToolResponse(
metadata={"success": True, "subtask": subtask},
content=[
TextBlock(
type="text",
text=f"Next unfinished subtask idx: {idx}",
),
TextBlock(
type="text",
text=subtask.model_dump_json(indent=2),
),
],
)
async def revise_roadmap(
self,
action: Literal["add_subtask", "revise_subtask", "remove_subtask"],
subtask_idx: int,
subtask_specification: Optional[SubTaskSpecification] = None,
update_to_subtask: Optional[Update] = None,
new_status: Literal["Planned", "In-process", "Done"] = "In-process",
) -> ToolResponse:
"""After subtasks are done by worker agents, use this function to
revise the progress and details of the current roadmap.
Updates the status of subtasks and potentially revises input/output
descriptions and required tools for tasks based on current progress
and available information.
Args:
action (
`Literal["add_subtask", "revise_subtask", "remove_subtask"]`
):
Action to perform on the roadmap.
subtask_idx (`int`):
Index of the subtask to revise its status. This index starts
with 0.
subtask_specification (`SubTaskSpecification`):
Revised subtask specification. When you use `add_subtask` or
`revise_subtask` action, you MUST provide this field with
revised `exact_input` and `expected_output` according to
the execution context.
update_to_subtask (`Update`):
Generate an update record for this subtask based on the
worker execution report. When you use `revise_subtask` action,
you MUST provide this field.
new_status (`Literal["Planned", "In-process", "Done"]`):
The new status of the subtask.
Returns:
ToolResponse:
Response indicating success/failure of the revision
and any updates made. May request additional human
input if needed.
"""
num_subtasks = len(self.planner_notebook.roadmap.decomposed_tasks)
if isinstance(subtask_specification, dict):
subtask_specification = SubTaskSpecification(
**subtask_specification,
)
elif subtask_specification is None and action in [
"add_subtask",
"revise_subtask",
]:
return ToolResponse(
metadata={"success": False},
content=[
TextBlock(
type="text",
text=(
f"Choosing {action} must have valid "
f"`subtask_specification` field."
),
),
],
)
if isinstance(update_to_subtask, dict):
update_to_subtask = Update(
**update_to_subtask,
)
elif update_to_subtask is None and action == "revise_subtask":
return ToolResponse(
metadata={"success": False},
content=[
TextBlock(
type="text",
text=(
f"Choosing {action} must have valid "
f"`update_to_subtask` field."
),
),
],
)
if subtask_idx >= num_subtasks and action == "add_subtask":
self.planner_notebook.roadmap.decomposed_tasks.append(
SubTaskStatus(
subtask_specification=subtask_specification,
status="Planned",
updates=update_to_subtask,
),
)
return ToolResponse(
metadata={"success": True},
content=[
TextBlock(
type="text",
text=f"add new subtask with index {subtask_idx}.",
),
],
)
elif subtask_idx >= num_subtasks:
return ToolResponse(
metadata={"success": False},
content=[
TextBlock(
type="text",
text=(
f"Fail to update subtask {subtask_idx} status."
f"There are {num_subtasks} subtasks, "
f"idx {subtask_idx} is not supported with "
f"action {action}."
),
),
],
)
elif action == "revise_subtask" and update_to_subtask:
subtask = self.planner_notebook.roadmap.decomposed_tasks[
subtask_idx
]
subtask.status = new_status
subtask.updates.append(update_to_subtask)
return ToolResponse(
metadata={"success": True},
content=[
TextBlock(
type="text",
text=f"Update subtask {subtask_idx} status.",
),
TextBlock(
type="text",
text=self.planner_notebook.roadmap.decomposed_tasks[
subtask_idx
].model_dump_json(indent=2),
),
],
)
elif action == "remove_subtask":
self.planner_notebook.roadmap.decomposed_tasks.pop(subtask_idx)
return ToolResponse(
metadata={"success": True},
content=[
TextBlock(
type="text",
text=f"Remove subtask {subtask_idx} from roadmap.",
),
],
)
else:
raise ValueError(
f"Not support action {action} on subtask {subtask_idx}",
)

525
functionality/meta_planner_agent/_planning_tools/_worker_manager.py Normal file
View File

@@ -0,0 +1,525 @@
# -*- coding: utf-8 -*-
"""
Coordination handler module for meta planner
"""
import json
import os
from pathlib import Path
from typing import List, Literal, Optional
from agentscope import logger
from agentscope.agent import ReActAgent
from agentscope.formatter import DashScopeChatFormatter, FormatterBase
from agentscope.memory import InMemoryMemory, MemoryBase
from agentscope.message import Msg, TextBlock, ToolResultBlock, ToolUseBlock
from agentscope.model import ChatModelBase, DashScopeChatModel
from agentscope.module import StateModule
from agentscope.tool import Toolkit, ToolResponse
from ._planning_notebook import PlannerNoteBook, WorkerInfo, WorkerResponse
def rebuild_reactworker(
worker_info: WorkerInfo,
old_toolkit: Toolkit,
new_toolkit: Toolkit,
memory: Optional[MemoryBase] = None,
model: Optional[ChatModelBase] = None,
formatter: Optional[FormatterBase] = None,
exclude_tools: Optional[list[str]] = None,
) -> ReActAgent:
"""
Rebuild a ReActAgent worker with specified configuration and tools.
Creates a new ReActAgent using worker information and toolkit
configuration. Tools are shared from the old toolkit to the new one,
excluding any specified tools.
Args:
worker_info (WorkerInfo): Information about the worker including name,
system prompt, and tool lists.
old_toolkit (Toolkit): Source toolkit containing available tools.
new_toolkit (Toolkit): Destination toolkit to receive shared tools.
memory (Optional[MemoryBase], optional): Memory instance for the agent.
Defaults to InMemoryMemory() if None.
model (Optional[ChatModelBase], optional): Chat model instance.
Defaults to DashscopeChatModel with deepseek-r1 if None.
formatter (Optional[FormatterBase], optional): Message formatter.
Defaults to DashScopeChatFormatter() if None.
exclude_tools (Optional[list[str]], optional): List of tool names to
exclude from sharing. Defaults to empty list if None.
Returns:
ReActAgent: A configured ReActAgent instance ready for use.
Note:
- The default model uses the DASHSCOPE_API_KEY environment variable
- Tools are shared based on worker_info.tool_lists minus excluded tools
- The agent is configured with thinking enabled and streaming support
"""
if exclude_tools is None:
exclude_tools = []
tool_list = [
tool_name
for tool_name in worker_info.tool_lists
if tool_name not in exclude_tools
]
share_tools(old_toolkit, new_toolkit, tool_list)
model = (
model
if model
else DashScopeChatModel(
api_key=os.environ.get("DASHSCOPE_API_KEY"),
model_name="deepseek-r1",
enable_thinking=True,
stream=True,
)
)
return ReActAgent(
name=worker_info.worker_name,
sys_prompt=worker_info.sys_prompt,
model=model,
formatter=formatter if formatter else DashScopeChatFormatter(),
toolkit=new_toolkit,
memory=InMemoryMemory() if memory is None else memory,
max_iters=20, # hardcoded the max iteration for now
)
async def check_file_existence(file_path: str, toolkit: Toolkit) -> bool:
"""
Check if a file exists using the read_file tool from the provided toolkit.
This function attempts to verify file existence by calling the read_file
tool and checking the response for error indicators. It requires the
toolkit to have a 'read_file' tool available.
Args:
file_path (str): The path to the file to check for existence.
toolkit (Toolkit): The toolkit containing the read_file tool.
Returns:
bool: True if the file exists and is readable, False otherwise.
Note:
- Returns False if the 'read_file' tool is not available in the toolkit
- Returns False if any exception occurs during the file read attempt
- Uses error message detection ("no such file or directory") to
determine existence
"""
if "read_file" in toolkit.tools:
params = {
"path": file_path,
}
read_file_block = ToolUseBlock(
type="tool_use",
id="manual_check_file_existence",
name="read_file",
input=params,
)
try:
tool_res = await toolkit.call_tool_function(read_file_block)
tool_res_msg = Msg(
"system",
[
ToolResultBlock(
type="tool_result",
id="",
name="read_file",
output=[],
),
],
"system",
)
async for chunk in tool_res:
# Turn into a tool result block
tool_res_msg.content[0]["output"] = chunk.content # type: ignore[index]
if "no such file or directory" in str(tool_res_msg.content):
return False
else:
return True
except Exception as _: # noqa: F841
return False
else:
return False
def share_tools(
old_toolkit: Toolkit,
new_toolkit: Toolkit,
tool_list: list[str],
) -> None:
"""
Share specified tools from an old toolkit to a new toolkit.
This function copies tools from one toolkit to another based on the
provided tool list. If a tool doesn't exist in the old toolkit,
a warning is logged.
Args:
old_toolkit (Toolkit):
The source toolkit containing tools to be shared.
new_toolkit (Toolkit):
The destination toolkit to receive the tools.
tool_list (list[str]):
List of tool names to be copied from old to new toolkit.
Returns:
None
Note:
This function modifies the new_toolkit in place.
If a tool in tool_list is not found in old_toolkit,
a warning is logged but execution continues.
"""
for tool in tool_list:
if tool in old_toolkit.tools and tool not in new_toolkit.tools:
new_toolkit.tools[tool] = old_toolkit.tools[tool]
else:
logger.warning(
"No tool %s in the provided worker_tool_toolkit",
tool,
)
class WorkerManager(StateModule):
"""
Handles coordination between meta planner and worker agents.
This class manages the creation, selection, and execution of worker agents
to accomplish subtasks in a roadmap. It provides functionality for dynamic
worker creation, worker selection based on task requirements, and
processing worker responses to update the overall task progress.
"""
def __init__(
self,
worker_model: ChatModelBase,
worker_formatter: FormatterBase,
planner_notebook: PlannerNoteBook,
worker_full_toolkit: Toolkit,
agent_working_dir: str,
worker_pool: Optional[dict[str, tuple[WorkerInfo, ReActAgent]]] = None,
):
"""Initialize the CoordinationHandler.
Args:
worker_model (ChatModelBase):
Main language model for coordination decisions
worker_formatter (FormatterBase):
Message formatter for model communication
planner_notebook (PlannerNoteBook):
Notebook containing roadmap and file information
worker_full_toolkit (Toolkit):
Complete toolkit available to workers
agent_working_dir (str):
Working directory for the agent operations
worker_pool: dict[str, tuple[WorkerInfo, ReActAgent]]:
workers that has already been created
"""
super().__init__()
self.planner_notebook = planner_notebook
self.worker_model = worker_model
self.worker_formatter = worker_formatter
self.worker_pool: dict[str, tuple[WorkerInfo, ReActAgent]] = (
worker_pool if worker_pool else {}
)
self.agent_working_dir = agent_working_dir
self.worker_full_toolkit = worker_full_toolkit
def reconstruct_workerpool(worker_pool_dict: dict) -> dict:
rebuild_worker_pool = {}
for k, v in worker_pool_dict.items():
worker_info = WorkerInfo(**v)
rebuild_worker_pool[k] = (
worker_info,
rebuild_reactworker(
worker_info=worker_info,
old_toolkit=self.worker_full_toolkit,
new_toolkit=Toolkit(),
model=self.worker_model,
formatter=self.worker_formatter,
exclude_tools=["generate_response"],
),
)
return rebuild_worker_pool
self.register_state(
"worker_pool",
lambda x: {k: v[0].model_dump() for k, v in x.items()},
custom_from_json=reconstruct_workerpool,
)
def _register_worker(
self,
agent: ReActAgent,
description: Optional[str] = None,
worker_type: Literal["built-in", "dynamic-built"] = "dynamic",
) -> None:
"""
Register a worker agent in the worker pool.
Adds a worker agent to the available pool with appropriate metadata.
Handles name conflicts by appending version numbers when necessary.
Args:
agent (ReActAgent):
The worker agent to register
description (Optional[str]):
Description of the worker's capabilities
worker_type (Literal["built-in", "dynamic-built"]):
Type of worker agent
"""
worker_info = WorkerInfo(
worker_name=agent.name,
description=description,
worker_type=worker_type,
status="ready-to-work",
)
if worker_type == "dynamic-built":
worker_info.sys_prompt = agent.sys_prompt
worker_info.tool_lists = list(agent.toolkit.tools.keys())
if agent.name in self.worker_pool:
name = agent.name
version = 1
while name in self.worker_pool:
name = agent.name + f"_v{version}"
version += 1
agent.name, worker_info.worker_name = name, name
self.worker_pool[name] = (worker_info, agent)
else:
self.worker_pool[agent.name] = (worker_info, agent)
@staticmethod
def _no_more_subtask_return() -> ToolResponse:
"""
Return response when no more unfinished subtasks exist.
Returns:
ToolResponse: Response indicating no more subtasks are available
"""
return ToolResponse(
metadata={"success": False},
content=[
TextBlock(
type="text",
text="No more subtask exists. "
"Check whether the task is "
"completed solved.",
),
],
)
async def create_worker(
self,
worker_name: str,
worker_system_prompt: str,
tool_names: Optional[List[str]] = None,
agent_description: str = "",
) -> ToolResponse:
"""
Create a worker agent for the next unfinished subtask.
Dynamically creates a specialized worker agent based on the
requirements of the next unfinished subtask in the roadmap.
The worker is configured with appropriate tools and system prompts
based on the task needs.
Args:
worker_name (str): The name of the worker agent.
worker_system_prompt (str): The system prompt for the worker agent.
tool_names (Optional[List[str]], optional):
List of tools that should be assigned to the worker agent so
that it can finish the subtask. MUST be from the
`Available Tools for workers`
agent_description (str, optional):
A brief description of the worker's capabilities.
Returns:
ToolResponse: Response containing the creation result and worker
details
"""
if tool_names is None:
tool_names = []
worker_toolkit = Toolkit()
share_tools(
self.worker_full_toolkit,
worker_toolkit,
tool_names
+ [
"read_file",
"write_file",
"edit_file",
"search_files",
"list_directory",
],
)
with open(
Path(__file__).parent.parent
/ "_built_in_long_sys_prompt"
/ "_worker_additional_sys_prompt.md",
"r",
encoding="utf-8",
) as f:
additional_worker_prompt = f.read()
with open(
Path(__file__).parent.parent
/ "_built_in_long_sys_prompt"
/ "_tool_usage_rules.md",
"r",
encoding="utf-8",
) as f:
additional_worker_prompt += str(f.read()).format_map(
{"agent_working_dir": self.agent_working_dir},
)
worker = ReActAgent(
name=worker_name,
sys_prompt=(worker_system_prompt + additional_worker_prompt),
model=self.worker_model,
formatter=self.worker_formatter,
memory=InMemoryMemory(),
toolkit=worker_toolkit,
)
self._register_worker(
worker,
description=agent_description,
worker_type="dynamic-built",
)
return ToolResponse(
metadata={"success": True},
content=[
TextBlock(
type="text",
text=(
f"Successfully created a worker agent:\n"
f"Worker name: {worker_name}"
f"Worker tools: {tool_names}"
f"Worker system prompt: {worker.sys_prompt}"
),
),
],
)
async def show_current_worker_pool(self) -> ToolResponse:
"""
List all currently available worker agents with
their system prompts and tools.
"""
worker_info: dict[str, dict] = {
name: info.model_dump()
for name, (info, _) in self.worker_pool.items()
}
return ToolResponse(
metadata={"success": True},
content=[
TextBlock(
type="text",
text=json.dumps(worker_info, ensure_ascii=False, indent=2),
),
],
)
async def execute_worker(
self,
subtask_idx: int,
selected_worker_name: str,
detailed_instruction: str,
) -> ToolResponse:
"""
Execute a worker agent for the next unfinished subtask.
Args:
subtask_idx (int):
Index of the subtask to execute.
selected_worker_name (str):
Select a worker agent to execute by its name. If you are unsure
what are the available agents, call `show_current_worker_pool`
before using this function.
detailed_instruction (str):
Generate detailed instruction for the worker based on the
next unfinished subtask in the roadmap. If you are unsure
what is the next unavailable subtask, check with
`get_next_unfinished_subtask_from_roadmap` to get more info.
"""
if selected_worker_name not in self.worker_pool:
worker_info: dict[str, WorkerInfo] = {
name: info for name, (info, _) in self.worker_pool.items()
}
current_agent_pool = json.dumps(
worker_info,
ensure_ascii=False,
indent=2,
)
return ToolResponse(
metadata={"success": False},
content=[
TextBlock(
type="text",
text=(
f"There is no {selected_worker_name} in current"
"agent pool."
"Current agent pool:\n```json"
f"{current_agent_pool}\n"
"```"
),
),
],
)
worker = self.worker_pool[selected_worker_name][1]
question_msg = Msg(
role="user",
name="user",
content=detailed_instruction,
)
worker_response_msg = await worker(
question_msg,
structured_model=WorkerResponse,
)
if worker_response_msg.metadata is not None:
worker_response = WorkerResponse(
**worker_response_msg.metadata,
)
self.planner_notebook.roadmap.decomposed_tasks[
subtask_idx
].workers.append(
self.worker_pool[selected_worker_name][0],
)
# double-check to ensure the generated files exists
for filepath, desc in worker_response.generated_files.items():
if await check_file_existence(
filepath,
self.worker_full_toolkit,
):
self.planner_notebook.files[filepath] = desc
else:
worker_response.generated_files.pop(filepath)
return ToolResponse(
metadata={
"success": True,
"worker_response": worker_response.model_dump_json(),
},
content=[
TextBlock(
type="text",
text=worker_response.model_dump_json(),
),
],
)
else:
return ToolResponse(
metadata={
"success": False,
"worker_response": worker_response_msg.content,
},
content=[
TextBlock(
type="text",
text=str(worker_response_msg.content),
),
],
)

224
functionality/meta_planner_agent/main.py Normal file
View File

@@ -0,0 +1,224 @@
# -*- coding: utf-8 -*-
"""Main entry point for the Meta-planner agent example.
This module provides a conversational interface for the MetaPlanner agent,
which is designed to handle complex tasks through a planning-execution pattern.
The agent can break down complex requests into manageable steps and execute
them using various tools and MCP (Model Context Protocol) clients.
The key points in this script includes:
- Setting up MCP clients for external tool integration
(Tavily search, filesystem)
- Configuring toolkits for both planner and worker agents
- Managing agent state persistence and recovery
- Providing an interactive chat interface
Example:
Run the agent interactively:
$ python main.py
Load from a previous state:
$ python main.py --load_state ./agent-states/run-xxxx/state-xxx.json
Required Environment Variables:
ANTHROPIC_API_KEY: API key for Anthropic Claude model
TAVILY_API_KEY: API key for Tavily search functionality
Optional Environment Variables:
AGENT_OPERATION_DIR: Custom working directory for agent operations
"""
import argparse
import asyncio
import json
import os
from datetime import datetime
from _meta_planner import MetaPlanner # pylint: disable=C0411
from agentscope import logger
from agentscope.agent import UserAgent
from agentscope.formatter import AnthropicChatFormatter
from agentscope.mcp import StatefulClientBase, StdIOStatefulClient
from agentscope.memory import InMemoryMemory
from agentscope.message import ToolUseBlock
from agentscope.model import AnthropicChatModel
from agentscope.tool import (
Toolkit,
ToolResponse,
execute_shell_command,
view_text_file,
)
def chunking_too_long_tool_response(
tool_use: ToolUseBlock, # pylint: disable=W0613
tool_response: ToolResponse,
) -> ToolResponse:
"""Post-process tool responses to prevent content overflow.
This function ensures that tool responses don't exceed a predefined budget
to prevent overwhelming the model with too much information. It truncates
text content while preserving the structure of the response.
Args:
tool_use: The tool use block that triggered the response (unused).
tool_response: The tool response to potentially truncate.
Note:
The budget is set to approximately 40K tokens (8194 * 5 characters)
to ensure responses remain manageable for the language model.
"""
# Set budget to prevent overwhelming the model with too much content
budget = 8194 * 5 # Approximately 40KB of content
for i, block in enumerate(tool_response.content):
if block["type"] == "text":
text = block["text"]
text_len = len(text)
# If budget is exhausted, truncate remaining blocks
if budget <= 0:
tool_response.content = tool_response.content[:i]
break
# If this block exceeds remaining budget, truncate it
if text_len > budget:
# Calculate truncation threshold (80% of proportional budget)
threshold = int(budget / text_len * len(text) * 0.8)
tool_response.content[i]["text"] = text[:threshold]
budget -= text_len
return tool_response
def _add_tool_postprocessing_func(worker_toolkit: Toolkit) -> None:
"""Add postprocessing functions to specific tools in the worker toolkit.
This function applies content truncation to tools that might return
large amounts of data, specifically Tavily search tools, to prevent
overwhelming the language model.
Args:
worker_toolkit: The toolkit containing worker tools to modify.
"""
for tool_func, _ in worker_toolkit.tools.items():
# Apply truncation to Tavily search tools
if tool_func.startswith("tavily"):
worker_toolkit.tools[
tool_func
].postprocess_func = chunking_too_long_tool_response
async def main() -> None:
"""The main entry point for the Meta-planner agent example."""
logger.setLevel("DEBUG")
time_str = datetime.now().strftime("%Y%m%d%H%M%S")
planner_toolkit = Toolkit()
worker_toolkit = Toolkit()
worker_toolkit.register_tool_function(execute_shell_command)
worker_toolkit.register_tool_function(view_text_file)
mcp_clients = []
assert os.getenv("TAVILY_API_KEY") is not None
tavily_key = os.getenv("TAVILY_API_KEY")
mcp_clients.append(
StdIOStatefulClient(
name="tavily_mcp",
command="npx",
args=["-y", "tavily-mcp@latest"],
env={"TAVILY_API_KEY": tavily_key},
),
)
# Note: You can add more MCP/tools for more diverse tasks
default_working_dir = os.path.join(
os.path.dirname(__file__),
"meta_agent_demo_env",
)
agent_working_dir = os.getenv(
"AGENT_OPERATION_DIR",
default_working_dir,
)
os.makedirs(agent_working_dir, exist_ok=True)
mcp_clients.append(
StdIOStatefulClient(
name="file_system_mcp",
command="npx",
args=[
"-y",
"@modelcontextprotocol/server-filesystem",
agent_working_dir,
],
),
)
try:
for mcp_client in mcp_clients:
if isinstance(mcp_client, StatefulClientBase):
await mcp_client.connect()
await worker_toolkit.register_mcp_client(mcp_client)
_add_tool_postprocessing_func(worker_toolkit)
agent = MetaPlanner(
name="Task-Meta-Planner",
model=AnthropicChatModel(
api_key=os.environ.get("ANTHROPIC_API_KEY"),
model_name="claude-sonnet-4-20250514",
stream=True,
),
formatter=AnthropicChatFormatter(),
toolkit=planner_toolkit,
worker_full_toolkit=worker_toolkit,
agent_working_dir=agent_working_dir,
memory=InMemoryMemory(),
state_saving_dir=f"./agent-states/run-{time_str}",
max_iters=100,
)
user = UserAgent("Bob")
msg = None
skip_user_input = False
if args.load_state:
state_file_path = args.load_state
with open(state_file_path, "r", encoding="utf-8") as f:
state_dict = json.load(f)
agent.load_state_dict(state_dict)
agent.resume_planner_tools()
skip_user_input = True
while True:
if skip_user_input:
skip_user_input = False
else:
msg = await user(msg)
if msg.get_text_content() == "exit":
break
msg = await agent(msg)
except Exception as e:
logger.exception(e)
finally:
for mcp_client in mcp_clients:
if isinstance(mcp_client, StatefulClientBase):
await mcp_client.close()
def parse_args() -> argparse.Namespace:
"""parsing args from command line"""
parser = argparse.ArgumentParser(
description="Run the ReAct agent example with a specified state.",
)
parser.add_argument(
"--load_state",
type=str,
help="The input file name to load the state from.",
)
return parser.parse_args()
if __name__ == "__main__":
args = parse_args()
asyncio.run(main())

1
functionality/meta_planner_agent/requirements.txt Normal file
View File

@@ -0,0 +1 @@
agentscope[full]>=1.0.5

29
functionality/multiagent_concurrent/README.md Normal file
View File

@@ -0,0 +1,29 @@
# Multiagent Concurrent
This example demonstrates how to run multiple agents concurrently in AgentScope, where each agent operates
independently and can perform tasks simultaneously.
Specifically, we showcase two ways to achieve concurrency:
- Using Python's `asyncio.gather` to run multiple agents asynchronously.
- Using `fanout_pipeline` to execute multiple agents in parallel and gather their results.
The fanout pipeline will distribute the input to multiple agents and collect their outputs, which is appropriate for
scenarios like voting or parallel question answering.
## QuickStart
Install the agentscope package if you haven't already:
```bash
pip install agentscope
```
Then run the example script:
```bash
python main.py
```
## Further Reading
- [Pipelines](https://doc.agentscope.io/tutorial/task_pipeline.html)

92
functionality/multiagent_concurrent/main.py Normal file
View File

@@ -0,0 +1,92 @@
# -*- coding: utf-8 -*-
"""Parallel Multi-Perspective Discussion System."""
import asyncio
from datetime import datetime
from typing import Any
import numpy as np
from agentscope.agent import AgentBase
from agentscope.message import Msg
from agentscope.pipeline import fanout_pipeline
class ExampleAgent(AgentBase):
"""The example agent used to label the time."""
def __init__(self, name: str) -> None:
"""The constructor of the example agent
Args:
name (`str`):
The agent name.
"""
super().__init__()
self.name = name
async def reply(self, *args: Any, **kwargs: Any) -> Msg:
"""The reply function of the example agent."""
# we record the start time
start_time = datetime.now()
await self.print(
Msg(
self.name,
f"begins at {start_time.strftime('%H:%M:%S.%f')}",
"assistant",
),
)
# Sleep some time
await asyncio.sleep(np.random.choice([2, 3, 4]))
end_time = datetime.now()
msg = Msg(
self.name,
f"finishes at {end_time.strftime('%H:%M:%S.%f')}",
"user",
# Add some metadata for demonstration
metadata={
"time": (end_time - start_time).total_seconds(),
},
)
await self.print(msg)
return msg
async def handle_interrupt(
self,
*args: Any,
**kwargs: Any,
) -> Msg:
"""We leave this function unimplemented in this example, because we
won't use the interrupt functionality"""
async def observe(self, *args: Any, **kwargs: Any) -> None:
"""Similar with the handle_interrupt function, leaving this empty"""
async def main() -> None:
"""The main entry of the concurrent example."""
alice = ExampleAgent("Alice")
bob = ExampleAgent("Bob")
chalice = ExampleAgent("Chalice")
print("Use 'asyncio.gather' to run the agents concurrently:")
futures = [alice(), bob(), chalice()]
await asyncio.gather(*futures)
print("\n\nUse fanout pipeline to run the agents concurrently:")
collected_res = await fanout_pipeline(
agents=[alice, bob, chalice],
enable_gather=True,
)
# Print the collected results
print("\n\nThe collected time used by each agent:")
for res in collected_res:
print(f"{res.name}: {res.metadata['time']} seconds")
print("\nThe average time used:")
avg_time = np.mean([res.metadata["time"] for res in collected_res])
print(f"{avg_time} seconds")
asyncio.run(main())

1
functionality/multiagent_concurrent/requirements.txt Normal file
View File

@@ -0,0 +1 @@
agentscope[full]>=1.0.5

30
functionality/plan/README.md Normal file
View File

@@ -0,0 +1,30 @@
# Plan with ReAct Agent
This example demonstrates how to use the plan module in AgentScope to make an agent create and manage a plan formally.
Specifically, we provide two examples: manual specification plan and Agent-managed plan.
## Manual Specification Plan
In this example, we first manually specify a plan for the agent to follow, then we let the agent execute the plan step by step.
To execute this example, run:
```bash
python main_manual_plan.py
```
## Agent-managed Plan
In this example, we let the agent create and manage its own plan.
Specifically, we use a query "Review the recent changes in AgentScope GitHub repository over the past month."
To run the example, execute:
```bash
python main_agent_managed_plan.py
```
> Note: The example is built with DashScope chat model. If you want to change the model in this example, don't forget
> to change the **formatter** at the same time! The corresponding relationship between built-in models and formatters
> are list in [our tutorial](https://doc.agentscope.io/tutorial/task_prompt.html#id1)

65
functionality/plan/main_agent_managed_plan.py Normal file
View File

@@ -0,0 +1,65 @@
# -*- coding: utf-8 -*-
"""The main entry point of the plan example."""
import asyncio
import os
from agentscope.agent import ReActAgent, UserAgent
from agentscope.formatter import DashScopeChatFormatter
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.plan import PlanNotebook
from agentscope.tool import (
Toolkit,
execute_python_code,
execute_shell_command,
insert_text_file,
view_text_file,
write_text_file,
)
async def main() -> None:
"""The main entry point for the plan example."""
toolkit = Toolkit()
toolkit.register_tool_function(execute_shell_command)
toolkit.register_tool_function(execute_python_code)
toolkit.register_tool_function(write_text_file)
toolkit.register_tool_function(insert_text_file)
toolkit.register_tool_function(view_text_file)
agent = ReActAgent(
name="Friday",
sys_prompt="""You're a helpful assistant named Friday.
# Target
Your target is to finish the given task with careful planning.
# Note
- You can equip yourself with plan related tools to help you plan and execute the given task.
- The resouces from search engines are not always correct, you should collect information from multiple sources and give the final answer after careful consideration.
""", # noqa
model=DashScopeChatModel(
model_name="qwen3-max-preview",
api_key=os.environ["DASHSCOPE_API_KEY"],
),
formatter=DashScopeChatFormatter(),
toolkit=toolkit,
enable_meta_tool=True,
plan_notebook=PlanNotebook(),
)
user = UserAgent(name="user")
msg = Msg(
"user",
"Review the recent changes in AgentScope GitHub repository "
"over the past month.",
"user",
)
while True:
msg = await agent(msg)
msg = await user(msg)
if msg.get_text_content() == "exit":
break
asyncio.run(main())

101
functionality/plan/main_manual_plan.py Normal file
View File

@@ -0,0 +1,101 @@
# -*- coding: utf-8 -*-
"""Manual specification plan example."""
import asyncio
import os
from agentscope.agent import ReActAgent, UserAgent
from agentscope.formatter import DashScopeChatFormatter
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.plan import PlanNotebook, SubTask
from agentscope.tool import (
Toolkit,
execute_python_code,
execute_shell_command,
insert_text_file,
view_text_file,
write_text_file,
)
plan_notebook = PlanNotebook()
async def main() -> None:
"""The main entry point for the manual plan example."""
# Create the plan manually
await plan_notebook.create_plan(
name="Comprehensive Report on AgentScope",
description="Study the code of AgentScope and write a comprehensive "
"report about this framework.",
expected_outcome="A markdown format report summarizing the features, "
"architecture, advantages/disadvantages, and "
"potential improvements of AgentScope.",
subtasks=[
SubTask(
name="Clone the repository",
description="Clone the AgentScope GitHub repository from "
"agentscope-ai/agentscope, and ensure it's the "
"latest version.",
expected_outcome="A local copy of the AgentScope repository.",
),
SubTask(
name="View the documentation",
description="View the documentation of AgentScope in the "
"repository.",
expected_outcome="A comprehensive understanding of the "
"features and usage of AgentScope.",
),
SubTask(
name="Study the code",
description="Study the code of AgentScope, focusing on the "
"core modules and their interactions.",
expected_outcome="A deep understanding of the architecture "
"and implementation of AgentScope.",
),
SubTask(
name="Summarize the findings",
description="Summarize the findings from the documentation "
"and code study, and write a comprehensive report "
"in markdown format.",
expected_outcome="A markdown format report",
),
],
)
# Add basic tools
toolkit = Toolkit()
toolkit.register_tool_function(execute_shell_command)
toolkit.register_tool_function(execute_python_code)
toolkit.register_tool_function(write_text_file)
toolkit.register_tool_function(insert_text_file)
toolkit.register_tool_function(view_text_file)
# Create the agent
agent = ReActAgent(
name="Friday",
sys_prompt="You're a helpful assistant named Friday. Your target is "
"to finish the given task with careful planning.",
model=DashScopeChatModel(
model_name="qwen3-max-preview",
api_key=os.environ["DASHSCOPE_API_KEY"],
),
formatter=DashScopeChatFormatter(),
toolkit=toolkit,
plan_notebook=plan_notebook,
)
user = UserAgent(name="user")
msg = Msg(
"user",
"Now start to finish the task by the given plan",
"user",
)
while True:
msg = await agent(msg)
msg = await user(msg)
if msg.get_text_content() == "exit":
break
asyncio.run(main())

1
functionality/plan/requirements.txt Normal file
View File

@@ -0,0 +1 @@
agentscope[full]>=1.0.5

40
functionality/rag/README.md Normal file
View File

@@ -0,0 +1,40 @@
# RAG in AgentScope
This example includes three scripts to demonstrate how to use Retrieval-Augmented Generation (RAG) in AgentScope:
- the basic usage of RAG module in AgentScope in ``basic_usage.py``,
- a simple agentic use case of RAG in ``agentic_usage.py``, and
- integrate RAG into ``ReActAgent`` class by retrieving input message(s) at the beginning of each reply in ``react_agent_integration.py``.
- build multimodal RAG in ``multimodal_rag.py``.
> The agentic usage and static integration has their own advantages and limitations.
> - The agentic usage requires more powerful LLMs to manage the retrieval process, but it's more flexible and the agent can adjust the retrieval strategy dynamically
> - The static integration is more straightforward and easier to implement, but it's less flexible and the input message maybe not specific enough, leading to less relevant retrieval results.
> Note: The example is built with DashScope chat model. If you want to change the model in this example, don't forget
> to change the formatter at the same time! The corresponding relationship between built-in models and formatters are
> list in [our tutorial](https://doc.agentscope.io/tutorial/task_prompt.html#id1)
## Quick Start
Install the latest agentscope library from PyPI or source, then run the following command to run the example:
- the basic usage:
```bash
python basic_usage.py
```
- the agentic usage:
```bash
python agentic_usage.py
```
- the static integration:
```bash
python react_agent_integration.py
```
- the multimodal RAG:
```bash
python multimodal_rag.py
```

101
functionality/rag/agentic_usage.py Normal file
View File

@@ -0,0 +1,101 @@
# -*- coding: utf-8 -*-
"""The agentic usage example for RAG in AgentScope, where the agent is
equipped with RAG tools to answer questions based on a knowledge base.
The example is more challenging for the agent, requiring the agent to
adjust the retrieval parameters to get relevant results.
"""
import asyncio
import os
from agentscope.agent import ReActAgent, UserAgent
from agentscope.embedding import DashScopeTextEmbedding
from agentscope.formatter import DashScopeChatFormatter
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.rag import QdrantStore, SimpleKnowledge, TextReader
from agentscope.tool import Toolkit
# Create a knowledge base instance
knowledge = SimpleKnowledge(
embedding_store=QdrantStore(
location=":memory:",
collection_name="test_collection",
dimensions=1024, # The dimension of the embedding vectors
),
embedding_model=DashScopeTextEmbedding(
api_key=os.environ["DASHSCOPE_API_KEY"],
model_name="text-embedding-v4",
),
)
async def main() -> None:
"""The main entry of the agent usage example for RAG in AgentScope."""
# Store some things into the knowledge base for demonstration
# In practice, the VDB store would be pre-filled with relevant data
reader = TextReader(chunk_size=1024, split_by="sentence")
documents = await reader(
text=(
# Fake personal profile for demonstration
"I'm John Doe, 28 years old. My best friend is James "
"Smith. I live in San Francisco. I work at OpenAI as a "
"software engineer. I love hiking and photography. "
"My father is Michael Doe, a doctor. I'm very proud of him. "
"My mother is Sarah Doe, a teacher. She is very kind and "
"always helps me with my studies.\n"
"I'm now a PhD student at Stanford University, majoring in "
"Computer Science. My advisor is Prof. Jane Williams, who is "
"a leading expert in artificial intelligence. I have published "
"several papers in top conferences, such as NeurIPS and ICML. "
),
)
await knowledge.add_documents(documents)
# Create a toolkit and register the RAG tool function
toolkit = Toolkit()
toolkit.register_tool_function(
knowledge.retrieve_knowledge,
func_description=( # Provide a clear description for the tool
"Retrieve relevant documents from the knowledge base, which is "
"relevant to John Doe's profile. Note the `query` parameter is "
"very important for the retrieval quality, and you can try many "
"different queries to get the best results. Adjust the `limit` "
"and `score_threshold` parameters to get more or fewer results."
),
)
# Create an agent and a user
agent = ReActAgent(
name="Friday",
sys_prompt=(
"You're a helpful assistant named Friday. "
"You're equipped with a 'retrieve_knowledge' tool to help you "
"know about the user named John Doe. "
"NOTE to adjust the `score_threshold` parameters when you cannot "
"get relevant results. "
),
toolkit=toolkit,
model=DashScopeChatModel(
api_key=os.environ["DASHSCOPE_API_KEY"],
model_name="qwen3-max-preview",
),
formatter=DashScopeChatFormatter(),
)
user = UserAgent(name="User")
# A simple conversation loop beginning with a preset question
msg = Msg(
"user",
"I'm John Doe. Do you know my father?",
"user",
)
while True:
msg = await agent(msg)
msg = await user(msg)
if msg.get_text_content() == "exit":
break
asyncio.run(main())

74
functionality/rag/basic_usage.py Normal file
View File

@@ -0,0 +1,74 @@
# -*- coding: utf-8 -*-
"""The main entry point of the RAG example."""
import asyncio
import os
from agentscope.embedding import DashScopeTextEmbedding
from agentscope.rag import PDFReader, QdrantStore, SimpleKnowledge, TextReader
async def main() -> None:
"""The main entry point of the RAG example."""
# Create readers with chunking arguments
reader = TextReader(chunk_size=1024)
pdf_reader = PDFReader(chunk_size=1024, split_by="sentence")
# Read documents
documents = await reader(
text="I'm Tony Stank, my password is 123456. My best friend is James "
"Rhodes.",
)
# Read a sample PDF file
pdf_path = os.path.join(
os.path.abspath(os.path.dirname(__file__)),
"example.pdf",
)
pdf_documents = await pdf_reader(pdf_path=pdf_path)
# Create a knowledge base with Qdrant as the embedding store and
# DashScope as the embedding model
knowledge = SimpleKnowledge(
embedding_store=QdrantStore(
location=":memory:",
collection_name="test_collection",
dimensions=1024, # The dimension of the embedding vectors
),
embedding_model=DashScopeTextEmbedding(
api_key=os.environ["DASHSCOPE_API_KEY"],
model_name="text-embedding-v4",
),
)
# Insert documents into the knowledge base
await knowledge.add_documents(documents + pdf_documents)
# Retrieve relevant documents based on a given query
docs = await knowledge.retrieve(
query="What is Tony Stank's password?",
limit=3,
score_threshold=0.7,
)
print("Q1: What is Tony Stank's password?")
for doc in docs:
print(
f"Document ID: {doc.id}, Score: {doc.score}, "
f"Content: {doc.metadata.content['text']}",
)
# Retrieve documents from the PDF file based on a query
docs = await knowledge.retrieve(
query="climate change",
limit=3,
score_threshold=0.2,
)
print("\n\nQ2: climate change")
for doc in docs:
print(
f"Document ID: {doc.id}, Score: {doc.score}, "
f"Content: {repr(doc.metadata.content['text'])}",
)
asyncio.run(main())

BIN
functionality/rag/example.pdf Normal file
View File

Binary file not shown.

70
functionality/rag/multimodal_rag.py Normal file
View File

@@ -0,0 +1,70 @@
# -*- coding: utf-8 -*-
"""The example of how to use multimodal RAG in AgentScope"""
import asyncio
import json
import os
from agentscope.agent import ReActAgent
from agentscope.embedding import DashScopeMultiModalEmbedding
from agentscope.formatter import DashScopeChatFormatter
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.rag import ImageReader, QdrantStore, SimpleKnowledge
from matplotlib import pyplot as plt
path_image = "./example.png"
plt.figure(figsize=(8, 3))
plt.text(0.5, 0.5, "My name is Ming Li", ha="center", va="center", fontsize=30)
plt.axis("off")
plt.savefig(path_image, bbox_inches="tight", pad_inches=0.1)
plt.close()
async def example_multimodal_rag() -> None:
"""Example for multimodal RAG"""
# Reading the image and converting it to documents
reader = ImageReader()
docs = await reader(image_url=path_image)
# Create a knowledge base and add documents
knowledge = SimpleKnowledge(
embedding_model=DashScopeMultiModalEmbedding(
api_key=os.environ["DASHSCOPE_API_KEY"],
model_name="multimodal-embedding-v1",
dimensions=1024,
),
embedding_store=QdrantStore(
location=":memory:",
collection_name="test_collection",
dimensions=1024,
),
)
await knowledge.add_documents(docs)
agent = ReActAgent(
name="Friday",
sys_prompt="You're a helpful assistant named Friday.",
model=DashScopeChatModel(
api_key=os.environ["DASHSCOPE_API_KEY"],
model_name="qwen3-vl-plus",
),
formatter=DashScopeChatFormatter(),
knowledge=knowledge,
)
await agent(
Msg(
"user",
"Do you know my name?",
"user",
),
)
# Let's see if the agent has stored the retrieved document in its memory
print("\nThe retrieved document stored in the agent's memory:")
content = (await agent.memory.get_memory())[-4].content
print(json.dumps(content, indent=2, ensure_ascii=False))
asyncio.run(example_multimodal_rag())

78
functionality/rag/react_agent_integration.py Normal file
View File

@@ -0,0 +1,78 @@
# -*- coding: utf-8 -*-
"""The example of integrating ReAct agent with RAG."""
import asyncio
import os
from agentscope.agent import ReActAgent, UserAgent
from agentscope.embedding import DashScopeTextEmbedding
from agentscope.formatter import DashScopeChatFormatter
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.rag import QdrantStore, SimpleKnowledge, TextReader
async def main() -> None:
"""The main entry point for the ReAct agent with RAG example."""
# Create an in-memory knowledge base instance
print("Creating the knowledge base...")
knowledge = SimpleKnowledge(
embedding_store=QdrantStore(
location=":memory:",
collection_name="test_collection",
dimensions=1024, # The dimension of the embedding vectors
),
embedding_model=DashScopeTextEmbedding(
api_key=os.environ["DASHSCOPE_API_KEY"],
model_name="text-embedding-v4",
),
)
# Insert some documents into the knowledge base
# This could be done offline and only once
print("Inserting documents into the knowledge base...")
reader = TextReader(chunk_size=100, split_by="char")
documents = await reader(
# Fake personal profile for demonstration
"I'm John Doe, 28 years old. My best friend is James "
"Smith. I live in San Francisco. I work at OpenAI as a "
"software engineer. I love hiking and photography. "
"My father is Michael Doe, a doctor. I'm very proud of him. "
"My mother is Sarah Doe, a teacher. She is very kind and "
"always helps me with my studies.\n"
"I'm now a PhD student at Stanford University, majoring in "
"Computer Science. My advisor is Prof. Jane Williams, who is "
"a leading expert in artificial intelligence. I have published "
"several papers in top conferences, such as NeurIPS and ICML. ",
)
print("Inserting documents into the knowledge base...")
await knowledge.add_documents(documents)
# Integrate into the ReActAgent by the `knowledge` argument
print("Creating the agent...")
agent = ReActAgent(
name="Friday",
sys_prompt="You are a helpful assistant named Friday.",
model=DashScopeChatModel(
api_key=os.environ["DASHSCOPE_API_KEY"],
model_name="qwen-max",
),
formatter=DashScopeChatFormatter(),
# Equip the agent with the knowledge base
knowledge=knowledge,
print_hint_msg=True,
)
user = UserAgent(name="user")
# Start the conversation
print("Start the conversation...")
msg = Msg("user", "Do you know who is my best friend?", "user")
while True:
msg = await agent(msg)
msg = await user(msg)
if msg.get_text_content() == "exit":
break
asyncio.run(main())

4
functionality/rag/requirements.txt Normal file
View File

@@ -0,0 +1,4 @@
agentscope[full]>=1.0.5
matplotlib >= 3.10.7
nltk >= 3.9.2
pypdf >= 6.1.1

25
functionality/session_with_sqlite/README.md Normal file
View File

@@ -0,0 +1,25 @@
# Session Management with Sqlite DB
This example demonstrates how to implement session management with a database backend. We use SQLite for simplicity,
but the approach can be adapted for other databases.
Specifically, we implement a ``SqliteSession`` class that persists and retrieves session data from a SQLite table.
The table schema includes fields for session ID, session data (stored as JSON), and timestamps for creation and last
update.
We will create a simple agent and chat with it, then store the session data in the SQLite database. Then in the
``test_load_session`` function, we will load the session data from the database and continue the chat.
## Quick Start
Install agentscope from Pypi or source code.
```bash
pip install agentscope
```
Run the example by the following command
```bash
python main.py
```

75
functionality/session_with_sqlite/main.py Normal file
View File

@@ -0,0 +1,75 @@
# -*- coding: utf-8 -*-
"""The main entry point for the session with SQLite example."""
import asyncio
import os
from agentscope.agent import ReActAgent
from agentscope.formatter import DashScopeChatFormatter
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from sqlite_session import SqliteSession
SQLITE_PATH = "./session.db"
async def main(username: str, query: str) -> None:
"""Create an agent, load from session, chat with it, and save its state
to SQLite.
Args:
username (`str`):
The username to identify the session.
query (`str`):
The user input query.
"""
agent = ReActAgent(
name="friday",
sys_prompt="You are a helpful assistant named Friday.",
model=DashScopeChatModel(
model_name="qwen-max",
api_key=os.environ["DASHSCOPE_API_KEY"],
),
formatter=DashScopeChatFormatter(),
)
# Create the SQLite session
session = SqliteSession(SQLITE_PATH)
# Load the agent state by the given key "friday_of_user"
# The load_session_state supports multiple state modules
await session.load_session_state(
session_id=username,
friday_of_user=agent,
)
# Chat with it to generate some state
await agent(
Msg("user", query, "user"),
)
# Save the agent state by the given key "friday_of_user"
# Also support multiple state modules (e.g. multiple agents)
await session.save_session_state(
session_id=username,
friday_of_user=agent,
)
print("User named Alice chats with the agent ...")
asyncio.run(main("alice", "What's the capital of America?"))
print("User named Bob chats with the agent ...")
asyncio.run(main("bob", "What's the capital of China?"))
print(
"\nNow, let's recover the session for Alice and ask about what the user "
"asked before.",
)
asyncio.run(
main(
"alice",
"What did I ask you before, what's your answer and how many "
"questions have I asked you?",
),
)

1
functionality/session_with_sqlite/requirements.txt Normal file
View File

@@ -0,0 +1 @@
agentscope[full]>=1.0.5

167
functionality/session_with_sqlite/sqlite_session.py Normal file
View File

@@ -0,0 +1,167 @@
# -*- coding: utf-8 -*-
"""The SQLite session class."""
import json
import os
import sqlite3
from agentscope import logger
from agentscope.module import StateModule
from agentscope.session import SessionBase
class SqliteSession(SessionBase):
"""A session that uses SQLite for storage."""
def __init__(
self,
sqlite_path: str,
) -> None:
"""Initialize the session.
Args:
sqlite_path (`str`):
The path to the SQLite database file.
"""
self.sqlite_path = sqlite_path
async def save_session_state(
self,
session_id: str,
**state_modules_mapping: StateModule,
) -> None:
"""Save the session state to the SQLite database."""
with sqlite3.connect(self.sqlite_path) as conn:
cursor = conn.cursor()
# Prepare the session data as a dictionary
session_data = {
name: module.state_dict()
for name, module in state_modules_mapping.items()
}
json_data = json.dumps(session_data)
cursor.execute(
"""
CREATE TABLE IF NOT EXISTS as_session (
session_id TEXT,
session_data JSON,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (session_id)
)
""",
)
# Insert or replace the session data
cursor.execute(
"""
INSERT INTO as_session (session_id, session_data, updated_at)
VALUES (?, json(?), CURRENT_TIMESTAMP)
ON CONFLICT(session_id) DO UPDATE SET
session_data = excluded.session_data,
updated_at = excluded.updated_at
""",
(session_id, json_data),
)
conn.commit()
cursor.close()
async def load_session_state(
self,
session_id: str,
allow_not_exist: bool = True,
**state_modules_mapping: StateModule,
) -> None:
"""Get the state dictionary from the SQLite database.
Args:
session_id (`str`):
The session id.
allow_not_exist (`bool`, defaults to `True`):
Whether to allow the session to not exist. If `False`, raises
an error if the session does not exist.
**state_modules_mapping (`list[StateModule]`):
The list of state modules to be loaded.
"""
if not os.path.exists(self.sqlite_path):
if allow_not_exist:
logger.info(
"SQLite database %s does not exist. "
"Skipping load for session_id %s.",
self.sqlite_path,
session_id,
)
return
raise ValueError(
"Failed to load session state because the SQLite database "
f"file '{self.sqlite_path}' does not exist.",
)
with sqlite3.connect(self.sqlite_path) as conn:
cursor = conn.cursor()
try:
# If the table does not exist, return
cursor.execute(
"""
SELECT name FROM sqlite_master WHERE type='table' AND
name='as_session';
""",
)
if cursor.fetchone() is None:
if allow_not_exist:
logger.info(
"Session table does not exist in database %s. "
"Skipping load for session_id %s.",
self.sqlite_path,
session_id,
)
return
raise ValueError(
"Failed to load session state because the session "
"table 'as_session' does not exist in database "
f"{self.sqlite_path}.",
)
# Query the session data
cursor.execute(
"SELECT session_data FROM as_session WHERE session_id = ?",
(session_id,),
)
row = cursor.fetchone()
if row is None:
if allow_not_exist:
logger.info(
"Session_id %s does not exist in database %s. "
"Skip loading.",
session_id,
self.sqlite_path,
)
return
raise ValueError(
f"Failed to load session state for session_id "
f"{session_id} does not exist.",
)
session_data = json.loads(row[0])
for name, module in state_modules_mapping.items():
if name in session_data:
module.load_state_dict(session_data[name])
else:
raise ValueError(
f"State module '{name}' not found in session "
"data.",
)
logger.info(
"Load session state for session_id %s from "
"database %s successfully.",
session_id,
self.sqlite_path,
)
finally:
cursor.close()

28
functionality/stream_printing_messages/README.md Normal file
View File

@@ -0,0 +1,28 @@
# Stream Printing Messages
The AgentScope agent is designed to communicate with the user and the other agents by passing messages explicitly.
However, we notice the requirements that obtain the printing messages from the agent in a streaming manner.
Therefore, in example we demonstrate how to gather and yield the printing messages from a single agent and
multi-agent systems in a streaming manner.
## Quick Start
Run the following command to see the streaming printing messages from the agent.
Note the messages with the same ID are the chunks of the same message in accumulated manner.
- For single-agent:
```bash
python single_agent.py
```
- For multi-agent:
```bash
python multi_agent.py
```
> Note: The example is built with DashScope chat model. If you want to change the model in this example, don't forget
> to change the formatter at the same time! The corresponding relationship between built-in models and formatters are
> list in [our tutorial](https://doc.agentscope.io/tutorial/task_prompt.html#id1)

62
functionality/stream_printing_messages/multi_agent.py Normal file
View File

@@ -0,0 +1,62 @@
# -*- coding: utf-8 -*-
"""Example for gather the printing messages from multiple agents."""
import asyncio
import os
from agentscope.agent import ReActAgent
from agentscope.formatter import DashScopeMultiAgentFormatter
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.pipeline import MsgHub, stream_printing_messages
def create_agent(name: str) -> ReActAgent:
"""Create an agent with the given name."""
return ReActAgent(
name=name,
sys_prompt=f"You are a student named {name}.",
model=DashScopeChatModel(
api_key=os.environ["DASHSCOPE_API_KEY"],
model_name="qwen-max",
stream=False, # close streaming for simplicity
),
formatter=DashScopeMultiAgentFormatter(),
)
async def workflow(
alice: ReActAgent,
bob: ReActAgent,
charlie: ReActAgent,
) -> None:
"""The example workflow for multiple agents."""
async with MsgHub(
participants=[alice, bob, charlie],
announcement=Msg(
"user",
"Alice, Bob and Charlie, welcome to the meeting! Let's "
"meet each other first.",
"user",
),
):
# agent speaks in turn
await alice()
await bob()
await charlie()
async def main() -> None:
"""The main entry for the example."""
# Create agents
alice, bob, charlie = [
create_agent(_) for _ in ["Alice", "Bob", "Charlie"]
]
async for msg, last in stream_printing_messages(
agents=[alice, bob, charlie],
coroutine_task=workflow(alice, bob, charlie),
):
print(msg, last)
asyncio.run(main())

1
functionality/stream_printing_messages/requirements.txt Normal file
View File

@@ -0,0 +1 @@
agentscope[full]>=1.0.5

62
functionality/stream_printing_messages/single_agent.py Normal file
View File

@@ -0,0 +1,62 @@
# -*- coding: utf-8 -*-
"""The example demonstrating how to obtain the messages from the agent in a
streaming way."""
import asyncio
import os
from agentscope.agent import ReActAgent
from agentscope.formatter import DashScopeChatFormatter
from agentscope.memory import InMemoryMemory
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.pipeline import stream_printing_messages
from agentscope.tool import (
Toolkit,
execute_python_code,
execute_shell_command,
view_text_file,
)
async def main() -> None:
"""The main function."""
toolkit = Toolkit()
toolkit.register_tool_function(execute_shell_command)
toolkit.register_tool_function(execute_python_code)
toolkit.register_tool_function(view_text_file)
agent = ReActAgent(
name="Friday",
sys_prompt="You are a helpful assistant named Friday.",
# Change the model and formatter together if you want to try other
# models
model=DashScopeChatModel(
api_key=os.environ.get("DASHSCOPE_API_KEY"),
model_name="qwen-max",
enable_thinking=False,
stream=True,
),
formatter=DashScopeChatFormatter(),
toolkit=toolkit,
memory=InMemoryMemory(),
)
# Prepare a user message
user_msg = Msg(
"user",
"Hi! Who are you?",
"user",
)
# We disable the terminal printing to avoid messy outputs
agent.set_console_output_enabled(False)
# obtain the printing messages from the agent in a streaming way
async for msg, last in stream_printing_messages(
agents=[agent],
coroutine_task=agent(user_msg),
):
print(msg, last)
asyncio.run(main())

96
functionality/structured_output/README.md Normal file
View File

@@ -0,0 +1,96 @@
# Structured Output Example
## What This Example Demonstrates
This example showcases **structured output generation** using AgentScope with Pydantic models. It demonstrates how to constrain AI model outputs to follow specific data structures and formats, ensuring consistent and parseable responses.
### Key Features:
- **Structured Data Generation**: Forces agent responses to conform to
predefined schemas
- **Pydantic Integration**: Uses Pydantic models to define output structure with validation
- **Type Safety**: Ensures output data types match expected formats
- **Field Validation**: Includes constraints like age limits (0-120) and enum choices
- **JSON Output**: Generates clean, structured JSON responses
### Example Models:
1. **TableModel**: Structured person information
- `name`: Person's name (string)
- `age`: Person's age (integer,0-120)
- `intro`: One-sentence introduction (string)
- `honors`: List of honors/achievements (array of strings)
2. **ChoiceModel**: Constrained choice selection
- `choice`: Must be one of "apple", "banana", or "orange"
### Use Cases:
- **Data Extraction**: Extract structured information from unstructured text
- **Form Generation**: Generate consistent data for databases or APIs
- **Survey Responses**: Ensure responses fit predefined categories
- **Content Classification**: Categorize content into specific types
## How to Run This Example
1. **Set Environment Variable:**
```bash
export DASHSCOPE_API_KEY="your_dashscope_api_key_here"
```
2. **Run the script:**
```bash
python main.py
```
3. **Expected Output:**
The program will generate two structured responses like below:
```
Structured Output 1:
{
"name": "Albert Einstein",
"age": 76,
"intro": 1,
"honors": [
"Nobel Prize in Physics (1921)",
"Copley Medal (1925)"
]
}
Structured Output 2:
{
"choice": "apple"
}
```
>💡**Note:** The specific content will vary with each run since the agent generates different responses, but the JSON structure will always conform to the predefined Pydantic models (`TableModel` and `ChoiceModel`).
## How It Works:
1. The agent receives a query along with a structured_model parameter
2. The agent generates a response that conforms to the Pydantic model schema
3. The structured data is returned in res.metadata as a validated JSON object
4. Pydantic ensures all field types and constraints are satisfied
## Custom Pydantic Models
Create your own structured output models for specific use cases, for example:
```
from typing import Optional
from pydantic import BaseModel, Field, EmailStr
class BusinessModel(BaseModel):
"""Business information extraction model."""
company_name: str = Field(description="Name of the company")
industry: str = Field(description="Industry sector")
founded_year: int = Field(description="Year founded", ge=1800, le=2024)
headquarters: str = Field(description="Location of headquarters")
employee_count: Optional[int] = Field(description="Number of employees", ge=1)
email: Optional[EmailStr] = Field(description="Contact email address")
website: Optional[str] = Field(description="Company website URL")
# Usage
query = Msg("user", "Tell me about Tesla Inc.", "user")
res = await agent(query, structured_model=BusinessModel)
```
## Best Practices for Structured Output
1. **Use Descriptive Field Names:** Make field purposes clear
2. **Add Field Descriptions:** Help the agent understand what data to generate
3. **Set Validation Constraints:** Use Pydantic validators for data integrity
4. **Choose Appropriate Types:** Use specific types like EmailStr, datetime, etc.

79
functionality/structured_output/main.py Normal file
View File

@@ -0,0 +1,79 @@
# -*- coding: utf-8 -*-
"""The main entry point of the structured output example."""
import asyncio
import json
import os
from typing import Literal
from agentscope.agent import ReActAgent
from agentscope.formatter import DashScopeChatFormatter
from agentscope.memory import InMemoryMemory
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from agentscope.tool import Toolkit
from pydantic import BaseModel, Field
class TableModel(BaseModel):
"""A simple table model for structured output."""
name: str = Field(description="The name of the person")
age: int = Field(description="The age of the person", ge=0, le=120)
intro: str = Field(description="A one-sentence introduction of the person")
honors: list[str] = Field(
description="A list of honors received by this person",
)
class ChoiceModel(BaseModel):
"""A simple choice model for structured output."""
choice: Literal["apple", "banana", "orange"] = Field(
description="Your choice of fruit",
)
async def main() -> None:
"""The main entry point for the structured output example."""
toolkit = Toolkit()
agent = ReActAgent(
name="Friday",
sys_prompt="You are a helpful assistant named Friday.",
model=DashScopeChatModel(
api_key=os.environ.get("DASHSCOPE_API_KEY"),
model_name="qwen-max",
stream=True,
),
formatter=DashScopeChatFormatter(),
toolkit=toolkit,
memory=InMemoryMemory(),
)
query_msg_1 = Msg(
"user",
"Please introduce Einstein",
"user",
)
res = await agent(query_msg_1, structured_model=TableModel)
print(
"Structured Output 1:\n"
"```\n"
f"{json.dumps(res.metadata, indent=4)}\n"
"```",
)
query_msg_2 = Msg(
"user",
"Choose one of your favorite fruit",
"user",
)
res = await agent(query_msg_2, structured_model=ChoiceModel)
print(
"Structured Output 2:\n"
"```\n"
f"{json.dumps(res.metadata, indent=4)}\n"
"```",
)
asyncio.run(main())

1
functionality/structured_output/requirements.txt Normal file
View File

@@ -0,0 +1 @@
agentscope[full]>=1.0.5