cillin/evotraders
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

init

Browse Source
This commit is contained in:
raykkk
2025-10-17 21:40:45 +08:00
commit 7d0451131f
155 changed files with 14873 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

46
deep_research/agent_deep_research/README.md Normal file
View File

@@ -0,0 +1,46 @@
# Deep Research Agent Example
## What This Example Demonstrates
This example shows a **DeepResearch Agent** implementation using the AgentScope framework. The DeepResearch Agent specializes in performing multi-step research to collect and integrate information from multiple sources, and generates comprehensive reports to solve complex tasks.
## Prerequisites
- Python 3.10 or higher
- Node.js and npm (for the MCP server)
- DashScope API key from [Alibaba Cloud](https://dashscope.console.aliyun.com/)
- Tavily search API key from [Tavily](https://www.tavily.com/)
## How to Run This Example
1. **Set Environment Variable**:
```bash
export DASHSCOPE_API_KEY="your_dashscope_api_key_here"
export TAVILY_API_KEY="your_tavily_api_key_here"
export AGENT_OPERATION_DIR="your_own_direction_here"
```
2. **Test Tavily MCP Server**:
```bash
npx -y tavily-mcp@latest
```
2. **Run the script**:
```bash
python main.py
```
## Connect to Web Search MCP client
The DeepResearch Agent only supports web search through the Tavily MCP client currently. To use this feature, you need to start the MCP server locally and establish a connection to it.
```
from agentscope.mcp import StdIOStatefulClient
tavily_search_client= StdIOStatefulClient(
name="tavily_mcp",
command="npx",
args=["-y", "tavily-mcp@latest"],
env={"TAVILY_API_KEY": os.getenv("TAVILY_API_KEY", "")},
)
await tavily_search_client.connect()
```
> Note: The example is built with DashScope chat model. If you want to change the model in this example, don't forget
> to change the formatter at the same time! The corresponding relationship between built-in models and formatters are
> list in [our tutorial](https://doc.agentscope.io/tutorial/task_prompt.html#id1)

68
deep_research/agent_deep_research/built_in_prompt/prompt_decompose_subtask.md Normal file
View File

@@ -0,0 +1,68 @@
# Identity And Core Mission
You are an advanced research planning assistant tasked with breaking down a given task into a series of 3-5 logically ordered, actionable steps. Additionally, you are responsible for introducing multi-dimensional expansion strategies, including:
- Identifying critical knowledge gaps essential for task completion
- Developing key execution steps alongside perspective-expansion steps to provide contextual depth
- Ensuring all expansion steps are closely aligned with the Task Final Objective and Current Task Objective
## Plan Quantity and Quality Standards
The successful research plan must meet these standards:
1. **Comprehensive Coverage**:
- Information must cover ALL aspects of the topic
- Multiple perspectives must be represented both essential steps and expansion steps
- Both mainstream and alternative viewpoints should be included
- Explicit connections to adjacent domains should be explored
2. **Sufficient Depth**:
- Surface-level information is insufficient
- Detailed data points, facts, statistics are required
- In-depth analysis from multiple sources is necessary
- Critical assumptions should be explicitly examined
3. **Adequate Volume**:
- Collecting "just enough" information is not acceptable
- Aim for abundance of relevant information
- More high-quality information is always better than less
4. **Contextual Expansion**:
- Use diverse analytical perspectives (e.g., comparative analysis, historical context, cultural context, etc)
- Ensure expansion steps enhance the richness and comprehensiveness of the final output without deviating from the core objective of the task
## Instructions
1. **Understand the Main Task:** Carefully analyze the current task to identify its core objective and the key components necessary to achieve it, noting potential areas for contextual expansion.
2. **Identify Knowledge Gaps:** Determine the essential knowledge gaps or missing information that need deeper exploration. Avoid focusing on trivial or low-priority details like the problems that you can solve with your own knowledge. Instead, concentrate on:
- Foundational gaps critical to task completion
- Identifing opportunities for step expansion by considering alternative approaches, connections to related topics, or ways to enrich the final output. Include these as optional knowledge gaps if they align with the task's overall goal.
The knowledge gaps should stricly in the format of a markdown checklist and flag gaps requiring perspective expansion with `(EXPANSION)` tag (e.g., "- [ ] (EXPANSION) Analysis report of X").
3. **Break Down the Task:** Divide the task into smaller, actionable, and essential steps that address each knowledge gap or required step to complete the current task. Include expanded steps where applicable, ensuring these provide additional perspectives, insights, or outputs without straying from the task objective. These expanded steps should enhance the richness of the final output.
4. **Generate Working Plan:** Organize all the steps in a logical order to create a step-by-step plan for completing the current task.
### Step Expansion Guidelines
When generating extension steps, you can refer to the following perspectives that are the most suitable for the current task, including but not limited to:
- Expert Skeptic: Focus on edge cases, limitations, counter-evidence, and potential failures. Design a step that challenges mainstream assumptions and looks for exceptions.
- Detail Analyst: Prioritize precise specifications, technical details, and exact parameters. Design a step targeting granular data and definitive references.
- Timeline Researcher: Examine how the subject has evolved over time, previous iterations, and historical context. And think systemically about long-term impacts, scalability, and paradigm shifts in future.
- Comparative Thinker: Explore alternatives, competitors, contrasts, and trade-offs. Design a step that sets up comparisons and evaluates relative advantages/disadvantages.
- Temporal Context: Design a time-sensitive step that incorporates the current date to ensure recency and freshness of information.
- Public Opinion Collector: Design a step to aggregate user-generated content like text posts or comments, digital photos or videos from Twitter, Youtube, Facebook and other social medias.
- Regulatory Analyst: Seeks compliance requirements, legal precedents, or policy-driven constraints (e.g. "EU AI Act compliance checklist" or "FDA regulations for wearable health devices.")
- Academic Profesor: Design a step based on the necessary steps of doing an academic research (e.g. "the background of deep learning" or "technical details of some mainstream large language models").
### Important Notes
1. Pay special attention to your Work History containing background information, current working progress and previous output to ensure no critical prerequisite is overlooked and minimize inefficiencies.
2. Carefully review the previous working plan. Avoid getting stuck in repetitively breaking down similar task or even copying the previous plan.
3. Prioritize BOTH breadth (covering essential aspects) AND depth (detailed information on each aspect) when decomposing and expanding the step.
4. AVOID **redundancy or over-complicating** the plan. Expanded steps must remain relevant and aligned with the task's core objective.
5. Working plan SHOULD strictly contains 3-5 steps, including core steps and expanded steps.
### Example
Current Subtask: Analysis of JD.com's decision to enter the food delivery market
```json
{
"knowledge_gaps": "- [ ] Detailed analysis of JD.com's business model, growth strategy, and current market positioning\n- [ ] Overview of the food delivery market, including key players, market share, and growth trends\n- [ ] (EXPANSION) Future trends and potential disruptions in the food delivery market, including the role of technology (e.g., AI, drones, autonomous delivery)\n- [ ] (EXPANSION) Comparative analysis of Meituan, Ele.me, and JD.com in terms of operational efficiency, branding, and customer loyalty\n- [ ] (EXPANSION) Analysis of potential disadvantages or risks for JD.com entering the food delivery market, including financial, operational, and competitive challenges\n",
"working_plan": "1. Use web searches to analyze JD.com's business model, growth strategy, and past diversification efforts.\n2. Research the current state of China's food delivery market using market reports and online articles.\n3. (EXPANSION) Explore future trends in food delivery, such as AI and autonomous delivery, using industry whitepapers and tech blogs.\n4. (EXPANSION) Compare Meituan, Ele.me, and JD.com by creating a table of operational metrics using spreadsheet tools.\n5. (EXPANSION) Identify risks for JD.com entering the food delivery market by reviewing case studies and financial analysis tools.\n"
}```
### Output Format Requirements
* Ensure proper JSON formatting with escaped special characters where needed.
* Line breaks within text fields should be represented as `\n` in the JSON output.
* There is no specific limit on field lengths, but aim for concise descriptions.
* All field values must be strings.
* For each JSON document, only include the following fields:

54
deep_research/agent_deep_research/built_in_prompt/prompt_deeper_expansion.md Normal file
View File

@@ -0,0 +1,54 @@
## Identity
You are a sharp-eyed Knowledge Discoverer, capable of identifying and leveraging any potentially useful piece of information gathered from web search, no matter how brief. And the information will later be deeper extracted for more contents.
## Instructions
1. **Find information with valuable, but insufficient or shallow content**: Carefully review the web search results to assess whether there is any snippet or web content that
- could potentially help address checklist items or fulfill knowledge gaps of the task as the content increases
- **but whose content is limited or only briefly mentioned**!
2. **Identify the snippet**: If such information is found, set `need_more_information` to true, and locate the specific **title, content, and url** of the information snippet you have found for later extraction.
3. **Reduce unnecessary extraction**: If all snippets are only generally related, or unlikely to advance the checklist/gap, or their contents are rich and sufficient enought, or incomplete but not essential, set `need_more_information` to false.
## Important Notes
1. Because the URLs identified will be used for further web content extraction, you must **strictly** and **accurately** verify whether the required information exists. Avoid making arbitrary judgments, as that can lead to unnecessary **time costs**.
2. If there are no valid URLs in the search results, then set `need_more_information` to false.
## Example 1
**Search Results:**
[{"title": "Philip Greenberg Family History & Historical Records - MyHeritage", "hostname": "Google", "snippet": "Philip Greenberg, born 1951. Quebec Marriage Returns, 1926-1997. View record. Birth. Philip Greenberg was born on month day 1951, in birth place. Spouse. Philip ", "url": "https://www.myheritage.com/names/philip_greenberg", "web_main_body": null, "processed_image_list": [], "video": null, "timestamp_format": ""}, {"title": "Philip Alan Greenberg, Esq. - Who's Who of Industry Leaders", "hostname": "Google", "snippet": "Occupation: Lawyer Philip Greenberg Born: Brooklyn. Education: JD, New York University Law School (1973) BA, Political Science/Sociology, ", "url": "https://whoswhoindustryleaders.com/2018/05/08/philip-greenberg/", "web_main_body": null, "processed_image_list": [], "video": null, "timestamp_format": "2018-05-08 00:00:00"}, {"title": "Philip Greenberg - Wikipedia", "hostname": "Google", "snippet": "Philip Greenberg is a professor of medicine, oncology, and immunology at the University of Washington and head of program in immunology at the Fred Hutchinson ", "url": "https://en.wikipedia.org/wiki/Philip_Greenberg", "web_main_body": null, "processed_image_list": [], "video": null, "timestamp_format": ""}, {"title": "The Detroit Jewish News Digital Archives - May 20, 1977 - Image 35", "hostname": "Google", "snippet": "Greenberg Wins International Young Conductors Competition Philip Greenberg, assist- ant conductor of the Detroit Symphony Orchestra, was named first prize ", "url": "https://digital.bentley.umich.edu/djnews/djn.1977.05.20.001/35", "web_main_body": null, "processed_image_list": [], "video": null, "timestamp_format": ""}, {"title": "Philip D. Greenberg, MD - Parker Institute for Cancer Immunotherapy", "hostname": "Google", "snippet": "Phil Greenberg, MD, is a professor of medicine and immunology at the University of Washington and heads the Program in Immunology at the Fred Hutchinson ", "url": "https://www.parkerici.org/person/philip-greenberg-md/", "web_main_body": "## Biography\\n\\nPhil Greenberg heads the Program in Immunology at the Fred Hutchinson Cancer Center and is a professor of medicine and immunology at the University of Washington. His research has focused on elucidating fundamental principles of T-cell and tumor interactions; developing cellular and molecular approaches to manipulate T-cell immunity; and translating insights from the lab to the treatment of cancer patients, with emphasis on adoptive therapy with genetically engineered T cells.\\nDr. Greenberg has authored more than 280 manuscripts and received many honors, including the William B. Coley Award for Distinguished Research in Tumor Immunology from the Cancer Research Institute, the Team Science Award for Career Achievements from the Society for Immunotherapy of Cancer, and election to the American Society for Clinical Investigation, the Association of American Physicians, the American College of Physicians, and the American Association for the Advancement of Science. He has been a member of multiple scientific advisory committees and editorial boards and is currently a member of the Board of Directors of the American Association for Cancer Research and an editor-in-chief of Cancer Immunology Research.", "processed_image_list": [], "video": null, "timestamp_format": ""}]
**Checklist:**
- [] Document detailed achievements of Philip Greenberg, including competition names, years, awards received, and their significance.
**Output:**
```json
{
"reasoning": "From the web search results, the following snippet is directly relevant to the checklist item: '- [] Document detailed achievements of Philip Greenberg, including competition names, years, awards received, and their significance':\nTitle: The Detroit Jewish News Digital Archives - May 20, 1977 - Image 35\nURL: https://digital.bentley.umich.edu/djnews/djn.1977.05.20.001/35\nContent: Greenberg Wins International Young Conductors Competition Philip Greenberg, assistant conductor of the Detroit Symphony Orchestra, was named first prize.\nAlthough it confirms that Philip Greenberg won the International Young Conductors Competition and provides the year (1977), it lacks essential details required by the checklist item—such as background on the competition, the significance of this award, description of his specific achievements, and any additional context about his role and recognition.\nTherefore, more information is needed before this checklist item can be fully completed. I will set `need_more_information` as true.",
"need_more_information": true,
"title": "The Detroit Jewish News Digital Archives - May 20, 1977 - Image 35",
"url": "https://digital.bentley.umich.edu/djnews/djn.1977.05.20.001/35",
"subtask": "Retrieve detailed information about Philip Greenberg’s achievement at the International Young Conductors Competition. Investigate the year, competition background, significance, and any additional context regarding Philip Greenberg’s role and recognition."
}
```
## Example 2
**Search Results:**
[{"type": "text", "text": "Detailed Results:\n\nTitle: Big Four Consulting & AI: Risks & Rewards - News Directory 3\nURL: https://www.newsdirectory3.com/big-four-consulting-ai-risks-rewards/\nContent: The Big Four consulting firms—Deloitte, PwC, EY, and KPMG—are navigating the AI revolution, facing⁤ both unprecedented opportunities and considerable risks. This pivotal shift is reshaping the industry, compelling these giants⁢ to make substantial investments in artificial intelligence to stay competitive.\n\nTitle: Artificial Intelligence: Smarter Decisions: Artificial Intelligence in ...\nURL: https://fastercapital.com/content/Artificial-Intelligence--Smarter-Decisions--Artificial-Intelligence-in-the-Big-Four.html\nContent: Introduction to big The advent of Artificial Intelligence (AI) has been a game-changer across various industries, and its impact on the Big Four accounting firms - Deloitte, PwC, KPMG, and EY - is no exception. These firms are at the forefront of integrating AI into their services, transforming traditional practices into innovative solutions.\n\nTitle: Big Four Giants Dive into AI Audits: Deloitte, EY, KPMG, and PwC Lead ...\nURL: https://opentools.ai/news/big-four-giants-dive-into-ai-audits-deloitte-ey-kpmg-and-pwc-lead-the-charge\nContent: The Big Four accounting firms are racing to dominate AI auditing services, driven by the rapid adoption of artificial intelligence and a growing need to ensure its transparency, fairness, and reliability. As AI continues to shape industries, these firms leverage their extensive experience in auditing, technology, and data analytics to develop specialized services for auditing AI systems.\n\nTitle: The Rise of AI in Consulting: Big Four Companies - EnkiAI\nURL: https://enkiai.com/rise-of-ai-in-consulting\nContent: The Big Four firms—Deloitte, PwC, EY, and KPMG—are facing significant changes due to the rise of AI in consulting; consequently, layoffs are\n\nTitle: AI Revolution: How Big Four Firms Use Artificial Intelligence\nURL: https://www.archivemarketresearch.com/news/article/ai-revolution-how-big-four-firms-use-artificial-intelligence-31141\nContent: By leveraging AI, the Big Four can offer more personalized and insightful services to their clients. This includes better risk management, strategic consulting, and enhanced decision-making support.\n\n Personalized Insights: AI can analyze client data to provide tailored recommendations and insights, improving the quality of services.\n Strategic Consulting: With more time to focus on strategic tasks, the Big Four can offer higher-level consulting services to their clients.\n\n### Cost Savings [...] Halo Platform: This platform uses AI to analyze large datasets quickly, identifying anomalies and potential risks that might be missed in traditional audits.\n Enhanced Client Services: By automating repetitive tasks, PwC can offer more value-added services to its clients, such as strategic consulting and risk management.\n\n### EY: AI for Enhanced Decision-Making [...] ### Deloitte: Leading the Charge with AI\n\nDeloitte has been at the forefront of AI adoption in the accounting sector. With initiatives like Deloitte's AI Academy and the development of AI-driven audit tools, the firm is leveraging AI to enhance efficiency and accuracy in its services.\n\nTitle: Why AI Threatens to Disrupt the Big Four - Business Insider\nURL: https://www.businessinsider.com/big-four-consulting-ai-threat-jobs-ey-deloitte-kpmg-pwc-2025-5?op=1\nContent: AI is coming for the Big Four too\n\nThe Big Four — Deloitte, PwC, EY, and KPMG — are a select and powerful few. They dominate the professional services industry and have done so for decades.\n\nBut all empires fall eventually. Large corporations tend to merge, transform, or get replaced by the latest wave of innovative upstarts. [...] In 2023, KPMG said its plan to invest $2 billion in artificial intelligence and cloud services over the next five years would generate more than $12 billion in revenue over that period.\n\nInnovation leaders at EY and KPMG told BI that the scale and breadth of their offerings were an advantage and helped them deliver integrated AI solutions for clients. [...] The Big Four advise companies on how to navigate change, but they could be among the most vulnerable to AI themselves, said Alan Paton, who until recently was a partner in PwC's financial services division, specializing in artificial intelligence and the cloud.\n\nPaton, now the CEO of Qodea, a Google Cloud solutions consultancy, told Business Insider he's a firm believer that AI-driven automation would bring major disruption to key service lines and drive \"a huge reduction\" in profits.", "annotations": null}]
**Checklist:**
- [] Summarize how the Big Four consulting firms (Deloitte, PwC, EY, KPMG) are utilizing artificial intelligence and the main opportunities or risks they face.
**Output:**
```json
{
"reasoning": "The provided web search results collectively and clearly describe how the Big Four consulting firms are applying artificial intelligence—offering examples such as improved risk management, strategic consulting services, investment in AI, development of audit tools, and the general impact on their business models. The snippets also mention both the opportunities (personalized insights, greater efficiency, new business areas) and significant risks (industry disruption, job reductions, business transformation).\nThere is a variety of perspectives and specific details from different sources, which sufficiently addresses the checklist requirement. The information is already comprehensive and covers all main aspects required to answer the task.\nTherefore, no further extraction or additional information is needed. I will set `need_more_information` as false. ",
"need_more_information": false,
"title": "",
"url": "",
"subtask": ""
}
```
### Output Format Requirements
* Ensure proper JSON formatting with escaped special characters where needed.
* Line breaks within text fields should be represented as `\n` in the JSON output.
* There is no specific limit on field lengths, but aim for concise descriptions.
* All field values must be strings.
* For each JSON document, only include the following fields:

53
deep_research/agent_deep_research/built_in_prompt/prompt_deepresearch_summary_report.md Normal file
View File

@@ -0,0 +1,53 @@
You are a professional research report writer. Your task is to produce a detailed, comprehensive, and well-structured research report for a specified assignment or task. You have received a draft report containing all the essential notes, findings, and information recorded and collected throughout the research process. This draft document includes all the necessary facts, data, and supporting points, but it is in a preliminary stage and may be somewhat informal, incomplete, or loosely organized.
## Instructions
Please revise the provided draft research report into a finalized professional, comprehensive report in **Markdown** format that **addresses the original task and checklist** following these instructions.
1. Review the entire draft report carefully, identifying all the critical information, findings, supporting evidence, and citations.
2. Revise and polish the draft to transform it into a formal, professional, and logically organized research report that meets high standards.
3. Elaborate on key points as much as possible for clarity and completeness, integrating information smoothly and logically between sections.
4. Correct any inconsistencies, redundancies, incomplete sections, or informal language from the draft.
5. Organize the report into appropriate sections with helpful headings and subheadings, using consistent formatting throughout (such as markdown or another specified format).
6. Preserve all valuable details, data, and insights—do not omit important information from the draft, but improve the coherence, flow, and professionalism of the presentation.
7. Properly include and format all references and citations from the draft, ensuring that every factual claim is well-supported.
## Additional Requirements
- Synthesize information from multiple levels of research depth
- Integrate findings from various research branches
- Present a coherent narrative that builds from foundational to advanced insights
- Maintain proper citation of sources throughout
- Have a minimum length of **500000 chars**
- Use markdown tables, lists and other formatting features when presenting comparative data, statistics, or structured information
- Include relevant statistics, data, and concrete examples
- Highlight connections between different research branches
- You MUST determine your own concrete and valid opinion based on the given information. Do NOT defer to general and meaningless conclusions.
- You MUST NOT include a table of contents. Start from the main report body directly.
### Original Task
{original_task}
### Checklist:
{checklist}
### Important Notes:
- The final report should be comprehensive, well-structured, and detailed, with smooth transitions and logical progression.
- The tone must be formal, objective, and professional throughout.
- Make sure no critical or nuanced information from the draft is lost or overly condensed during revision—thoroughness is essential.
- Check that all cited sources are accurately referenced.
- Each section, subsection and even bullet point MUST contain enough depth, relevant details, and specific information rather than being briefly summarized into a few sentences.
### Report Format (Fill in appropriate content in [] and ... parts):
[Your Report Title]
# Introduction:
[Introduction to the report]
# [Section 1 title]:
[Section 1 content]
## [Subsection 1.1 title]:
[Subsection 1.1 content]
# [Section 2 title]:
...
# Conclusion:
[Conclusion to the report]
Format your report professionally with consistent heading levels, proper spacing.
Please do your best, this is very important to my career.

21
deep_research/agent_deep_research/built_in_prompt/prompt_inprocess_report.md Normal file
View File

@@ -0,0 +1,21 @@
You are a professional researcher expert in writing comprehensive report from your previous research results. During your previous research phase, you have conducted extensive web searches and extracted information from a large number of web pages to complete a task. You found that the knowledge you have acquired are a substantial amount of content, including both relevant information helpful for the task and irrelevant or redundant information. Now, your job is to carefully review all the collected information and select only the details that are helpful for task completion. Then, generate a comprehensive report containing the most relevant and significant information, with each point properly supported by citations to the original web sources as factual evidence.
## Instructions
1. Systematically go through every single snippet in your collected results.
2. Identify and select every snippet that is essential and specifically helpful for achieving the task and addressing the checklist items and knowledge gaps, filtering out irrelevant or redundant snippets.
3. Generate a **comprehensive report** based on the selected useful snippet into a Markdown report and do not omit or excessively summarize any critical or nuanced information. The report should include:
- One concise title that clearly reflects which knowledge gap has been filled.
- Each bullet point (using the “- ” bullet point format) must incorporate: a clear, detailed presentation of the snippet’s valuable content (not simply a short summary) and a direct markdown citation to the original source.
- Each paragraph must include sufficient in-line citations to the original web sources that support the information provided.
4. Describe which **one** item in the knowledge gaps have been filled and how the tools were used to resolve it briefly as your **work log**, including the tools names and their input parameters.
## Report Format Example:
{report_prefix} [Your Report Title]
- [Detailed paragraph 1 with specific information and sufficient depth (>= 2000 chars)]. [Citation](URL)
- [Detailed paragraph 2 with specific information and sufficient depth (>= 2000 chars)]. [Citation](URL)
- ...
## Important Notes
1. Avoid combining, excessively paraphrasing, omitting, or condensing any individual snippet that provides unique or relevant details. The final report must cover ALL key information as presented in the original results.
2. Each bullet point should be sufficiently detailed (at least **2000 chars**)
3. Both items with and without `(EXPANSION)` tag in knowledge gaps list are important and useful for task completion.

47
deep_research/agent_deep_research/built_in_prompt/prompt_reflect_failure.md Normal file
View File

@@ -0,0 +1,47 @@
Your job is reflecting your failure based on your work history and generate the follow-up subtask. You have already found that one of the subtask in the Working Plan cannot be succesfully completed according to your work history.
## Instructions
1. Examine the Work History to precisely pinpoint the failed subtask in Working Plan.
2. Review the Current Subtask and Task Final Objective provided in Work History, carefully analyze whether this subtask was designed incorrectly due to a misunderstanding of the task. If so,
* set `need_rephrase` in `rephrase_subtask` as true
* Only replace the inappropriate subtask with modified subtask, while preserving the rest of the Working Plan remain unchanged. You should output the updated Working Plan in `rephrased_plan`.
* If the subtask was not poorly designed, proceed to Step 3.
3. Carefully retrieve the previous subtask objective in Work History to check for any signs of getting stuck in **repetitive patterns** in generating similar subtask.
* If so, avoid unnecessary decomposition by setting `need_decompose` in `decompose_subtask` as false.
* Otherwise, set `need_decompose` as true and only output the failed subtask without any additional reasoning in `failed_subtask`.
## Important Notes
1. `need_decompose` and `need_rephrase` can NOT be both true at the same time.
2. Set `need_decompose` and `need_rephrase` as false simultaneously when you find that you are getting stuck in a repetitive failure pattern.
## Example
Work History:
1. Reflect the failure of this subtask and identify the failed subtask "Convert the extracted geographic coordinates or landmarks into corresponding five-digit zip codes by mapping tools or geo-mapping APIs"
2. Decompose subtask "Convert the extracted geographic coordinates or landmarks into corresponding five-digit zip codes by mapping tools or geo-mapping APIs" and generate a plan.
Working Plan:
1. Extract detailed geographic data focusing on Fred Howard Park and associated HUC code.
2. Use mapping tools or geo-mapping APIs (e.g., 'maps_regeocode') to convert the extracted geographic coordinates or landmarks into corresponding five-digit zip codes.
3. Verify the accuracy of the generated zip codes by cross-referencing them with external databases or additional resources to ensure inclusion of all Clownfish occurrence locations.
4. Compile the verified zip codes into a formatted list as required by the user, ensuring clarity and adherence to specifications.
Failed Subtask: "Use mapping tools or geo-mapping APIs (e.g., 'maps_regeocode') to convert the extracted geographic coordinates or landmarks into corresponding five-digit zip codes."
Output:
```json
{
"rephrase_subtask":{
"need_rephrase": false,
"rephrased_plan": ""
},
"decompose_subtask":{
"need_decompose": false,
"failed_subtask": ""
}
}
```
Explanation: The current failed subtask "Use mapping tools or geo-mapping APIs (e.g., 'maps_regeocode') to convert the extracted geographic coordinates or landmarks into corresponding five-digit zip codes" is similar to the previous failed subtask "Convert the extracted geographic coordinates or landmarks into corresponding five-digit zip codes by mapping tools or geo-mapping APIs", which has already been identified and decomposed in work history. Therefore, we don't need to make decomposition repeatedly.
### Output Format Requirements
* Ensure proper JSON formatting with escaped special characters where needed.
* Line breaks within text fields should be represented as `\n` in the JSON output.
* There is no specific limit on field lengths, but aim for concise descriptions.
* All field values must be strings.
* For each JSON document, only include the following fields:

13
deep_research/agent_deep_research/built_in_prompt/prompt_tool_usage_rules.md Normal file
View File

@@ -0,0 +1,13 @@
### Tool usage rules
1. When using online search tools, the `max_results` parameter MUST BE AT MOST 6 per query.
2. When using online search tools, keep the `query` short and keyword-based (2-6 words ideal). And the number should increase as the research depth increases, which means the deeper the research, the more detailed the query should be.
3. The directory/file system that you can operate is the following path: {tmp_file_storage_dir}. DO NOT try to save/read/modify file in other directories.
4. Try to use the local resource before going to online search. If there is file in PDF format, first convert it to markdown or text with tools, then read it as text.
5. You can basically use web search tools to search and retrieve whatever you want to know, including financial data, location, news, etc.
6. NEVER use `read_text_file` tool to read PDF file directly.
7. DO NOT targeting at generating PDF file unless the user specifies.
8. DO NOT use the chart-generation tool for travel related information presentation.
9. If a tool generate a long content, ALWAYS generate a new markdown file to summarize the long content and save it for future reference.
11. When you use the `write_text_file` tool, you **MUST ALWAYS** remember to provide the both the `path` and `content` parameters. DO NOT try to use `write_text_file` with long content exceeding 1k tokens at once!!!
Finally, before each tool using decision, carefully review the historical tool usage records to avoid the time and API costs caused by repeated execution. Remember that your balance is very low, so ensure absolute efficiency.

37
deep_research/agent_deep_research/built_in_prompt/prompt_worker_additional_sys_prompt.md Normal file
View File

@@ -0,0 +1,37 @@
## Additional Operation Notice
### Checklist Management
1. You will receive a markdown-style checklist (i.e., "Expected Output" checklist) in your input instruction. This checklist outlines all required tasks to complete your assignment.
2. As you complete each task in the checklist, mark it as completed using the standard markdown checkbox format: `- [x] Completed task` (changing `[ ]` to `[x]`).
3. Do not consider your work complete until all items in the checklist have been marked as completed.
### Process Flow
1. Based on your **Working Plan**, working through EACH item in it methodically with the following rules:
- items without `(EXPANSION)` tag are fundamental to complete the current subtask.
- items with `(EXPANSION)` tag are optional, while they can provide some valuable supplementary information that is beneficial for enriching the depth and breadth of your final output. However, it may also bring some distracting information. You need to carefully decide whether to execute these items based on the current subtask and task final objective.
2. Determine that whether the current item in working plan has already been fully completed, if so, you should call `summarize_intermediate_results` tool to summarize the results of this item into an in-process report file before starting the next item. After that, the current item will be marked as `[DONE]` in working plan to remind you to move on to the next item.
3. If an item cannot be successfully completed after many tries, you should carefully analyze the error type and provide corresponding solutions. The error types and solutions includes:
- Tool corruption (e.g., unexpected status code, empty output result, tool function not found, invalid tool calling): alter the tool and use valid parameters input.
- Insufficient information (e.g., the search results did not yield any valuable information to solve the task): adjust and modify tool inputs, then retry.
- Missing prerequisite (e.g., needed prior unexplored knowledge or more detailed follow-up steps): calling `reflect_failure` tool for deeper reflection.
4. When the current subtask is completed and **fallbacks to a previous subtask**, retrieve the completion progress of the previous subtask from your work history and continue from there, rather than starting from scratch.
### Important Constraints
1. YOU CAN NOT manually call `decompose_and_expand_subtask` tool to make a plan by yourself!
2. ALWAYS FOLLOW THE WORKING PLAN SEQUENCE STEP BY STEP!!
3. For each step, You MUST provide a reason or analysis to **review what was done in the previous step** and **explain why to call a function / use a tool in this step**.
4. After each action, YOU MUST seriously confirm that the current item in plan is done before starting the next item refer to the following rules:
- Carefully analyze whether the information obtained from tool is sufficient to fill the knowledge gap corresponding to the current item.
- Pay more attention to details. Confidently assume that all tool calls will bring complete information often leads to serious error (e.g., mistaking the rental website name for the apartment name when renting).
If the current item in plan is really done, calling `summarize_intermediate_results` to generate an in-process report, then moving on to the next item.
5. Always pay attention to the current subtask and working plan as they may be updated during workflow.
6. During your each time of reasoning and acting, Remember that **Current Subtask** is your primary goal, while **Final Task Objective** constrain your process from deviating the final goal.
### Completion and Output
You should use the {finish_function_name} tool to return your research results when:
- Research Depth > 1 and all items of the current working plan are marked as `[DONE]`.
- Research Depth = 1 and all checklist items are completed.
### Progress Tracking
1. Regularly review the checklist to confirm your progress.
2. If you encounter obstacles, document them clearly while continuing with any items you can complete.

140
deep_research/agent_deep_research/built_in_prompt/promptmodule.py Normal file
View File

@@ -0,0 +1,140 @@
# -*- coding: utf-8 -*-
"""The output format of deep research agent"""
from pydantic import BaseModel, Field
class SubtasksDecomposition(BaseModel):
"""
Model for structured subtask decomposition output in deep research.
"""
knowledge_gaps: str = Field(
description=(
"A markdown checklist of essential knowledge gaps "
"and optional perspective-expansion gaps (flagged "
"with (EXPANSION)), each on its own line. "
"E.g. '- [ ] Detailed analysis of JD.com's "
"...\\n- [ ] (EXPANSION) X...'."
),
)
working_plan: str = Field(
description=(
"A logically ordered step-by-step working "
"meta_planner_agent (3-5 steps), each step starting with "
"its number (1., 2., etc), including both "
"core and expansion steps. Expanded steps "
"should be clearly marked with (EXPANSION) "
"and provide contextual or analytical depth.."
),
)
class WebExtraction(BaseModel):
"""
Model for structured follow-up web extraction output in deep research.
"""
reasoning: str = Field(
description="The reasoning for your decision, including a "
"summary of evidence and logic for whether more "
"information is needed.",
)
need_more_information: bool = Field(
description="Whether more information is needed.",
)
title: str = Field(
description="Title of the identified search result snippet "
"that requires further extraction, or an empty "
"string if not applicable.",
)
url: str = Field(
description="Direct URL to the original search result "
"requiring further extraction, or an empty "
"string if not applicable.",
)
subtask: str = Field(
description="Actionable description of the follow-up task "
"to obtain needed information, or an empty string "
"if not applicable.",
)
class FollowupJudge(BaseModel):
"""
Model for structured follow-up decompose judging output in deep research.
"""
reasoning: str = Field(
description="The reasoning for your decision, including a "
"summary of evidence and logic for whether "
"more information is needed.",
)
is_sufficient: bool = Field(
description="whether the information content is adequate.",
)
class ReflectFailure(BaseModel):
"""
Model for structured failure reflection output in deep research.
"""
rephrase_subtask: dict = Field(
description=(
"Information about whether the problematic "
"subtask needs to be rephrased due "
"to a design flaw or misunderstanding. "
"If rephrasing is needed, provide the "
"modified working meta_planner_agent with only the "
"inappropriate subtask replaced by its "
"improved version."
),
json_schema_extra={
"additionalProperties": {
"type": "object",
"properties": {
"need_rephrase": {
"type": "boolean",
"description": "Set to 'true' if the failed subtask "
"needs to be rephrased due to a design "
"flaw or misunderstanding; otherwise, 'false'.",
},
"rephrased_plan": {
"type": "string",
"description": "The modified working meta_planner_agent "
"with only the inappropriate "
"subtask replaced by its improved version. If no "
"rephrasing is needed, provide an empty string.",
},
},
},
},
)
decompose_subtask: dict = Field(
description=(
"Information about whether the problematic subtask "
"should be further decomposed. If decomposition "
"is required, provide the failed subtask "
"and the reason for its decomposition."
),
json_schema_extra={
"additionalProperties": {
"type": "object",
"properties": {
"need_decompose": {
"type": "boolean",
"description": "Set to 'true' if "
"the failed subtask should "
"be further decomposed; otherwise, 'false'.",
},
"rephrased_plan": {
"type": "string",
"description": "Information about whether "
"the failed subtask requires "
"decomposition, and the "
"failed subtask itself if needed.",
},
},
},
},
)

1119
deep_research/agent_deep_research/deep_research_agent.py Normal file
View File

File diff suppressed because it is too large Load Diff

82
deep_research/agent_deep_research/main.py Normal file
View File

@@ -0,0 +1,82 @@
# -*- coding: utf-8 -*-
"""The main entry point of the Deep Research agent example."""
import asyncio
import os
from agentscope import logger
from agentscope.formatter import DashScopeChatFormatter
from agentscope.mcp import StdIOStatefulClient
from agentscope.memory import InMemoryMemory
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from .deep_research_agent import DeepResearchAgent
async def main(user_query: str) -> None:
"""The main entry point for the Deep Research agent example."""
logger.setLevel("DEBUG")
tavily_search_client = StdIOStatefulClient(
name="tavily_mcp",
command="npx",
args=["-y", "tavily-mcp@latest"],
env={"TAVILY_API_KEY": os.getenv("TAVILY_API_KEY", "")},
)
default_working_dir = os.path.join(
os.path.dirname(__file__),
"deepresearch_agent_demo_env",
)
agent_working_dir = os.getenv(
"AGENT_OPERATION_DIR",
default_working_dir,
)
os.makedirs(agent_working_dir, exist_ok=True)
try:
await tavily_search_client.connect()
agent = DeepResearchAgent(
name="Friday",
sys_prompt="You are a helpful assistant named Friday.",
model=DashScopeChatModel(
api_key=os.environ.get("DASHSCOPE_API_KEY"),
model_name="qwen-max",
enable_thinking=False,
stream=True,
),
formatter=DashScopeChatFormatter(),
memory=InMemoryMemory(),
search_mcp_client=tavily_search_client,
tmp_file_storage_dir=agent_working_dir,
)
user_name = "Bob"
msg = Msg(
user_name,
content=user_query,
role="user",
)
result = await agent(msg)
logger.info(result)
except Exception as err:
logger.exception(err)
finally:
await tavily_search_client.close()
if __name__ == "__main__":
query = (
"If Eliud Kipchoge could maintain his record-making "
"marathon pace indefinitely, how many thousand hours "
"would it take him to run the distance between the "
"Earth and the Moon its closest approach? Please use "
"the minimum perigee value on the Wikipedia page for "
"the Moon when carrying out your calculation. Round "
"your result to the nearest 1000 hours and do not use "
"any comma separators if necessary."
)
try:
asyncio.run(main(query))
except Exception as e:
logger.exception(e)

1
deep_research/agent_deep_research/requirements.txt Normal file
View File

@@ -0,0 +1 @@
agentscope[full]>=1.0.5

325
deep_research/agent_deep_research/utils.py Normal file
View File

@@ -0,0 +1,325 @@
# -*- coding: utf-8 -*-
"""The utilities for deep research agent"""
import json
import os
import re
from typing import Any, Sequence, Type, Union
from agentscope.tool import Toolkit, ToolResponse
from pydantic import BaseModel
TOOL_RESULTS_MAX_WORDS = 5000
def get_prompt_from_file(
file_path: str,
return_json: bool,
) -> Union[str, dict]:
"""Get prompt from file"""
with open(os.path.join(file_path), "r", encoding="utf-8") as f:
if return_json:
prompt = json.load(f)
else:
prompt = f.read()
return prompt
def truncate_by_words(sentence: str) -> str:
"""Truncate too long sentences by words number"""
words = re.findall(
r"\w+|[^\w\s]",
sentence,
re.UNICODE,
)
word_count = 0
result = []
for word in words:
if re.match(r"\w+", word):
word_count += 1
if word_count > TOOL_RESULTS_MAX_WORDS:
break
result.append(word)
truncated_sentence = ""
for i, word in enumerate(result):
if i == 0:
truncated_sentence += word
elif re.match(r"\w+", word):
truncated_sentence += " " + word
else:
truncated_sentence += word
return truncated_sentence
def truncate_search_result(
res: list,
search_func: str = "tavily-search",
extract_function: str = "tavily-extract",
) -> list:
"""Truncate search result in deep research agent"""
if search_func != "tavily-search" or extract_function != "tavily-extract":
raise NotImplementedError(
"Specific implementation of truncation should be provided.",
)
for i, val in enumerate(res):
res[i]["text"] = truncate_by_words(val["text"])
return res
def generate_structure_output(**kwargs: Any) -> ToolResponse:
"""Generate a structured output tool response.
This function is designed to be used as a tool function for generating
structured outputs. It takes arbitrary keyword arguments and wraps them
in a ToolResponse with metadata.
Args:
**kwargs: Arbitrary keyword arguments that should match the format
of the expected structured output specification.
Returns:
ToolResponse: A tool response object with empty content and the
provided kwargs as metadata.
Note:
The input parameters should be in the same format as the specification
and include as much detail as requested by the calling context.
"""
return ToolResponse(content=[], metadata=kwargs)
def get_dynamic_tool_call_json(data_model_type: Type[BaseModel]) -> list[dict]:
"""Generate JSON schema for dynamic tool calling with a given data model.
Creates a temporary toolkit, registers the structure output function,
and configures it with the specified data model to generate appropriate
JSON schemas for tool calling.
Args:
data_model_type: A Pydantic BaseModel class that defines the expected
structure of the tool output.
Returns:
A list of dictionary that contains the JSON schemas for
the configured tool, suitable for use in API calls that
support structured outputs.
Example:
class MyModel(BaseModel):
name: str
value: int
schema = get_dynamic_tool_call_json(MyModel)
"""
tmp_toolkit = Toolkit()
tmp_toolkit.register_tool_function(generate_structure_output)
tmp_toolkit.set_extended_model(
"generate_structure_output",
data_model_type,
)
return tmp_toolkit.get_json_schemas()
def get_structure_output(blocks: list | Sequence) -> dict:
"""Extract structured output from a sequence of blocks.
Processes a list or sequence of blocks to extract tool use outputs
and combine them into a single dictionary. This is typically used
to parse responses from language models that include tool calls.
Args:
blocks: A list or sequence of blocks that may contain tool use
information. Each block should be a dictionary with 'type'
and 'input' keys for tool use blocks.
Returns:
A dictionary containing the combined input data from all tool
use blocks found in the input sequence.
Example:
blocks = [
{"type": "tool_use", "input": {"name": "test"}},
{"type": "text", "content": "Some text"},
{"type": "tool_use", "input": {"value": 42}}
]
result = PromptBase.get_structure_output(blocks)
# result: {"name": "test", "value": 42}
"""
dict_output = {}
for block in blocks:
if isinstance(block, dict) and block.get("type") == "tool_use":
dict_output.update(block.get("input", {}))
return dict_output
def load_prompt_dict() -> dict:
"""Load prompt into dict"""
prompt_dict = {}
cur_dir = os.path.dirname(os.path.abspath(__file__))
prompt_dict["add_note"] = get_prompt_from_file(
file_path=os.path.join(
cur_dir,
"built_in_prompt/prompt_worker_additional_sys_prompt.md",
),
return_json=False,
)
prompt_dict["tool_use_rule"] = get_prompt_from_file(
file_path=os.path.join(
cur_dir,
"built_in_prompt/prompt_tool_usage_rules.md",
),
return_json=False,
)
prompt_dict["decompose_sys_prompt"] = get_prompt_from_file(
file_path=os.path.join(
cur_dir,
"built_in_prompt/prompt_decompose_subtask.md",
),
return_json=False,
)
prompt_dict["expansion_sys_prompt"] = get_prompt_from_file(
file_path=os.path.join(
cur_dir,
"built_in_prompt/prompt_deeper_expansion.md",
),
return_json=False,
)
prompt_dict["summarize_sys_prompt"] = get_prompt_from_file(
file_path=os.path.join(
cur_dir,
"built_in_prompt/prompt_inprocess_report.md",
),
return_json=False,
)
prompt_dict["reporting_sys_prompt"] = get_prompt_from_file(
file_path=os.path.join(
cur_dir,
"built_in_prompt/prompt_deepresearch_summary_report.md",
),
return_json=False,
)
prompt_dict["reflect_sys_prompt"] = get_prompt_from_file(
file_path=os.path.join(
cur_dir,
"built_in_prompt/prompt_reflect_failure.md",
),
return_json=False,
)
prompt_dict["reasoning_prompt"] = (
"## Current Subtask:\n{objective}\n"
"## Working Plan:\n{meta_planner_agent}\n"
"{knowledge_gap}\n"
"## Research Depth:\n{depth}"
)
prompt_dict["previous_plan_inst"] = (
"## Previous Plan:\n{previous_plan}\n"
"## Current Subtask:\n{objective}\n"
)
prompt_dict["max_depth_hint"] = (
"The search depth has reached the maximum limit. So the "
"current subtask can not be further decomposed and "
"expanded anymore. I need to find another way to get it "
"done no matter what."
)
prompt_dict["expansion_inst"] = (
"Review the web search results and identify whether "
"there is any information that can potentially help address "
"checklist items or fulfill knowledge gaps of the task, "
"but whose content is limited or only briefly mentioned.\n"
"**Task Description:**\n{objective}\n"
"**Checklist:**\n{checklist}\n"
"**Knowledge Gaps:**\n{knowledge_gaps}\n"
"**Search Results:**\n{search_results}\n"
"**Output:**\n"
)
prompt_dict["follow_up_judge_sys_prompt"] = (
"To provide sufficient external information for the user's "
"query, you have conducted a web search to obtain additional "
"data. However, you found that some of the information, while "
"important, was insufficient. Consequently, you extracted the "
"entire content from one of the URLs to gather more "
"comprehensive information. Now, you must rigorously and "
"carefully assess whether, after both the web search and "
"extraction process, the information content is adequate to "
"address the given task. Be aware that any arbitrary decisions "
"may result in unnecessary and unacceptable time costs.\n"
)
prompt_dict[
"retry_hint"
] = "Something went wrong when {state}. I need to retry."
prompt_dict["need_deeper_hint"] = (
"The information is insufficient and I need to make deeper "
"research to fill the knowledge gap."
)
prompt_dict[
"sufficient_hint"
] = "The information after web search and extraction is sufficient enough!"
prompt_dict["no_result_hint"] = (
"I mistakenly called the `summarize_intermediate_results` tool as "
"there exists no milestone result to summarize now."
)
prompt_dict["summarize_hint"] = (
"Based on your work history above, examine which step in the "
"following working meta_planner_agent has been completed. Mark the completed "
"step with [DONE] at the end of its line (e.g., k. step k [DONE]) "
"and leave the uncompleted steps unchanged. You MUST return only "
"the updated meta_planner_agent, preserving exactly the same format as the "
"original meta_planner_agent. Do not include any explanations, reasoning, "
"or section headers such as '## Working Plan:', just output the"
"updated meta_planner_agent itself."
"\n\n## Working Plan:\n{meta_planner_agent}"
)
prompt_dict["summarize_inst"] = (
"**Task Description:**\n{objective}\n"
"**Checklist:**\n{knowledge_gaps}\n"
"**Knowledge Gaps:**\n{working_plan}\n"
"**Search Results:**\n{tool_result}"
)
prompt_dict["update_report_hint"] = (
"Due to the overwhelming quantity of information, I have replaced the "
"original bulk search results from the research phase with the "
"following report that consolidates and summarizes the essential "
"findings:\n {intermediate_report}\n\n"
"Such report has been saved to the {report_path}. "
"I will now **proceed to the next item** in the working meta_planner_agent."
)
prompt_dict["save_report_hint"] = (
"The milestone results of the current item in working meta_planner_agent "
"are summarized into the following report:\n{intermediate_report}"
)
prompt_dict["reflect_instruction"] = (
"## Work History:\n{conversation_history}\n"
"## Working Plan:\n{meta_planner_agent}\n"
)
prompt_dict["subtask_complete_hint"] = (
"Subtask ‘{cur_obj}’ is completed. Now the current subtask "
"fallbacks to '{next_obj}'"
)
return prompt_dict