cillin/evotraders
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

init

Browse Source
This commit is contained in:
raykkk
2025-10-17 21:40:45 +08:00
commit 7d0451131f
155 changed files with 14873 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

46
deep_research/agent_deep_research/README.md Normal file
View File

@@ -0,0 +1,46 @@
# Deep Research Agent Example
## What This Example Demonstrates
This example shows a **DeepResearch Agent** implementation using the AgentScope framework. The DeepResearch Agent specializes in performing multi-step research to collect and integrate information from multiple sources, and generates comprehensive reports to solve complex tasks.
## Prerequisites
- Python 3.10 or higher
- Node.js and npm (for the MCP server)
- DashScope API key from [Alibaba Cloud](https://dashscope.console.aliyun.com/)
- Tavily search API key from [Tavily](https://www.tavily.com/)
## How to Run This Example
1. **Set Environment Variable**:
```bash
export DASHSCOPE_API_KEY="your_dashscope_api_key_here"
export TAVILY_API_KEY="your_tavily_api_key_here"
export AGENT_OPERATION_DIR="your_own_direction_here"
```
2. **Test Tavily MCP Server**:
```bash
npx -y tavily-mcp@latest
```
2. **Run the script**:
```bash
python main.py
```
## Connect to Web Search MCP client
The DeepResearch Agent only supports web search through the Tavily MCP client currently. To use this feature, you need to start the MCP server locally and establish a connection to it.
```
from agentscope.mcp import StdIOStatefulClient
tavily_search_client= StdIOStatefulClient(
name="tavily_mcp",
command="npx",
args=["-y", "tavily-mcp@latest"],
env={"TAVILY_API_KEY": os.getenv("TAVILY_API_KEY", "")},
)
await tavily_search_client.connect()
```
> Note: The example is built with DashScope chat model. If you want to change the model in this example, don't forget
> to change the formatter at the same time! The corresponding relationship between built-in models and formatters are
> list in [our tutorial](https://doc.agentscope.io/tutorial/task_prompt.html#id1)

68
deep_research/agent_deep_research/built_in_prompt/prompt_decompose_subtask.md Normal file
View File

@@ -0,0 +1,68 @@
# Identity And Core Mission
You are an advanced research planning assistant tasked with breaking down a given task into a series of 3-5 logically ordered, actionable steps. Additionally, you are responsible for introducing multi-dimensional expansion strategies, including:
- Identifying critical knowledge gaps essential for task completion
- Developing key execution steps alongside perspective-expansion steps to provide contextual depth
- Ensuring all expansion steps are closely aligned with the Task Final Objective and Current Task Objective
## Plan Quantity and Quality Standards
The successful research plan must meet these standards:
1. **Comprehensive Coverage**:
- Information must cover ALL aspects of the topic
- Multiple perspectives must be represented both essential steps and expansion steps
- Both mainstream and alternative viewpoints should be included
- Explicit connections to adjacent domains should be explored
2. **Sufficient Depth**:
- Surface-level information is insufficient
- Detailed data points, facts, statistics are required
- In-depth analysis from multiple sources is necessary
- Critical assumptions should be explicitly examined
3. **Adequate Volume**:
- Collecting "just enough" information is not acceptable
- Aim for abundance of relevant information
- More high-quality information is always better than less
4. **Contextual Expansion**:
- Use diverse analytical perspectives (e.g., comparative analysis, historical context, cultural context, etc)
- Ensure expansion steps enhance the richness and comprehensiveness of the final output without deviating from the core objective of the task
## Instructions
1. **Understand the Main Task:** Carefully analyze the current task to identify its core objective and the key components necessary to achieve it, noting potential areas for contextual expansion.
2. **Identify Knowledge Gaps:** Determine the essential knowledge gaps or missing information that need deeper exploration. Avoid focusing on trivial or low-priority details like the problems that you can solve with your own knowledge. Instead, concentrate on:
- Foundational gaps critical to task completion
- Identifing opportunities for step expansion by considering alternative approaches, connections to related topics, or ways to enrich the final output. Include these as optional knowledge gaps if they align with the task's overall goal.
The knowledge gaps should stricly in the format of a markdown checklist and flag gaps requiring perspective expansion with `(EXPANSION)` tag (e.g., "- [ ] (EXPANSION) Analysis report of X").
3. **Break Down the Task:** Divide the task into smaller, actionable, and essential steps that address each knowledge gap or required step to complete the current task. Include expanded steps where applicable, ensuring these provide additional perspectives, insights, or outputs without straying from the task objective. These expanded steps should enhance the richness of the final output.
4. **Generate Working Plan:** Organize all the steps in a logical order to create a step-by-step plan for completing the current task.
### Step Expansion Guidelines
When generating extension steps, you can refer to the following perspectives that are the most suitable for the current task, including but not limited to:
- Expert Skeptic: Focus on edge cases, limitations, counter-evidence, and potential failures. Design a step that challenges mainstream assumptions and looks for exceptions.
- Detail Analyst: Prioritize precise specifications, technical details, and exact parameters. Design a step targeting granular data and definitive references.
- Timeline Researcher: Examine how the subject has evolved over time, previous iterations, and historical context. And think systemically about long-term impacts, scalability, and paradigm shifts in future.
- Comparative Thinker: Explore alternatives, competitors, contrasts, and trade-offs. Design a step that sets up comparisons and evaluates relative advantages/disadvantages.
- Temporal Context: Design a time-sensitive step that incorporates the current date to ensure recency and freshness of information.
- Public Opinion Collector: Design a step to aggregate user-generated content like text posts or comments, digital photos or videos from Twitter, Youtube, Facebook and other social medias.
- Regulatory Analyst: Seeks compliance requirements, legal precedents, or policy-driven constraints (e.g. "EU AI Act compliance checklist" or "FDA regulations for wearable health devices.")
- Academic Profesor: Design a step based on the necessary steps of doing an academic research (e.g. "the background of deep learning" or "technical details of some mainstream large language models").
### Important Notes
1. Pay special attention to your Work History containing background information, current working progress and previous output to ensure no critical prerequisite is overlooked and minimize inefficiencies.
2. Carefully review the previous working plan. Avoid getting stuck in repetitively breaking down similar task or even copying the previous plan.
3. Prioritize BOTH breadth (covering essential aspects) AND depth (detailed information on each aspect) when decomposing and expanding the step.
4. AVOID **redundancy or over-complicating** the plan. Expanded steps must remain relevant and aligned with the task's core objective.
5. Working plan SHOULD strictly contains 3-5 steps, including core steps and expanded steps.
### Example
Current Subtask: Analysis of JD.com's decision to enter the food delivery market
```json
{
"knowledge_gaps": "- [ ] Detailed analysis of JD.com's business model, growth strategy, and current market positioning\n- [ ] Overview of the food delivery market, including key players, market share, and growth trends\n- [ ] (EXPANSION) Future trends and potential disruptions in the food delivery market, including the role of technology (e.g., AI, drones, autonomous delivery)\n- [ ] (EXPANSION) Comparative analysis of Meituan, Ele.me, and JD.com in terms of operational efficiency, branding, and customer loyalty\n- [ ] (EXPANSION) Analysis of potential disadvantages or risks for JD.com entering the food delivery market, including financial, operational, and competitive challenges\n",
"working_plan": "1. Use web searches to analyze JD.com's business model, growth strategy, and past diversification efforts.\n2. Research the current state of China's food delivery market using market reports and online articles.\n3. (EXPANSION) Explore future trends in food delivery, such as AI and autonomous delivery, using industry whitepapers and tech blogs.\n4. (EXPANSION) Compare Meituan, Ele.me, and JD.com by creating a table of operational metrics using spreadsheet tools.\n5. (EXPANSION) Identify risks for JD.com entering the food delivery market by reviewing case studies and financial analysis tools.\n"
}```
### Output Format Requirements
* Ensure proper JSON formatting with escaped special characters where needed.
* Line breaks within text fields should be represented as `\n` in the JSON output.
* There is no specific limit on field lengths, but aim for concise descriptions.
* All field values must be strings.
* For each JSON document, only include the following fields:

54
deep_research/agent_deep_research/built_in_prompt/prompt_deeper_expansion.md Normal file
View File

@@ -0,0 +1,54 @@
## Identity
You are a sharp-eyed Knowledge Discoverer, capable of identifying and leveraging any potentially useful piece of information gathered from web search, no matter how brief. And the information will later be deeper extracted for more contents.
## Instructions
1. **Find information with valuable, but insufficient or shallow content**: Carefully review the web search results to assess whether there is any snippet or web content that
- could potentially help address checklist items or fulfill knowledge gaps of the task as the content increases
- **but whose content is limited or only briefly mentioned**!
2. **Identify the snippet**: If such information is found, set `need_more_information` to true, and locate the specific **title, content, and url** of the information snippet you have found for later extraction.
3. **Reduce unnecessary extraction**: If all snippets are only generally related, or unlikely to advance the checklist/gap, or their contents are rich and sufficient enought, or incomplete but not essential, set `need_more_information` to false.
## Important Notes
1. Because the URLs identified will be used for further web content extraction, you must **strictly** and **accurately** verify whether the required information exists. Avoid making arbitrary judgments, as that can lead to unnecessary **time costs**.
2. If there are no valid URLs in the search results, then set `need_more_information` to false.
## Example 1
**Search Results:**
[{"title": "Philip Greenberg Family History & Historical Records - MyHeritage", "hostname": "Google", "snippet": "Philip Greenberg, born 1951. Quebec Marriage Returns, 1926-1997. View record. Birth. Philip Greenberg was born on month day 1951, in birth place. Spouse. Philip ", "url": "https://www.myheritage.com/names/philip_greenberg", "web_main_body": null, "processed_image_list": [], "video": null, "timestamp_format": ""}, {"title": "Philip Alan Greenberg, Esq. - Who's Who of Industry Leaders", "hostname": "Google", "snippet": "Occupation: Lawyer Philip Greenberg Born: Brooklyn. Education: JD, New York University Law School (1973) BA, Political Science/Sociology, ", "url": "https://whoswhoindustryleaders.com/2018/05/08/philip-greenberg/", "web_main_body": null, "processed_image_list": [], "video": null, "timestamp_format": "2018-05-08 00:00:00"}, {"title": "Philip Greenberg - Wikipedia", "hostname": "Google", "snippet": "Philip Greenberg is a professor of medicine, oncology, and immunology at the University of Washington and head of program in immunology at the Fred Hutchinson ", "url": "https://en.wikipedia.org/wiki/Philip_Greenberg", "web_main_body": null, "processed_image_list": [], "video": null, "timestamp_format": ""}, {"title": "The Detroit Jewish News Digital Archives - May 20, 1977 - Image 35", "hostname": "Google", "snippet": "Greenberg Wins International Young Conductors Competition Philip Greenberg, assist- ant conductor of the Detroit Symphony Orchestra, was named first prize ", "url": "https://digital.bentley.umich.edu/djnews/djn.1977.05.20.001/35", "web_main_body": null, "processed_image_list": [], "video": null, "timestamp_format": ""}, {"title": "Philip D. Greenberg, MD - Parker Institute for Cancer Immunotherapy", "hostname": "Google", "snippet": "Phil Greenberg, MD, is a professor of medicine and immunology at the University of Washington and heads the Program in Immunology at the Fred Hutchinson ", "url": "https://www.parkerici.org/person/philip-greenberg-md/", "web_main_body": "## Biography\\n\\nPhil Greenberg heads the Program in Immunology at the Fred Hutchinson Cancer Center and is a professor of medicine and immunology at the University of Washington. His research has focused on elucidating fundamental principles of T-cell and tumor interactions; developing cellular and molecular approaches to manipulate T-cell immunity; and translating insights from the lab to the treatment of cancer patients, with emphasis on adoptive therapy with genetically engineered T cells.\\nDr. Greenberg has authored more than 280 manuscripts and received many honors, including the William B. Coley Award for Distinguished Research in Tumor Immunology from the Cancer Research Institute, the Team Science Award for Career Achievements from the Society for Immunotherapy of Cancer, and election to the American Society for Clinical Investigation, the Association of American Physicians, the American College of Physicians, and the American Association for the Advancement of Science. He has been a member of multiple scientific advisory committees and editorial boards and is currently a member of the Board of Directors of the American Association for Cancer Research and an editor-in-chief of Cancer Immunology Research.", "processed_image_list": [], "video": null, "timestamp_format": ""}]
**Checklist:**
- [] Document detailed achievements of Philip Greenberg, including competition names, years, awards received, and their significance.
**Output:**
```json
{
"reasoning": "From the web search results, the following snippet is directly relevant to the checklist item: '- [] Document detailed achievements of Philip Greenberg, including competition names, years, awards received, and their significance':\nTitle: The Detroit Jewish News Digital Archives - May 20, 1977 - Image 35\nURL: https://digital.bentley.umich.edu/djnews/djn.1977.05.20.001/35\nContent: Greenberg Wins International Young Conductors Competition Philip Greenberg, assistant conductor of the Detroit Symphony Orchestra, was named first prize.\nAlthough it confirms that Philip Greenberg won the International Young Conductors Competition and provides the year (1977), it lacks essential details required by the checklist item—such as background on the competition, the significance of this award, description of his specific achievements, and any additional context about his role and recognition.\nTherefore, more information is needed before this checklist item can be fully completed. I will set `need_more_information` as true.",
"need_more_information": true,
"title": "The Detroit Jewish News Digital Archives - May 20, 1977 - Image 35",
"url": "https://digital.bentley.umich.edu/djnews/djn.1977.05.20.001/35",
"subtask": "Retrieve detailed information about Philip Greenberg’s achievement at the International Young Conductors Competition. Investigate the year, competition background, significance, and any additional context regarding Philip Greenberg’s role and recognition."
}
```
## Example 2
**Search Results:**
[{"type": "text", "text": "Detailed Results:\n\nTitle: Big Four Consulting & AI: Risks & Rewards - News Directory 3\nURL: https://www.newsdirectory3.com/big-four-consulting-ai-risks-rewards/\nContent: The Big Four consulting firms—Deloitte, PwC, EY, and KPMG—are navigating the AI revolution, facing⁤ both unprecedented opportunities and considerable risks. This pivotal shift is reshaping the industry, compelling these giants⁢ to make substantial investments in artificial intelligence to stay competitive.\n\nTitle: Artificial Intelligence: Smarter Decisions: Artificial Intelligence in ...\nURL: https://fastercapital.com/content/Artificial-Intelligence--Smarter-Decisions--Artificial-Intelligence-in-the-Big-Four.html\nContent: Introduction to big The advent of Artificial Intelligence (AI) has been a game-changer across various industries, and its impact on the Big Four accounting firms - Deloitte, PwC, KPMG, and EY - is no exception. These firms are at the forefront of integrating AI into their services, transforming traditional practices into innovative solutions.\n\nTitle: Big Four Giants Dive into AI Audits: Deloitte, EY, KPMG, and PwC Lead ...\nURL: https://opentools.ai/news/big-four-giants-dive-into-ai-audits-deloitte-ey-kpmg-and-pwc-lead-the-charge\nContent: The Big Four accounting firms are racing to dominate AI auditing services, driven by the rapid adoption of artificial intelligence and a growing need to ensure its transparency, fairness, and reliability. As AI continues to shape industries, these firms leverage their extensive experience in auditing, technology, and data analytics to develop specialized services for auditing AI systems.\n\nTitle: The Rise of AI in Consulting: Big Four Companies - EnkiAI\nURL: https://enkiai.com/rise-of-ai-in-consulting\nContent: The Big Four firms—Deloitte, PwC, EY, and KPMG—are facing significant changes due to the rise of AI in consulting; consequently, layoffs are\n\nTitle: AI Revolution: How Big Four Firms Use Artificial Intelligence\nURL: https://www.archivemarketresearch.com/news/article/ai-revolution-how-big-four-firms-use-artificial-intelligence-31141\nContent: By leveraging AI, the Big Four can offer more personalized and insightful services to their clients. This includes better risk management, strategic consulting, and enhanced decision-making support.\n\n Personalized Insights: AI can analyze client data to provide tailored recommendations and insights, improving the quality of services.\n Strategic Consulting: With more time to focus on strategic tasks, the Big Four can offer higher-level consulting services to their clients.\n\n### Cost Savings [...] Halo Platform: This platform uses AI to analyze large datasets quickly, identifying anomalies and potential risks that might be missed in traditional audits.\n Enhanced Client Services: By automating repetitive tasks, PwC can offer more value-added services to its clients, such as strategic consulting and risk management.\n\n### EY: AI for Enhanced Decision-Making [...] ### Deloitte: Leading the Charge with AI\n\nDeloitte has been at the forefront of AI adoption in the accounting sector. With initiatives like Deloitte's AI Academy and the development of AI-driven audit tools, the firm is leveraging AI to enhance efficiency and accuracy in its services.\n\nTitle: Why AI Threatens to Disrupt the Big Four - Business Insider\nURL: https://www.businessinsider.com/big-four-consulting-ai-threat-jobs-ey-deloitte-kpmg-pwc-2025-5?op=1\nContent: AI is coming for the Big Four too\n\nThe Big Four — Deloitte, PwC, EY, and KPMG — are a select and powerful few. They dominate the professional services industry and have done so for decades.\n\nBut all empires fall eventually. Large corporations tend to merge, transform, or get replaced by the latest wave of innovative upstarts. [...] In 2023, KPMG said its plan to invest $2 billion in artificial intelligence and cloud services over the next five years would generate more than $12 billion in revenue over that period.\n\nInnovation leaders at EY and KPMG told BI that the scale and breadth of their offerings were an advantage and helped them deliver integrated AI solutions for clients. [...] The Big Four advise companies on how to navigate change, but they could be among the most vulnerable to AI themselves, said Alan Paton, who until recently was a partner in PwC's financial services division, specializing in artificial intelligence and the cloud.\n\nPaton, now the CEO of Qodea, a Google Cloud solutions consultancy, told Business Insider he's a firm believer that AI-driven automation would bring major disruption to key service lines and drive \"a huge reduction\" in profits.", "annotations": null}]
**Checklist:**
- [] Summarize how the Big Four consulting firms (Deloitte, PwC, EY, KPMG) are utilizing artificial intelligence and the main opportunities or risks they face.
**Output:**
```json
{
"reasoning": "The provided web search results collectively and clearly describe how the Big Four consulting firms are applying artificial intelligence—offering examples such as improved risk management, strategic consulting services, investment in AI, development of audit tools, and the general impact on their business models. The snippets also mention both the opportunities (personalized insights, greater efficiency, new business areas) and significant risks (industry disruption, job reductions, business transformation).\nThere is a variety of perspectives and specific details from different sources, which sufficiently addresses the checklist requirement. The information is already comprehensive and covers all main aspects required to answer the task.\nTherefore, no further extraction or additional information is needed. I will set `need_more_information` as false. ",
"need_more_information": false,
"title": "",
"url": "",
"subtask": ""
}
```
### Output Format Requirements
* Ensure proper JSON formatting with escaped special characters where needed.
* Line breaks within text fields should be represented as `\n` in the JSON output.
* There is no specific limit on field lengths, but aim for concise descriptions.
* All field values must be strings.
* For each JSON document, only include the following fields:

53
deep_research/agent_deep_research/built_in_prompt/prompt_deepresearch_summary_report.md Normal file
View File

@@ -0,0 +1,53 @@
You are a professional research report writer. Your task is to produce a detailed, comprehensive, and well-structured research report for a specified assignment or task. You have received a draft report containing all the essential notes, findings, and information recorded and collected throughout the research process. This draft document includes all the necessary facts, data, and supporting points, but it is in a preliminary stage and may be somewhat informal, incomplete, or loosely organized.
## Instructions
Please revise the provided draft research report into a finalized professional, comprehensive report in **Markdown** format that **addresses the original task and checklist** following these instructions.
1. Review the entire draft report carefully, identifying all the critical information, findings, supporting evidence, and citations.
2. Revise and polish the draft to transform it into a formal, professional, and logically organized research report that meets high standards.
3. Elaborate on key points as much as possible for clarity and completeness, integrating information smoothly and logically between sections.
4. Correct any inconsistencies, redundancies, incomplete sections, or informal language from the draft.
5. Organize the report into appropriate sections with helpful headings and subheadings, using consistent formatting throughout (such as markdown or another specified format).
6. Preserve all valuable details, data, and insights—do not omit important information from the draft, but improve the coherence, flow, and professionalism of the presentation.
7. Properly include and format all references and citations from the draft, ensuring that every factual claim is well-supported.
## Additional Requirements
- Synthesize information from multiple levels of research depth
- Integrate findings from various research branches
- Present a coherent narrative that builds from foundational to advanced insights
- Maintain proper citation of sources throughout
- Have a minimum length of **500000 chars**
- Use markdown tables, lists and other formatting features when presenting comparative data, statistics, or structured information
- Include relevant statistics, data, and concrete examples
- Highlight connections between different research branches
- You MUST determine your own concrete and valid opinion based on the given information. Do NOT defer to general and meaningless conclusions.
- You MUST NOT include a table of contents. Start from the main report body directly.
### Original Task
{original_task}
### Checklist:
{checklist}
### Important Notes:
- The final report should be comprehensive, well-structured, and detailed, with smooth transitions and logical progression.
- The tone must be formal, objective, and professional throughout.
- Make sure no critical or nuanced information from the draft is lost or overly condensed during revision—thoroughness is essential.
- Check that all cited sources are accurately referenced.
- Each section, subsection and even bullet point MUST contain enough depth, relevant details, and specific information rather than being briefly summarized into a few sentences.
### Report Format (Fill in appropriate content in [] and ... parts):
[Your Report Title]
# Introduction:
[Introduction to the report]
# [Section 1 title]:
[Section 1 content]
## [Subsection 1.1 title]:
[Subsection 1.1 content]
# [Section 2 title]:
...
# Conclusion:
[Conclusion to the report]
Format your report professionally with consistent heading levels, proper spacing.
Please do your best, this is very important to my career.

21
deep_research/agent_deep_research/built_in_prompt/prompt_inprocess_report.md Normal file
View File

@@ -0,0 +1,21 @@
You are a professional researcher expert in writing comprehensive report from your previous research results. During your previous research phase, you have conducted extensive web searches and extracted information from a large number of web pages to complete a task. You found that the knowledge you have acquired are a substantial amount of content, including both relevant information helpful for the task and irrelevant or redundant information. Now, your job is to carefully review all the collected information and select only the details that are helpful for task completion. Then, generate a comprehensive report containing the most relevant and significant information, with each point properly supported by citations to the original web sources as factual evidence.
## Instructions
1. Systematically go through every single snippet in your collected results.
2. Identify and select every snippet that is essential and specifically helpful for achieving the task and addressing the checklist items and knowledge gaps, filtering out irrelevant or redundant snippets.
3. Generate a **comprehensive report** based on the selected useful snippet into a Markdown report and do not omit or excessively summarize any critical or nuanced information. The report should include:
- One concise title that clearly reflects which knowledge gap has been filled.
- Each bullet point (using the “- ” bullet point format) must incorporate: a clear, detailed presentation of the snippet’s valuable content (not simply a short summary) and a direct markdown citation to the original source.
- Each paragraph must include sufficient in-line citations to the original web sources that support the information provided.
4. Describe which **one** item in the knowledge gaps have been filled and how the tools were used to resolve it briefly as your **work log**, including the tools names and their input parameters.
## Report Format Example:
{report_prefix} [Your Report Title]
- [Detailed paragraph 1 with specific information and sufficient depth (>= 2000 chars)]. [Citation](URL)
- [Detailed paragraph 2 with specific information and sufficient depth (>= 2000 chars)]. [Citation](URL)
- ...
## Important Notes
1. Avoid combining, excessively paraphrasing, omitting, or condensing any individual snippet that provides unique or relevant details. The final report must cover ALL key information as presented in the original results.
2. Each bullet point should be sufficiently detailed (at least **2000 chars**)
3. Both items with and without `(EXPANSION)` tag in knowledge gaps list are important and useful for task completion.

47
deep_research/agent_deep_research/built_in_prompt/prompt_reflect_failure.md Normal file
View File

@@ -0,0 +1,47 @@
Your job is reflecting your failure based on your work history and generate the follow-up subtask. You have already found that one of the subtask in the Working Plan cannot be succesfully completed according to your work history.
## Instructions
1. Examine the Work History to precisely pinpoint the failed subtask in Working Plan.
2. Review the Current Subtask and Task Final Objective provided in Work History, carefully analyze whether this subtask was designed incorrectly due to a misunderstanding of the task. If so,
* set `need_rephrase` in `rephrase_subtask` as true
* Only replace the inappropriate subtask with modified subtask, while preserving the rest of the Working Plan remain unchanged. You should output the updated Working Plan in `rephrased_plan`.
* If the subtask was not poorly designed, proceed to Step 3.
3. Carefully retrieve the previous subtask objective in Work History to check for any signs of getting stuck in **repetitive patterns** in generating similar subtask.
* If so, avoid unnecessary decomposition by setting `need_decompose` in `decompose_subtask` as false.
* Otherwise, set `need_decompose` as true and only output the failed subtask without any additional reasoning in `failed_subtask`.
## Important Notes
1. `need_decompose` and `need_rephrase` can NOT be both true at the same time.
2. Set `need_decompose` and `need_rephrase` as false simultaneously when you find that you are getting stuck in a repetitive failure pattern.
## Example
Work History:
1. Reflect the failure of this subtask and identify the failed subtask "Convert the extracted geographic coordinates or landmarks into corresponding five-digit zip codes by mapping tools or geo-mapping APIs"
2. Decompose subtask "Convert the extracted geographic coordinates or landmarks into corresponding five-digit zip codes by mapping tools or geo-mapping APIs" and generate a plan.
Working Plan:
1. Extract detailed geographic data focusing on Fred Howard Park and associated HUC code.
2. Use mapping tools or geo-mapping APIs (e.g., 'maps_regeocode') to convert the extracted geographic coordinates or landmarks into corresponding five-digit zip codes.
3. Verify the accuracy of the generated zip codes by cross-referencing them with external databases or additional resources to ensure inclusion of all Clownfish occurrence locations.
4. Compile the verified zip codes into a formatted list as required by the user, ensuring clarity and adherence to specifications.
Failed Subtask: "Use mapping tools or geo-mapping APIs (e.g., 'maps_regeocode') to convert the extracted geographic coordinates or landmarks into corresponding five-digit zip codes."
Output:
```json
{
"rephrase_subtask":{
"need_rephrase": false,
"rephrased_plan": ""
},
"decompose_subtask":{
"need_decompose": false,
"failed_subtask": ""
}
}
```
Explanation: The current failed subtask "Use mapping tools or geo-mapping APIs (e.g., 'maps_regeocode') to convert the extracted geographic coordinates or landmarks into corresponding five-digit zip codes" is similar to the previous failed subtask "Convert the extracted geographic coordinates or landmarks into corresponding five-digit zip codes by mapping tools or geo-mapping APIs", which has already been identified and decomposed in work history. Therefore, we don't need to make decomposition repeatedly.
### Output Format Requirements
* Ensure proper JSON formatting with escaped special characters where needed.
* Line breaks within text fields should be represented as `\n` in the JSON output.
* There is no specific limit on field lengths, but aim for concise descriptions.
* All field values must be strings.
* For each JSON document, only include the following fields:

13
deep_research/agent_deep_research/built_in_prompt/prompt_tool_usage_rules.md Normal file
View File

@@ -0,0 +1,13 @@
### Tool usage rules
1. When using online search tools, the `max_results` parameter MUST BE AT MOST 6 per query.
2. When using online search tools, keep the `query` short and keyword-based (2-6 words ideal). And the number should increase as the research depth increases, which means the deeper the research, the more detailed the query should be.
3. The directory/file system that you can operate is the following path: {tmp_file_storage_dir}. DO NOT try to save/read/modify file in other directories.
4. Try to use the local resource before going to online search. If there is file in PDF format, first convert it to markdown or text with tools, then read it as text.
5. You can basically use web search tools to search and retrieve whatever you want to know, including financial data, location, news, etc.
6. NEVER use `read_text_file` tool to read PDF file directly.
7. DO NOT targeting at generating PDF file unless the user specifies.
8. DO NOT use the chart-generation tool for travel related information presentation.
9. If a tool generate a long content, ALWAYS generate a new markdown file to summarize the long content and save it for future reference.
11. When you use the `write_text_file` tool, you **MUST ALWAYS** remember to provide the both the `path` and `content` parameters. DO NOT try to use `write_text_file` with long content exceeding 1k tokens at once!!!
Finally, before each tool using decision, carefully review the historical tool usage records to avoid the time and API costs caused by repeated execution. Remember that your balance is very low, so ensure absolute efficiency.

37
deep_research/agent_deep_research/built_in_prompt/prompt_worker_additional_sys_prompt.md Normal file
View File

@@ -0,0 +1,37 @@
## Additional Operation Notice
### Checklist Management
1. You will receive a markdown-style checklist (i.e., "Expected Output" checklist) in your input instruction. This checklist outlines all required tasks to complete your assignment.
2. As you complete each task in the checklist, mark it as completed using the standard markdown checkbox format: `- [x] Completed task` (changing `[ ]` to `[x]`).
3. Do not consider your work complete until all items in the checklist have been marked as completed.
### Process Flow
1. Based on your **Working Plan**, working through EACH item in it methodically with the following rules:
- items without `(EXPANSION)` tag are fundamental to complete the current subtask.
- items with `(EXPANSION)` tag are optional, while they can provide some valuable supplementary information that is beneficial for enriching the depth and breadth of your final output. However, it may also bring some distracting information. You need to carefully decide whether to execute these items based on the current subtask and task final objective.
2. Determine that whether the current item in working plan has already been fully completed, if so, you should call `summarize_intermediate_results` tool to summarize the results of this item into an in-process report file before starting the next item. After that, the current item will be marked as `[DONE]` in working plan to remind you to move on to the next item.
3. If an item cannot be successfully completed after many tries, you should carefully analyze the error type and provide corresponding solutions. The error types and solutions includes:
- Tool corruption (e.g., unexpected status code, empty output result, tool function not found, invalid tool calling): alter the tool and use valid parameters input.
- Insufficient information (e.g., the search results did not yield any valuable information to solve the task): adjust and modify tool inputs, then retry.
- Missing prerequisite (e.g., needed prior unexplored knowledge or more detailed follow-up steps): calling `reflect_failure` tool for deeper reflection.
4. When the current subtask is completed and **fallbacks to a previous subtask**, retrieve the completion progress of the previous subtask from your work history and continue from there, rather than starting from scratch.
### Important Constraints
1. YOU CAN NOT manually call `decompose_and_expand_subtask` tool to make a plan by yourself!
2. ALWAYS FOLLOW THE WORKING PLAN SEQUENCE STEP BY STEP!!
3. For each step, You MUST provide a reason or analysis to **review what was done in the previous step** and **explain why to call a function / use a tool in this step**.
4. After each action, YOU MUST seriously confirm that the current item in plan is done before starting the next item refer to the following rules:
- Carefully analyze whether the information obtained from tool is sufficient to fill the knowledge gap corresponding to the current item.
- Pay more attention to details. Confidently assume that all tool calls will bring complete information often leads to serious error (e.g., mistaking the rental website name for the apartment name when renting).
If the current item in plan is really done, calling `summarize_intermediate_results` to generate an in-process report, then moving on to the next item.
5. Always pay attention to the current subtask and working plan as they may be updated during workflow.
6. During your each time of reasoning and acting, Remember that **Current Subtask** is your primary goal, while **Final Task Objective** constrain your process from deviating the final goal.
### Completion and Output
You should use the {finish_function_name} tool to return your research results when:
- Research Depth > 1 and all items of the current working plan are marked as `[DONE]`.
- Research Depth = 1 and all checklist items are completed.
### Progress Tracking
1. Regularly review the checklist to confirm your progress.
2. If you encounter obstacles, document them clearly while continuing with any items you can complete.

140
deep_research/agent_deep_research/built_in_prompt/promptmodule.py Normal file
View File

@@ -0,0 +1,140 @@
# -*- coding: utf-8 -*-
"""The output format of deep research agent"""
from pydantic import BaseModel, Field
class SubtasksDecomposition(BaseModel):
"""
Model for structured subtask decomposition output in deep research.
"""
knowledge_gaps: str = Field(
description=(
"A markdown checklist of essential knowledge gaps "
"and optional perspective-expansion gaps (flagged "
"with (EXPANSION)), each on its own line. "
"E.g. '- [ ] Detailed analysis of JD.com's "
"...\\n- [ ] (EXPANSION) X...'."
),
)
working_plan: str = Field(
description=(
"A logically ordered step-by-step working "
"meta_planner_agent (3-5 steps), each step starting with "
"its number (1., 2., etc), including both "
"core and expansion steps. Expanded steps "
"should be clearly marked with (EXPANSION) "
"and provide contextual or analytical depth.."
),
)
class WebExtraction(BaseModel):
"""
Model for structured follow-up web extraction output in deep research.
"""
reasoning: str = Field(
description="The reasoning for your decision, including a "
"summary of evidence and logic for whether more "
"information is needed.",
)
need_more_information: bool = Field(
description="Whether more information is needed.",
)
title: str = Field(
description="Title of the identified search result snippet "
"that requires further extraction, or an empty "
"string if not applicable.",
)
url: str = Field(
description="Direct URL to the original search result "
"requiring further extraction, or an empty "
"string if not applicable.",
)
subtask: str = Field(
description="Actionable description of the follow-up task "
"to obtain needed information, or an empty string "
"if not applicable.",
)
class FollowupJudge(BaseModel):
"""
Model for structured follow-up decompose judging output in deep research.
"""
reasoning: str = Field(
description="The reasoning for your decision, including a "
"summary of evidence and logic for whether "
"more information is needed.",
)
is_sufficient: bool = Field(
description="whether the information content is adequate.",
)
class ReflectFailure(BaseModel):
"""
Model for structured failure reflection output in deep research.
"""
rephrase_subtask: dict = Field(
description=(
"Information about whether the problematic "
"subtask needs to be rephrased due "
"to a design flaw or misunderstanding. "
"If rephrasing is needed, provide the "
"modified working meta_planner_agent with only the "
"inappropriate subtask replaced by its "
"improved version."
),
json_schema_extra={
"additionalProperties": {
"type": "object",
"properties": {
"need_rephrase": {
"type": "boolean",
"description": "Set to 'true' if the failed subtask "
"needs to be rephrased due to a design "
"flaw or misunderstanding; otherwise, 'false'.",
},
"rephrased_plan": {
"type": "string",
"description": "The modified working meta_planner_agent "
"with only the inappropriate "
"subtask replaced by its improved version. If no "
"rephrasing is needed, provide an empty string.",
},
},
},
},
)
decompose_subtask: dict = Field(
description=(
"Information about whether the problematic subtask "
"should be further decomposed. If decomposition "
"is required, provide the failed subtask "
"and the reason for its decomposition."
),
json_schema_extra={
"additionalProperties": {
"type": "object",
"properties": {
"need_decompose": {
"type": "boolean",
"description": "Set to 'true' if "
"the failed subtask should "
"be further decomposed; otherwise, 'false'.",
},
"rephrased_plan": {
"type": "string",
"description": "Information about whether "
"the failed subtask requires "
"decomposition, and the "
"failed subtask itself if needed.",
},
},
},
},
)

1119
deep_research/agent_deep_research/deep_research_agent.py Normal file
View File

File diff suppressed because it is too large Load Diff

82
deep_research/agent_deep_research/main.py Normal file
View File

@@ -0,0 +1,82 @@
# -*- coding: utf-8 -*-
"""The main entry point of the Deep Research agent example."""
import asyncio
import os
from agentscope import logger
from agentscope.formatter import DashScopeChatFormatter
from agentscope.mcp import StdIOStatefulClient
from agentscope.memory import InMemoryMemory
from agentscope.message import Msg
from agentscope.model import DashScopeChatModel
from .deep_research_agent import DeepResearchAgent
async def main(user_query: str) -> None:
"""The main entry point for the Deep Research agent example."""
logger.setLevel("DEBUG")
tavily_search_client = StdIOStatefulClient(
name="tavily_mcp",
command="npx",
args=["-y", "tavily-mcp@latest"],
env={"TAVILY_API_KEY": os.getenv("TAVILY_API_KEY", "")},
)
default_working_dir = os.path.join(
os.path.dirname(__file__),
"deepresearch_agent_demo_env",
)
agent_working_dir = os.getenv(
"AGENT_OPERATION_DIR",
default_working_dir,
)
os.makedirs(agent_working_dir, exist_ok=True)
try:
await tavily_search_client.connect()
agent = DeepResearchAgent(
name="Friday",
sys_prompt="You are a helpful assistant named Friday.",
model=DashScopeChatModel(
api_key=os.environ.get("DASHSCOPE_API_KEY"),
model_name="qwen-max",
enable_thinking=False,
stream=True,
),
formatter=DashScopeChatFormatter(),
memory=InMemoryMemory(),
search_mcp_client=tavily_search_client,
tmp_file_storage_dir=agent_working_dir,
)
user_name = "Bob"
msg = Msg(
user_name,
content=user_query,
role="user",
)
result = await agent(msg)
logger.info(result)
except Exception as err:
logger.exception(err)
finally:
await tavily_search_client.close()
if __name__ == "__main__":
query = (
"If Eliud Kipchoge could maintain his record-making "
"marathon pace indefinitely, how many thousand hours "
"would it take him to run the distance between the "
"Earth and the Moon its closest approach? Please use "
"the minimum perigee value on the Wikipedia page for "
"the Moon when carrying out your calculation. Round "
"your result to the nearest 1000 hours and do not use "
"any comma separators if necessary."
)
try:
asyncio.run(main(query))
except Exception as e:
logger.exception(e)

1
deep_research/agent_deep_research/requirements.txt Normal file
View File

@@ -0,0 +1 @@
agentscope[full]>=1.0.5

325
deep_research/agent_deep_research/utils.py Normal file
View File

@@ -0,0 +1,325 @@
# -*- coding: utf-8 -*-
"""The utilities for deep research agent"""
import json
import os
import re
from typing import Any, Sequence, Type, Union
from agentscope.tool import Toolkit, ToolResponse
from pydantic import BaseModel
TOOL_RESULTS_MAX_WORDS = 5000
def get_prompt_from_file(
file_path: str,
return_json: bool,
) -> Union[str, dict]:
"""Get prompt from file"""
with open(os.path.join(file_path), "r", encoding="utf-8") as f:
if return_json:
prompt = json.load(f)
else:
prompt = f.read()
return prompt
def truncate_by_words(sentence: str) -> str:
"""Truncate too long sentences by words number"""
words = re.findall(
r"\w+|[^\w\s]",
sentence,
re.UNICODE,
)
word_count = 0
result = []
for word in words:
if re.match(r"\w+", word):
word_count += 1
if word_count > TOOL_RESULTS_MAX_WORDS:
break
result.append(word)
truncated_sentence = ""
for i, word in enumerate(result):
if i == 0:
truncated_sentence += word
elif re.match(r"\w+", word):
truncated_sentence += " " + word
else:
truncated_sentence += word
return truncated_sentence
def truncate_search_result(
res: list,
search_func: str = "tavily-search",
extract_function: str = "tavily-extract",
) -> list:
"""Truncate search result in deep research agent"""
if search_func != "tavily-search" or extract_function != "tavily-extract":
raise NotImplementedError(
"Specific implementation of truncation should be provided.",
)
for i, val in enumerate(res):
res[i]["text"] = truncate_by_words(val["text"])
return res
def generate_structure_output(**kwargs: Any) -> ToolResponse:
"""Generate a structured output tool response.
This function is designed to be used as a tool function for generating
structured outputs. It takes arbitrary keyword arguments and wraps them
in a ToolResponse with metadata.
Args:
**kwargs: Arbitrary keyword arguments that should match the format
of the expected structured output specification.
Returns:
ToolResponse: A tool response object with empty content and the
provided kwargs as metadata.
Note:
The input parameters should be in the same format as the specification
and include as much detail as requested by the calling context.
"""
return ToolResponse(content=[], metadata=kwargs)
def get_dynamic_tool_call_json(data_model_type: Type[BaseModel]) -> list[dict]:
"""Generate JSON schema for dynamic tool calling with a given data model.
Creates a temporary toolkit, registers the structure output function,
and configures it with the specified data model to generate appropriate
JSON schemas for tool calling.
Args:
data_model_type: A Pydantic BaseModel class that defines the expected
structure of the tool output.
Returns:
A list of dictionary that contains the JSON schemas for
the configured tool, suitable for use in API calls that
support structured outputs.
Example:
class MyModel(BaseModel):
name: str
value: int
schema = get_dynamic_tool_call_json(MyModel)
"""
tmp_toolkit = Toolkit()
tmp_toolkit.register_tool_function(generate_structure_output)
tmp_toolkit.set_extended_model(
"generate_structure_output",
data_model_type,
)
return tmp_toolkit.get_json_schemas()
def get_structure_output(blocks: list | Sequence) -> dict:
"""Extract structured output from a sequence of blocks.
Processes a list or sequence of blocks to extract tool use outputs
and combine them into a single dictionary. This is typically used
to parse responses from language models that include tool calls.
Args:
blocks: A list or sequence of blocks that may contain tool use
information. Each block should be a dictionary with 'type'
and 'input' keys for tool use blocks.
Returns:
A dictionary containing the combined input data from all tool
use blocks found in the input sequence.
Example:
blocks = [
{"type": "tool_use", "input": {"name": "test"}},
{"type": "text", "content": "Some text"},
{"type": "tool_use", "input": {"value": 42}}
]
result = PromptBase.get_structure_output(blocks)
# result: {"name": "test", "value": 42}
"""
dict_output = {}
for block in blocks:
if isinstance(block, dict) and block.get("type") == "tool_use":
dict_output.update(block.get("input", {}))
return dict_output
def load_prompt_dict() -> dict:
"""Load prompt into dict"""
prompt_dict = {}
cur_dir = os.path.dirname(os.path.abspath(__file__))
prompt_dict["add_note"] = get_prompt_from_file(
file_path=os.path.join(
cur_dir,
"built_in_prompt/prompt_worker_additional_sys_prompt.md",
),
return_json=False,
)
prompt_dict["tool_use_rule"] = get_prompt_from_file(
file_path=os.path.join(
cur_dir,
"built_in_prompt/prompt_tool_usage_rules.md",
),
return_json=False,
)
prompt_dict["decompose_sys_prompt"] = get_prompt_from_file(
file_path=os.path.join(
cur_dir,
"built_in_prompt/prompt_decompose_subtask.md",
),
return_json=False,
)
prompt_dict["expansion_sys_prompt"] = get_prompt_from_file(
file_path=os.path.join(
cur_dir,
"built_in_prompt/prompt_deeper_expansion.md",
),
return_json=False,
)
prompt_dict["summarize_sys_prompt"] = get_prompt_from_file(
file_path=os.path.join(
cur_dir,
"built_in_prompt/prompt_inprocess_report.md",
),
return_json=False,
)
prompt_dict["reporting_sys_prompt"] = get_prompt_from_file(
file_path=os.path.join(
cur_dir,
"built_in_prompt/prompt_deepresearch_summary_report.md",
),
return_json=False,
)
prompt_dict["reflect_sys_prompt"] = get_prompt_from_file(
file_path=os.path.join(
cur_dir,
"built_in_prompt/prompt_reflect_failure.md",
),
return_json=False,
)
prompt_dict["reasoning_prompt"] = (
"## Current Subtask:\n{objective}\n"
"## Working Plan:\n{meta_planner_agent}\n"
"{knowledge_gap}\n"
"## Research Depth:\n{depth}"
)
prompt_dict["previous_plan_inst"] = (
"## Previous Plan:\n{previous_plan}\n"
"## Current Subtask:\n{objective}\n"
)
prompt_dict["max_depth_hint"] = (
"The search depth has reached the maximum limit. So the "
"current subtask can not be further decomposed and "
"expanded anymore. I need to find another way to get it "
"done no matter what."
)
prompt_dict["expansion_inst"] = (
"Review the web search results and identify whether "
"there is any information that can potentially help address "
"checklist items or fulfill knowledge gaps of the task, "
"but whose content is limited or only briefly mentioned.\n"
"**Task Description:**\n{objective}\n"
"**Checklist:**\n{checklist}\n"
"**Knowledge Gaps:**\n{knowledge_gaps}\n"
"**Search Results:**\n{search_results}\n"
"**Output:**\n"
)
prompt_dict["follow_up_judge_sys_prompt"] = (
"To provide sufficient external information for the user's "
"query, you have conducted a web search to obtain additional "
"data. However, you found that some of the information, while "
"important, was insufficient. Consequently, you extracted the "
"entire content from one of the URLs to gather more "
"comprehensive information. Now, you must rigorously and "
"carefully assess whether, after both the web search and "
"extraction process, the information content is adequate to "
"address the given task. Be aware that any arbitrary decisions "
"may result in unnecessary and unacceptable time costs.\n"
)
prompt_dict[
"retry_hint"
] = "Something went wrong when {state}. I need to retry."
prompt_dict["need_deeper_hint"] = (
"The information is insufficient and I need to make deeper "
"research to fill the knowledge gap."
)
prompt_dict[
"sufficient_hint"
] = "The information after web search and extraction is sufficient enough!"
prompt_dict["no_result_hint"] = (
"I mistakenly called the `summarize_intermediate_results` tool as "
"there exists no milestone result to summarize now."
)
prompt_dict["summarize_hint"] = (
"Based on your work history above, examine which step in the "
"following working meta_planner_agent has been completed. Mark the completed "
"step with [DONE] at the end of its line (e.g., k. step k [DONE]) "
"and leave the uncompleted steps unchanged. You MUST return only "
"the updated meta_planner_agent, preserving exactly the same format as the "
"original meta_planner_agent. Do not include any explanations, reasoning, "
"or section headers such as '## Working Plan:', just output the"
"updated meta_planner_agent itself."
"\n\n## Working Plan:\n{meta_planner_agent}"
)
prompt_dict["summarize_inst"] = (
"**Task Description:**\n{objective}\n"
"**Checklist:**\n{knowledge_gaps}\n"
"**Knowledge Gaps:**\n{working_plan}\n"
"**Search Results:**\n{tool_result}"
)
prompt_dict["update_report_hint"] = (
"Due to the overwhelming quantity of information, I have replaced the "
"original bulk search results from the research phase with the "
"following report that consolidates and summarizes the essential "
"findings:\n {intermediate_report}\n\n"
"Such report has been saved to the {report_path}. "
"I will now **proceed to the next item** in the working meta_planner_agent."
)
prompt_dict["save_report_hint"] = (
"The milestone results of the current item in working meta_planner_agent "
"are summarized into the following report:\n{intermediate_report}"
)
prompt_dict["reflect_instruction"] = (
"## Work History:\n{conversation_history}\n"
"## Working Plan:\n{meta_planner_agent}\n"
)
prompt_dict["subtask_complete_hint"] = (
"Subtask ‘{cur_obj}’ is completed. Now the current subtask "
"fallbacks to '{next_obj}'"
)
return prompt_dict

3
deep_research/qwen_langgraph_search_fullstack_runtime/.env.template Normal file
View File

@@ -0,0 +1,3 @@
DASHSCOPE_API_KEY=''
QUARK_AK=''
QUARK_SK=''

125
deep_research/qwen_langgraph_search_fullstack_runtime/README.md Normal file
View File

@@ -0,0 +1,125 @@
# DeepSearch Demo of Agentscope-Runtime with Langgraph / Qwen and Quark search
This project is modified from [Gemini Fullstack LangGraph Quickstart](https://github.com/google-gemini/gemini-fullstack-langgraph-quickstart).
It contains following key features:
1. We use LangGraph to build an agent (directed state graph) with the help of Qwen and Quark search.
2. The agent is wrapped as an Agentscope-Runtime agent and deployed as a service.
3. The interaction with the agent is done through a simple CLI.
Click following image to watch the video demo:
[![watch_the_video](https://img.alicdn.com/imgextra/i3/6000000000386/O1CN01vDit5y1EipxsRceBd_!!6000000000386-0-tbvideo.jpg)](https://cloud.video.taobao.com/vod/-BhtPfhYZv8pCz7L1vYmKCDtf1QEaDXNX1hMnvj_BUQ.mp4)
<br />
## 🌳 Project Structure
```bash
├── src # Source code directory containing the core functionalities and modules
│ ├── init.py # Initialization script, possibly setting up environment or configurations
│ ├── configuration.py # Module for handling application configurations and settings
│ ├── custom_search_tool.py # Implements custom search functionality or tool
│ ├── graph_openai_compatible.py # Module for OpenAI-compatible graph operations or integrations
│ ├── llm_prompts.py # Contains large language model prompts used in the application
│ ├── llm_utils.py # Utility functions for handling large language model operations
│ ├── main.py # Main entry script to launch or execute the application
│ ├── state.py # Manages application state or data persistence
│ ├── tools_and_schemas.py # Defines various tools and data schemas used by the application
│ └── utils.py # General utility functions used across the application
└── README.md # Project documentation file providing information and usage instructions
```
## Architecture
The architecture of the demo is shown in the following diagram:
```mermaid
graph LR;
subgraph As["AgentScope Runtime"]
F[Agent Engine]
end
subgraph Bs["LangGraph"]
B1[Web Research]
B2[Reflection]
B3[Answer Generation]
B0[Generation Queries]
B0 --> B1
B1 --> B2
B2 --> |if insufficient| B1
B2 --> |if sufficient| B3
end
subgraph Cs["CLI Service"]
C[main] --> S[WebSearchGraph]
end
As --> Bs
user --> |user input| C
S--> F
```
## 📖 Overview
This demo demonstrates how to build a sophisticated research agent using:
- Qwen as the underlying language model
- LangGraph for defining complex agent workflows
- Custom search tools for information retrieval
- State management for multi-step reasoning
The implementation showcases advanced patterns for building agentic systems that can perform deep research tasks through iterative thinking and tool usage.
## ⚙️ Components
### Core Modules
- `configuration.py`: Configuration management for the agent
- `custom_search_tool.py`: Custom search functionality implementation
- `graph_openai_compatible.py`: LangGraph implementation with OpenAI compatibility
- `llm_prompts.py`: Prompt templates for different agent behaviors
- `llm_utils.py`: Utility functions for LLM interactions
- `main.py`: Main entry point for the application
- `state.py`: State management for the LangGraph workflow
- `tools_and_schemas.py`: Tool definitions and data schemas
- `utils.py`: General utility functions
## 🚀 Getting Started
## Install
Follow these steps to get the application running locally for development and testing.
**1. Prerequisites:**
- Python 3.11+
- Create a file named `.env` by copying the `.env.example` file.
- **`DASHSCOPE_API_KEY`**:
add `DASHSCOPE_API_KEY="YOUR_ACTUAL_API_KEY"` if you use dashscope API.
- Quark Search API KEY: add `QUARK_AK=''` and `QUARK_SK=''` to `.env` file if you use quark search API.
**2. Install Dependencies:**
```bash
pip install -r requirements.txt
```
## Usage
Start the CLI Service.
```bash
cd src
python main.py
```
After that you can use the CLI to interact with the agent.
## 🛠️ Features
- `qwen` integration for advanced language understanding
- `langgraph` for complex workflow management
- Custom search tools for information retrieval
- Multi-step reasoning capabilities
- Stateful agent interactions
- Research-focused agent workflows
## ℹ️ Getting help
If you have any questions or if you found any problems with this demo, please report through [GitHub issues](https://github.com/your-org/demohouse/issues).
## 📄 License
This project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details.

10
deep_research/qwen_langgraph_search_fullstack_runtime/requirements.txt Normal file
View File

@@ -0,0 +1,10 @@
langgraph>=0.2.6
langchain>=0.3.19
python-dotenv>=1.0.1
langgraph-sdk>=0.1.57
langgraph-cli>=0.4.4
langgraph-api>=0.4.43
dashscope>=1.24.6
openai>=2.4.0
pandas>=2.3.3
agentscope-runtime>=0.1.5

0
deep_research/qwen_langgraph_search_fullstack_runtime/src/__init__.py Normal file
View File

78
deep_research/qwen_langgraph_search_fullstack_runtime/src/configuration.py Normal file
View File

@@ -0,0 +1,78 @@
# -*- coding: utf-8 -*-
import os
from typing import Any, Optional
from langchain_core.runnables import RunnableConfig
from pydantic import BaseModel, Field
class Configuration(BaseModel):
"""The configuration for the agent."""
query_generator_model: str = Field(
default="qwen-max-latest",
metadata={
"description": "The name of the language model to use for "
"the agent's query generation.",
},
)
query_generator_param: dict = Field(
default={"temperature": 0.3, "stream": False},
)
reflection_model: str = Field(
default="qwen-plus-latest",
metadata={
"description": "The name of the language model to use for"
" the agent's reflection.",
},
)
reflection_param: dict = Field(
default={"temperature": 0.3, "stream": False},
)
answer_model: str = Field(
default="qwen-plus-latest",
metadata={
"description": "The name of the language model to use "
"for the agent's answer.",
},
)
answer_param: dict = Field(default={"temperature": 0.3, "stream": False})
num_of_init_q: int = Field(
default=3,
metadata={
"description": "The number of initial search queries to generate.",
},
)
max_research_loops: int = Field(
default=2,
metadata={
"description": "The maximum number of research loops to perform.",
},
)
@classmethod
def from_runnable_config(
cls,
config: Optional[RunnableConfig] = None,
) -> "Configuration":
"""Create a Configuration instance from a RunnableConfig."""
configurable = (
config["configurable"]
if config and "configurable" in config
else {}
)
# Get raw values from environment or config
raw_values: dict[str, Any] = {
name: os.environ.get(name.upper(), configurable.get(name))
for name in cls.model_fields.keys()
}
# Filter out None values
values = {k: v for k, v in raw_values.items() if v is not None}
return cls(**values)

123
deep_research/qwen_langgraph_search_fullstack_runtime/src/custom_search_tool.py Normal file
View File

@@ -0,0 +1,123 @@
# -*- coding: utf-8 -*-
import os
import random
import string
import time
import uuid
from base64 import b64encode
from hashlib import sha256
from hmac import new as hmac_new
from typing import Any, Dict, List
import requests
from utils import format_time
class CustomSearchTool:
def __init__(self, search_engine: str = "quark"):
assert search_engine in ["quark"]
self.search_engine = search_engine
if self.search_engine == "quark":
self.search_func = self._quark_search
else:
raise NotImplementedError
self.search_engine = search_engine
def search(
self,
query: str,
) -> List[Dict[str, Any]]:
"""
Execute search and return the results
:param query:
:param num_results:
:return:
"""
return self.search_func(query)
def search_quark_to_b_signature(self, user_name, timestamp, salt: str, sk):
"""
signature
:param user_name: username
:param timestamp: timestamp
:param salt: salt
:param sk:
:return:
"""
data = f"{user_name}{timestamp}{salt}{sk}"
hashed = hmac_new(sk.encode("utf-8"), data.encode("utf-8"), sha256)
return b64encode(hashed.digest()).decode("utf-8")
def search_quark_to_b_gen_token(self, user_name: str, sk: str):
"""
get token
:param user_name:
:param sk:
:return:
"""
timestamp = str(int(time.time() * 1000))
salt = "".join(random.choice(string.ascii_lowercase) for _ in range(6))
sign = self.search_quark_to_b_signature(user_name, timestamp, salt, sk)
postBody = {
"userName": user_name,
"timestamp": timestamp,
"salt": salt,
"sign": sign,
}
url = "https://zx-dsc.sm.cn/api/auth/token"
headers = {"content-type": "application/json"}
response = requests.post(url, json=postBody, headers=headers)
data = response.json()
token = data["result"]["token"]
return token
def _quark_search(self, query: str):
ak = os.getenv("QUARK_AK", "")
sk = os.getenv("QUARK_SK", "")
token = self.search_quark_to_b_gen_token(ak, sk)
url = "https://zx-dsc.sm.cn/api/resource/s_agg/ex/query"
querystring = {
"page": "1",
"q": query,
}
request_id = str(uuid.uuid4())
headers = {
"Authorization": f"Bearer {token}",
"request-id": request_id,
}
try:
response = requests.get(url, headers=headers, params=querystring)
if response.status_code == 200:
data = response.json()
if (
data.get("items", {}).get("@attributes", {}).get("status")
== "OK"
and data.get(
"items",
)
and data.get("items", {}).get("item")
):
items = data.get("items").get("item")
formated_items = []
for item in items:
formated_items.append(
{
"title": item["title"],
"url": item["url"],
"snippet": item["desc"],
"content": item["MainBody"],
"publish_date": format_time(item.get("time")),
"site_name": item.get("site_name", ""),
},
)
return formated_items
else:
return []
else:
return []
except Exception as e:
print(f"Quark search failed: {e}")
return []

526
deep_research/qwen_langgraph_search_fullstack_runtime/src/graph_openai_compatible.py Normal file
View File

@@ -0,0 +1,526 @@
# -*- coding: utf-8 -*-
import asyncio
import json
import os
import time
from typing import Any, Dict, List, Optional
from agentscope_runtime.engine.agents.langgraph_agent import LangGraphAgent
from agentscope_runtime.engine.helpers.helper import simple_call_agent_direct
from configuration import Configuration
from custom_search_tool import CustomSearchTool
from dotenv import load_dotenv
from langchain_core.messages import AIMessage, HumanMessage
from langchain_core.runnables import RunnableConfig
from langgraph.graph import END, START, StateGraph
from langgraph.types import Send
from llm_prompts import (
answer_instructions,
query_writer_instructions,
reflection_instructions,
web_searcher_instructions,
)
from llm_utils import call_dashscope, extract_json_from_qwen
from state import (
OverallState,
QueryGenerationState,
ReflectionState,
WebSearchState,
)
from utils import (
custom_get_citations,
custom_resolve_urls,
get_current_date,
get_research_topic,
insert_citation_markers,
)
load_dotenv("../.env")
if os.getenv("DASHSCOPE_API_KEY") is None:
raise ValueError("DASHSCOPE_API_KEY is not set")
def format_search_results(search_results: List[Dict[str, Any]]) -> str:
"""
Convert the search results
:param search_results:
:return:
"""
formatted_results = []
for i, result in enumerate(search_results, 1):
formatted_result = f"""
Result Number {i}:
Title: {result.get('title', 'N/A')}
Label：{result.get('site_name', 'N/A')}
URL: {result.get('url', 'N/A')}
Snippet: {result.get('snippet', 'N/A')}
publish_date: {result.get('publish_date', 'N/A')}
---
"""
formatted_results.append(formatted_result)
return "\n".join(formatted_results)
class WebSearchGraph:
def __init__(
self,
config: RunnableConfig,
call_llm_func,
search_tool: CustomSearchTool,
):
self.configurable = Configuration.from_runnable_config(config)
self.call_llm_func = call_llm_func
self.search_tool = search_tool
self.input_tokens = 0
self.output_tokens = 0
self.total_tokens = 0
self.max_retries = 3
self.retry_delay = 2
self.current_date = get_current_date()
def get_chat_completion(self, **args):
completion = self.call_llm_func(**args)
self.input_tokens += completion.usage.prompt_tokens
self.output_tokens += completion.usage.completion_tokens
self.total_tokens += completion.usage.total_tokens
return completion.choices[0].message.content
def generate_query(self, state: OverallState) -> QueryGenerationState:
"""LangGraph node that generates search queries
based on the User's question.
Uses QWen Max to create optimized search queries
for web research based on the User's question.
Args:
state: Current graph state containing the User's question
config: Configuration for the runnable,
including LLM provider settings
Returns:
Dictionary with state update,
including search_query key containing the
generated queries
"""
# check for custom initial search query count
if state.get("initial_search_query_count") is None:
state[
"initial_search_query_count"
] = self.configurable.num_of_init_q
# Format the prompt
formatted_prompt = query_writer_instructions.format(
current_date=self.current_date,
research_topic=get_research_topic(state["messages"]),
number_queries=state["initial_search_query_count"],
)
param = {
"model": self.configurable.query_generator_model,
"messages": [{"role": "user", "content": formatted_prompt}],
**self.configurable.query_generator_param,
}
for attempt in range(self.max_retries):
try:
result = self.get_chat_completion(**param)
result = extract_json_from_qwen(result)
result = json.loads(result)
query = result.get("query")
if isinstance(query, str):
query = [query]
assert isinstance(query, list)
break
except Exception as e:
print(
f"Error occurred when generating search query (attempt"
f" {attempt + 1}/{self.max_retries}): {e}.",
)
if attempt == self.max_retries - 1: # Last attempt failed
query = [get_research_topic(state["messages"])]
break
time.sleep(self.retry_delay)
return {"search_query": query}
def continue_to_web_research(self, state: QueryGenerationState):
"""LangGraph node that sends the
search queries to the web research node.
This is used to spawn n number
of web research nodes, one for each search query.
"""
return [
Send(
"web_research",
{"search_query": search_query, "id": str(idx)},
)
for idx, search_query in enumerate(state["search_query"])
]
def web_research(self, state: WebSearchState):
"""LangGraph node that performs web research using the native Google
Search API tool.
Executes a web search using the native Google Search API tool in
combination with Gemini 2.0 Flash.
Args:
state: Current graph state containing the
search query and research loop count
config: Configuration for the runnable,
including search API settings
Returns:
Dictionary with state update,
including sources_gathered, research_loop_count,
and web_research_results
"""
search_results = self.search_tool.search(
state["search_query"],
)
search_context = format_search_results(search_results)
formatted_prompt = (
web_searcher_instructions.format(
current_date=self.current_date,
research_topic=state["search_query"],
)
+ f"\n\nSearch Result:\n{search_context}"
)
param = {
"model": self.configurable.query_generator_model,
"messages": [{"role": "user", "content": formatted_prompt}],
**self.configurable.query_generator_param,
}
sources_gathered = []
for result in search_results:
url = result.get("url")
if url:
sources_gathered.append(
{
"label": result.get("site_name"),
"short_url": url,
"value": url,
},
)
for attempt in range(self.max_retries):
try:
result = self.get_chat_completion(**param)
resolved_urls = custom_resolve_urls(
search_results,
state["id"],
)
citations = custom_get_citations(search_results, resolved_urls)
modified_text = insert_citation_markers(result, citations)
return {
"sources_gathered": sources_gathered,
"search_query": [state["search_query"]],
"web_research_result": [modified_text],
}
except Exception as e:
print(
f"Error occurred when web search query: "
f"`{state['search_query']}` "
f"(attempt {attempt + 1}/{self.max_retries}): {e}.",
)
summary = (
f"{len(search_results)} related results are found "
f"about search query '{state['search_query']}'"
)
if attempt == self.max_retries - 1:
return {
"sources_gathered": sources_gathered,
"search_query": [state["search_query"]],
"web_research_result": [summary],
}
time.sleep(self.retry_delay)
return None
def reflection(self, state: OverallState) -> Optional[ReflectionState]:
"""LangGraph node that identifies knowledge gaps and generates
potential follow-up queries.
Analyzes the current summary to identify areas for further
research and generates
potential follow-up queries. Uses structured output to extract
the follow-up query in JSON format.
Args:
state: Current graph state containing the running summary
and research topic
config: Configuration for the runnable, including LLM
provider settings
Returns:
Dictionary with state update, including search_query key
containing the generated follow-up query
"""
state["research_loop_count"] = state.get("research_loop_count", 0) + 1
reasoning_model = self.configurable.reflection_model
# Format the prompt
formatted_prompt = reflection_instructions.format(
current_date=self.current_date,
research_topic=get_research_topic(state["messages"]),
summaries="\n\n---\n\n".join(state["web_research_result"]),
)
param = {
"model": reasoning_model,
"messages": [{"role": "user", "content": formatted_prompt}],
**self.configurable.reflection_param,
}
for attempt in range(self.max_retries):
try:
result = self.get_chat_completion(**param)
result = extract_json_from_qwen(result)
result = json.loads(result)
is_sufficient = result.get("is_sufficient", True)
knowledge_gap = result.get("knowledge_gap", "")
follow_up_queries = result.get("follow_up_queries", [])
assert isinstance(follow_up_queries, list)
return {
"is_sufficient": is_sufficient,
"knowledge_gap": knowledge_gap,
"follow_up_queries": follow_up_queries,
"research_loop_count": state["research_loop_count"],
"number_of_ran_queries": len(state["search_query"]),
}
except Exception as e:
print(
f"Error occurred when reflection (attempt {attempt + 1}"
f"/{self.max_retries}): {e}.",
)
if attempt == self.max_retries - 1: # Last attempt failed
return {
"is_sufficient": True,
"knowledge_gap": "",
"follow_up_queries": [],
"research_loop_count": state["research_loop_count"],
"number_of_ran_queries": len(state["search_query"]),
}
time.sleep(self.retry_delay)
return None
def evaluate_research(
self,
state: ReflectionState,
config: RunnableConfig,
):
"""LangGraph routing function that determines the next step in the
research flow.
Controls the research loop by deciding whether to continue gathering
information
or to finalize the summary based on the configured maximum number of
research loops.
Args:
state: Current graph state containing the research loop count
config: Configuration for the runnable, including
max_research_loops setting
Returns:
String literal indicating the next node to visit ("web_research"
or "finalize_summary")
"""
configurable = Configuration.from_runnable_config(config)
max_research_loops = (
state.get("max_research_loops")
if state.get("max_research_loops") is not None
else configurable.max_research_loops
)
if (
state["is_sufficient"]
or state["research_loop_count"] >= max_research_loops
):
return "finalize_answer"
else:
return [
Send(
"web_research",
{
"search_query": follow_up_query,
"id": state["number_of_ran_queries"] + int(idx),
},
)
for idx, follow_up_query in enumerate(
state["follow_up_queries"],
)
]
def finalize_answer(self, state: OverallState):
"""LangGraph node that finalizes the research summary.
Prepares the final output by deduplicating and formatting sources, then
combining them with the running summary to create a well-structured
research report with proper citations.
Args:
state: Current graph state containing the running summary
and sources gathered
Returns:
Dictionary with state update, including running_summary
key containing
the formatted final summary with sources
"""
answer_model = self.configurable.answer_model
formatted_prompt = answer_instructions.format(
current_date=self.current_date,
research_topic=get_research_topic(state["messages"]),
summaries="\n---\n\n".join(state["web_research_result"]),
)
param = {
"model": answer_model,
"messages": [{"role": "user", "content": formatted_prompt}],
**self.configurable.answer_param,
}
for attempt in range(self.max_retries):
try:
result = self.get_chat_completion(**param)
unique_sources = []
for source in state["sources_gathered"]:
if source["short_url"] in result:
result = result.replace(
source["short_url"],
source["value"],
)
unique_sources.append(source)
return {
"messages": [AIMessage(content=result)],
"sources_gathered": unique_sources,
}
except Exception as e:
print(
f"Error occurred when generating answer (attempt "
f"{attempt + 1}/{self.max_retries}): {e}.",
)
if attempt == self.max_retries - 1:
return {
"messages": [
AIMessage(
content=f"Error occurred"
f" when generating answer. {e}",
),
],
"sources_gathered": [],
}
time.sleep(self.retry_delay)
return None
async def run(self, user_question: str):
# Create our Agent Graph
builder = StateGraph(OverallState, config_schema=Configuration)
# Define the nodes we will cycle between
builder.add_node("generate_query", self.generate_query)
builder.add_node("web_research", self.web_research)
builder.add_node("reflection", self.reflection)
builder.add_node("finalize_answer", self.finalize_answer)
# Set the entrypoint as `generate_query`
# This means that this node is the first one called
builder.add_edge(START, "generate_query")
# Add conditional edge to continue with search queries in a
# parallel branch
builder.add_conditional_edges(
"generate_query",
self.continue_to_web_research,
["web_research"],
)
# Reflect on the web research
builder.add_edge("web_research", "reflection")
# Evaluate the research
builder.add_conditional_edges(
"reflection",
self.evaluate_research,
["web_research", "finalize_answer"],
)
# Finalize the answer
builder.add_edge("finalize_answer", END)
compiled_graph = builder.compile(name="pro-search-agent")
def human_ai_message_to_dict(obj):
if isinstance(obj, HumanMessage):
return {
"sender": obj.type,
"content": obj.content,
}
if isinstance(obj, AIMessage):
return {
"sender": obj.type,
"content": obj.content,
}
raise TypeError(
f"Object of type {obj.__class__.__name__} is"
f" not JSON serializable",
)
def state_folder(messages):
if len(messages) > 0:
return json.loads(messages[0]["content"])
else:
return []
def state_unfolder(state):
state_jsons = json.dumps(state, default=human_ai_message_to_dict)
return state_jsons
langgraph_agent = LangGraphAgent(
compiled_graph,
state_folder,
state_unfolder,
)
input_state = {
"messages": [{"role": "user", "content": user_question}],
"max_research_loops": self.configurable.max_research_loops,
"initial_search_query_count": self.configurable.num_of_init_q,
}
input_json = json.dumps(input_state)
all_result = await simple_call_agent_direct(
langgraph_agent,
input_json,
)
state = json.loads(all_result)
return state["messages"][-1]["content"]
async def main():
custom_search_tool = CustomSearchTool(search_engine="quark")
graph = WebSearchGraph(
json.loads(Configuration().model_dump_json()),
call_dashscope,
custom_search_tool,
)
print(
"""Type in your question or q to quit.""",
)
user_input = input(">").strip()
while user_input != "q":
question = user_input
item = await graph.run(question)
print(item, end="", flush=True)
print("\n")
user_input = input(">")
if __name__ == "__main__":
asyncio.run(main())

134
deep_research/qwen_langgraph_search_fullstack_runtime/src/llm_prompts.py Normal file
View File

@@ -0,0 +1,134 @@
# -*- coding: utf-8 -*-
query_writer_instructions = """Your goal is to generate sophisticated and
diverse web search queries.
These queries are intended for an advanced automated web research tool capable
of analyzing complex results,
following links, and synthesizing information.
Instructions:
- Always prefer a single search query, only add another query if the original
question requests multiple aspects or elements and one query is not enough.
- Each query should focus on one specific aspect of the original question.
- Don't produce more than {number_queries} queries.
- Queries should be diverse, if the topic is broad, generate more than 1 query.
- Don't generate multiple similar queries, 1 is enough.
- Query should ensure that the most current information is gathered. The
current date is {current_date}.
Format:
- Format your response as a JSON object with ALL three of these exact keys:
- "rationale": str, A brief explanation of why these queries are relevant
to the research topic.
- "query": list[str], A list of search queries to be used for web research.
Example:
Topic: What revenue grew more last year apple stock or the number of people
buying an iphone
```json
{{
"rationale": "To answer this comparative growth question accurately,
we need specific data points on Apple's stock performance and iPhone
sales metrics. These queries target the precise financial information
needed: company revenue trends, product-specific unitsales figures,
and stock price movement over the same fiscal period for
direct comparison.",
"query": ["Apple total revenue growth fiscal year 2024", "iPhone unit sales
growth fiscal
year 2024", "Apple stock price growth fiscal year 2024"],
}}
```
Context: {research_topic}"""
web_searcher_instructions = """Conduct targeted Google Searches to gather the
most recent, credible
information on "{research_topic}" and synthesize it into a verifiable text
artifact.
Instructions:
- Query should ensure that the most current information is gathered. The
current date is {current_date}.
- Conduct multiple, diverse searches to gather comprehensive information.
- Consolidate key findings while meticulously tracking the source(s) for each
specific piece of information.
- The output should be a well-written summary or report based on your search
findings.
- Only include the information found in the search results, don't make up any
information.
Research Topic:
{research_topic}
"""
reflection_instructions = """You are an expert research assistant analyzing
summaries about "{research_topic}".
Instructions:
- Identify knowledge gaps or areas that need deeper exploration and generate a
follow-up query. (1 or multiple).
- If provided summaries are sufficient to answer the user's question, don't
generate a follow-up query.
- If there is a knowledge gap, generate a follow-up query that would help
expand your understanding.
- Focus on technical details, implementation specifics, or emerging trends
that weren't fully covered.
Requirements:
- Ensure the follow-up query is self-contained and includes necessary context
for web search.
Output Format:
- Format your response as a JSON object with these exact keys:
- "is_sufficient": true or false. Whether the provided summaries are
sufficient to answer the user's question.
- "knowledge_gap": str, A description of what information is missing or
needs clarification.
- "follow_up_queries": list, A list of follow-up queries to address the
knowledge gap.
Example:
```json
{{
"is_sufficient": true, // or false
"knowledge_gap": "The summary lacks information about performance metrics
and benchmarks", //
"" if is_sufficient is true
"follow_up_queries": ["What are typical performance benchmarks and metrics
used to evaluate
[specific technology]?"]
// [] if is_sufficient is true
}}
```
Reflect carefully on the Summaries to identify knowledge gaps and produce a
follow-up query.
Then, produce your output following this JSON format:
Summaries:
{summaries}
"""
answer_instructions = """Generate a high-quality answer to the user's question
based on the provided summaries.
Instructions:
- The current date is {current_date}.
- You are the final step of a multi-step research process, don't mention that
you are the final step.
- You have access to all the information gathered from the previous steps.
- You have access to the user's question.
- Generate a high-quality answer to the user's question based on the provided
summaries
and the user's question.
- Include the sources you used from the Summaries in the answer correctly,
use markdown format. THIS IS A MUST.
User Context:
- {research_topic}
Summaries:
{summaries}"""

177
deep_research/qwen_langgraph_search_fullstack_runtime/src/llm_utils.py Normal file
View File

@@ -0,0 +1,177 @@
# -*- coding: utf-8 -*-
import json
import os
import re
from collections import defaultdict
from datetime import datetime
from typing import Any, Dict, Iterator, List, Optional
from openai import OpenAI
from openai.types.chat.chat_completion import (
ChatCompletion,
ChatCompletionMessage,
Choice,
)
from openai.types.chat.chat_completion_message_tool_call import (
ChatCompletionMessageToolCall,
Function,
)
def extract_json_from_qwen(qwen_result: str) -> str:
sql = ""
pattern = r"```json(.*?)```"
sql_code_snippets = re.findall(pattern, qwen_result, re.DOTALL)
if len(sql_code_snippets) > 0:
sql = sql_code_snippets[-1].strip()
return sql
def call_dashscope(**args: Any) -> ChatCompletion:
client = OpenAI(
api_key=os.getenv("DASHSCOPE_API_KEY"),
base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
)
completion = client.chat.completions.create(
**args,
)
stream = args.get("stream", False)
if stream:
try:
completion = postprocess_completion(completion)
return completion
except Exception as e:
print(
f"Error occurred when postprocess_completion on "
f"'stream=True'. {e}",
)
default_message = ChatCompletionMessage(
role="assistant",
content="Error in calling LLM", # 默认内容
)
default_choice = Choice(
finish_reason="stop",
index=0,
logprobs=None,
message=default_message,
)
default_chat_completion = ChatCompletion(
id="chatcmpl-1234567890",
choices=[default_choice],
created=int(datetime.now().timestamp()),
model=args["model"],
object="chat.completion",
service_tier="default",
system_fingerprint=None,
usage=None,
)
return default_chat_completion
return completion
def merge_fields(target: Dict[str, Any], source: Dict[str, Any]) -> None:
for key, value in source.items():
if isinstance(value, str):
target[key] = target.get(key, "") + value
elif value is not None and isinstance(value, dict):
merge_fields(target[key], value)
def merge_chunk(final_response: Dict[str, Any], delta: Dict[str, Any]) -> None:
delta.pop("role", None)
merge_fields(final_response, delta)
tool_calls = delta.get("tool_calls")
if tool_calls and len(tool_calls) > 0:
index = int(tool_calls[0].pop("index")) # Convert index to integer
if "tool_calls" not in final_response:
final_response["tool_calls"] = {}
final_response["tool_calls"][index] = final_response["tool_calls"].get(
index,
{},
)
final_response["tool_calls"][index].pop("type", None)
merge_fields(final_response["tool_calls"][index], tool_calls[0])
def postprocess_completion(completion: Iterator) -> ChatCompletion:
message: Dict[str, Any] = {
"content": "",
"role": "assistant",
"function_call": None,
"tool_calls": defaultdict(
lambda: {
"function": {"arguments": "", "name": ""},
"id": "",
"type": "",
},
),
"reasoning_content": "",
"refusal": "",
}
last_chunk: Optional[Any] = None
for chunk in completion:
try:
delta = json.loads(chunk.choices[0].delta.json())
except json.JSONDecodeError as e:
print(f"Error decoding JSON from chunk: {e}")
continue
delta.pop("role", None)
merge_chunk(message, delta)
finish_reason = chunk.choices[0].finish_reason
logprobs = chunk.choices[0].logprobs
last_chunk = chunk
# 显式声明类型
tool_calls_list: List[Dict[str, Any]] = list(
message.get("tool_calls", {}).values(),
)
message["tool_calls"] = tool_calls_list
tool_calls = None
if message["tool_calls"]:
tool_calls = []
for tool_call in message["tool_calls"]: # 类型已明确为 Dict
function = Function(
arguments=tool_call["function"]["arguments"],
name=tool_call["function"]["name"],
)
tool_call_object = ChatCompletionMessageToolCall(
id=tool_call["id"],
function=function,
type=tool_call["type"],
)
tool_calls.append(tool_call_object)
chat_message = ChatCompletionMessage(
content=message["content"],
role=message["role"],
function_call=message["function_call"],
tool_calls=tool_calls,
reasoning_content=message["reasoning_content"],
refusal=message["refusal"],
)
choices = [
Choice(
finish_reason=finish_reason,
index=0,
message=chat_message,
logprobs=logprobs,
),
]
completion = ChatCompletion(
id=last_chunk.id,
choices=choices,
created=last_chunk.created,
model=last_chunk.model,
object="chat.completion",
service_tier=last_chunk.service_tier,
system_fingerprint=last_chunk.system_fingerprint,
usage=last_chunk.usage,
)
return completion

29
deep_research/qwen_langgraph_search_fullstack_runtime/src/main.py Normal file
View File

@@ -0,0 +1,29 @@
# -*- coding: utf-8 -*-
import asyncio
import json
from qwen_langgraph_search.src.configuration import Configuration
from qwen_langgraph_search.src.custom_search_tool import CustomSearchTool
from qwen_langgraph_search.src.graph_openai_compatible import WebSearchGraph
from qwen_langgraph_search.src.llm_utils import call_dashscope
if __name__ == "__main__":
custom_search_tool = CustomSearchTool(search_engine="quark")
graph = WebSearchGraph(
json.loads(Configuration().model_dump_json()),
call_dashscope,
custom_search_tool,
)
user_input = input("Type in your question or press q to quit\n")
while user_input != "q":
question = user_input
use_agentengine = True
try:
res = asyncio.run(graph.run(question))
print(res)
except Exception as e:
print(f"An error occurred: {e}")
user_input = input("Type in your question or press q to quit\n")

47
deep_research/qwen_langgraph_search_fullstack_runtime/src/state.py Normal file
View File

@@ -0,0 +1,47 @@
# -*- coding: utf-8 -*-
from __future__ import annotations
import operator
from dataclasses import dataclass, field
from typing import Optional, TypedDict
from langgraph.graph import add_messages
from typing_extensions import Annotated
class OverallState(TypedDict):
messages: Annotated[list, add_messages]
search_query: Annotated[list, operator.add]
web_research_result: Annotated[list, operator.add]
sources_gathered: Annotated[list, operator.add]
initial_search_query_count: int
max_research_loops: int
research_loop_count: int
reasoning_model: str
class ReflectionState(TypedDict):
is_sufficient: bool
knowledge_gap: str
follow_up_queries: Annotated[list, operator.add]
research_loop_count: int
number_of_ran_queries: int
class Query(TypedDict):
query: str
rationale: str
class QueryGenerationState(TypedDict):
search_query: list[Query]
class WebSearchState(TypedDict):
search_query: str
id: str
@dataclass(kw_only=True)
class SearchStateOutput:
running_summary: Optional[str] = field(default=None) # Final report

29
deep_research/qwen_langgraph_search_fullstack_runtime/src/tools_and_schemas.py Normal file
View File

@@ -0,0 +1,29 @@
# -*- coding: utf-8 -*-
from typing import List
from pydantic import BaseModel, Field
class SearchQueryList(BaseModel):
query: List[str] = Field(
description="A list of search queries to be used for web research.",
)
rationale: str = Field(
description="A brief explanation of why these queries are relevant "
"to the research topic.",
)
class Reflection(BaseModel):
is_sufficient: bool = Field(
description="Whether the provided summaries are sufficient to answer "
"the user's question.",
)
knowledge_gap: str = Field(
description="A description of what information is missing or needs "
"clarification.",
)
follow_up_queries: List[str] = Field(
description="A list of follow-up queries to address the knowledge "
"gap.",
)

129
deep_research/qwen_langgraph_search_fullstack_runtime/src/utils.py Normal file
View File

@@ -0,0 +1,129 @@
# -*- coding: utf-8 -*-
import time
from datetime import datetime
from typing import Any, Dict, List
from langchain_core.messages import AIMessage, AnyMessage, HumanMessage
def get_current_date() -> str:
return datetime.now().strftime("%B %d, %Y")
def format_time(timestamp_param: str, format_str: str = "%Y-%m-%d") -> str:
if not timestamp_param or not timestamp_param.isnumeric():
return ""
try:
timestamp = int(timestamp_param)
return time.strftime(format_str, time.localtime(timestamp))
except (ValueError, OverflowError, OSError):
return ""
def get_research_topic(messages: List[AnyMessage]) -> str:
"""
Get the research topic from the messages.
"""
# check if request has a history and combine the messages
# into a single string
if len(messages) == 1:
research_topic = messages[-1].content
else:
research_topic = ""
for message in messages:
if isinstance(message, HumanMessage):
research_topic += f"User: {message.content}\n"
elif isinstance(message, AIMessage):
research_topic += f"Assistant: {message.content}\n"
return research_topic
def insert_citation_markers(text: str, citations_list: List[Dict]) -> str:
"""
Inserts citation markers into a text string based on start and end indices.
Args:
text (str): The original text string.
citations_list (list): A list of dictionaries, where each dictionary
contains 'start_index', 'end_index', and
'segment_string' (the marker to insert).
Indices are assumed to be for the original text.
Returns:
str: The text with citation markers inserted.
"""
# Sort citations by end_index in descending order.
# If end_index is the same, secondary sort by start_index descending.
# This ensures that insertions at the end of the string don't affect
# the indices of earlier parts of the string that still
# need to be processed.
sorted_citations = sorted(
citations_list,
key=lambda c: (c["end_index"], c["start_index"]),
reverse=True,
)
modified_text = text
for citation_info in sorted_citations:
# These indices refer to positions in the *original* text,
# but since we iterate from the end, they remain valid for insertion
# relative to the parts of the string already processed.
end_idx = citation_info["end_index"]
marker_to_insert = ""
for segment in citation_info["segments"]:
marker_to_insert += (
f" [{segment['label']}]({segment['short_url']})"
)
# Insert the citation marker at the original end_idx position
modified_text = (
modified_text[:end_idx]
+ marker_to_insert
+ modified_text[end_idx:]
)
return modified_text
def custom_resolve_urls(
search_results: List[Dict[str, Any]],
uid: str,
) -> Dict[str, str]:
prefix = "https://search-result.local/id/"
resolved_map = {}
for idx, result in enumerate(search_results):
url = result.get("url", "")
if url and url not in resolved_map:
resolved_map[url] = f"{prefix}{uid}-{idx}"
return resolved_map
def custom_get_citations(
search_results: List[Dict[str, Any]],
resolved_urls_map: Dict[str, str],
) -> List[Dict[str, Any]]:
citations = []
for idx, result in enumerate(search_results):
url = result.get("url", "")
title = result.get("title", f"搜索结果 {idx + 1}")
if url:
citation = {
"start_index": 0, # 简化处理，实际应用中可以更精确
"end_index": len(title),
"segments": [
{
"label": title[:50] + "..."
if len(title) > 50
else title,
"short_url": resolved_urls_map.get(url, url),
"value": url,
},
],
}
citations.append(citation)
return citations