A new open-source search engine is rivaling top-tier AI products like Perplexity Pro and ChatGPT-Web.
MindSearch is an innovative AI search engine framework that combines LLMs and a multi-agent system to tackle three critical issues that often limit LLM-powered search engines:
1. LLMs struggle to decompose complex queries into simpler, actionable requests
2. Search results often contain too much noise, making it hard to filter and extract relevant information
3. Iterative searches can quickly overload the LLM’s input length capacity
MindSearch utilizes two main components:
WebPlanner - decomposes complex queries into sub-tasks and creates a dynamic graph structure for problem-solving
WebSearcher - conducts fine-grained searches and delivers summarized information back to WebPlanner for further refinement
This approach allows MindSearch to handle massive web content (e.g., more than 300 pages) effectively, surpassing limitations faced by traditional LLM-based search systems.
According to subjective evaluations from human experts, MindSearch significantly outperforms major search engines like ChatGPT-Web and Perplexity Pro. Its superior depth, breadth, and factual accuracy make it a breakthrough solution for both open-set and closed-set QA tasks.
Code github.com/InternLM/Min…