腾讯清华联合测试 主流AI搜索准确率最高不足五成

Tencent Hunyuan and Tsinghua University have jointly released the latest test results of mainstream AI search agents. Key findings are as follows:

1. The core problem of current AI search does not lie in errors during the retrieval phase. When user queries contain ambiguity, very few AI search systems proactively ask users for clarification. A single unresolved ambiguity can break the entire reasoning chain, and repeated retrieval without clarification performs worse than simply guessing an answer.

2. Even when given unambiguous prompts, all tested mainstream large model-powered AI search tools have an accuracy rate below 50%. Doubao Seed 2.0 Pro, the top performer, only scored 43.1%, while weaker models only reached just above 10%. Large models' built-in knowledge bases alone are completely insufficient for ambiguous search tasks, and accuracy plummets when search access is disabled.

3. Models that search first and then ask follow-up questions to clarify ambiguity achieve a 93.4% success rate, far exceeding the 56.5% success rate of guessing without follow-ups. AI search will definitely add interactive user clarification steps in the future, and users can proactively provide additional context to AI search when asking vague questions to improve accuracy.

These test results offer key insights for brands positioning themselves for AI search traffic and adapting to the new search environment. Key takeaways are as follows:

1. Current AI search requires extremely clear information. Removing ambiguity from queries can improve model accuracy by 26.8 to 40.2 percentage points, meaning the clarity of a brand's information directly determines whether AI search can correctly output brand-related results. Vague product parameters and scenario labels will sharply reduce the probability of a brand being retrieved and recommended by AI.

2. AI search relies heavily on public external search information. Even top-tier models see their accuracy drop by nearly half when access to search tools is cut off. This shows that building out public online brand information remains the core of customer acquisition in the AI search era, and brands do not need to over-rely on fine-tuning large models' built-in knowledge bases.

3. AI search will add interactive ambiguity clarification steps in the future. Brands should adapt to this new rule in advance by mapping users' vague search queries for their products across different scenarios, standardizing their public information, and getting a head start in capturing AI search traffic.

These test results clarify the risks and opportunities for sellers looking to tap into AI search as a new traffic channel. Key insights are as follows:

1. Current AI search performs extremely poorly when handling ambiguous user queries, and a single unresolved ambiguity can lead to completely incorrect search results. If a seller's product information is vaguely labeled, uses inconsistent keywords or unclear parameters, their chances of being correctly recommended by AI search will drop sharply, which is the core risk of entering AI search traffic today.

2. On the opportunity side, the overall accuracy of existing AI search is below 50%. As long as sellers standardize their product information and eliminate informational ambiguity, they can boost the accuracy of their products being retrieved and recommended by 20 to 40 percentage points, outperforming most peers and capturing early AI search traffic dividends ahead of competitors.

3. Leading large model developers are already optimizing their ambiguity handling capabilities, and AI search will certainly become an important new traffic channel. Sellers should keep up with technological changes, adjust their online information publishing rules in a timely manner, and adapt to the new search logic.

These test conclusions bring multiple insights for factories advancing digital transformation and e-commerce expansion. Key takeaways are as follows:

1. For online customer acquisition: AI search has become a new user traffic entry point, and current AI search requires extremely clear information. When publishing product information, factories should clearly label product parameters, applicable scenarios, versions and models to avoid AI retrieval errors caused by vague information that hurt customer acquisition performance.

2. For digital transformation: AI search technology can help factories improve the efficiency of supply chain information retrieval and market demand research, but current AI search performs poorly when handling unstandardized information. Factories that standardize internal information will see greatly improved efficiency when using AI tools, and can fully unlock the value of digital tools.

3. For new business opportunities: General-purpose AI search performs extremely poorly at handling ambiguity in professional production scenarios. Factories with technical capabilities can explore developing industry-specific AI search tools tailored to their production field to meet the demand for ambiguous information retrieval within the sector, and build a new growth curve.

This test clarifies the core pain points and development direction of the AI search field, and points out clear R&D directions for AI technology service providers. Key findings are as follows:

1. Core industry pain point: Mainstream large model-powered AI search performs extremely poorly when handling ambiguous queries in real-world scenarios. The average end-to-end accuracy is only 28.6% with unambiguous prompts, and even top models have accuracy below 50%. A single unresolved ambiguity can invalidate the entire result, leading to poor user experience, and the market has strong demand for mature optimization solutions.

2. Technology R&D direction: Prompting models to identify ambiguity only improves ambiguity recognition rates, and cannot effectively boost final retrieval accuracy. The core of future R&D is building a mechanism that converts search uncertainty into effective user interaction, to improve models' ability to proactively clarify ambiguity.

3. Leading players have already explored viable optimization directions: for example, Anthropic has increased the frequency of uncertainty tagging, and Perplexity has restructured its search workflow. Service providers can reference these directions to develop targeted solutions that match their own advantages, and capture first-mover advantage in the market.

This test reveals the core problems of current AI search platforms, and points out the direction for platform operation optimization. Key insights are as follows:

1. Core user demand: Most real-world user search queries are vague, ambiguous, or even incorrect, and existing AI search cannot meet this core user need. Platforms must prioritize improving interactive ambiguity clarification as a core optimization direction to boost user satisfaction and retention.

2. Risk mitigation: All current models rely heavily on external search tools, and built-in knowledge bases alone cannot complete search tasks. Even top models see accuracy plummet when search access is removed. Platforms should avoid the wrong strategic direction of over-relying on built-in knowledge bases and abandoning external search integration.

3. Ecosystem and operations: Ambiguity handling technology for AI search is still immature. Platforms can partner with relevant technology service providers to co-build an ecosystem and fill technical gaps. For model testing, platforms can adopt the newly released DiscoBench benchmark, which aligns with the typical characteristics of Chinese web search, can more accurately reflect models' real-world capabilities, and helps platforms screen for high-performing models.

This study delivers new research findings and directions for the AI search field. Key contributions are as follows:

1. New industry insight: The core bottleneck of current AI search agents is not retrieval or reasoning capability, but ambiguity handling and user interaction. The highest end-to-end accuracy among all tested mainstream models is below 50%, meaning technical maturity remains low and there is still large room for R&D and innovation.

2. New progress in research: All previously commonly used industry test benchmarks assume that user queries are unambiguous, which does not match real search scenarios. The newly launched DiscoBench benchmark fills this gap. It covers 11 knowledge domains, includes 211 tasks with 463 Chinese ambiguity instances, categorizes ambiguities into four main types, and is far more aligned with the actual characteristics of Chinese search.

3. Future research direction: This study finds that there is no strong correlation between ambiguity recognition ability and the quality of clarification follow-ups. Only improving ambiguity recognition rates cannot effectively boost final search accuracy. Future research should focus on how models can convert recognized uncertainty into effective user clarification interactions, which is the new core research question in this field.