Scenario in benchmark · Web Search for AI · Version 1
The retrieved sources disagree with each other
When retrieved sources give different values for the same fact.
What we check
- Whether the answer recognises that the retrieved sources conflict on the same fact.
- Whether the answer avoids presenting one source’s value as the fact.
- Whether the answer conveys that the sources disagree instead of choosing a side without support.
Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
Brave Search APIPassed1 of 1 test case passedSurfaced the disagreement rather than choosing one value.View Result →
Its answer gave a range for the 2025 global AI market and attributed nine estimates to their respective firms.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
SerpAPIPassed1 of 1 test case passedSurfaced disagreement between reports instead of choosing one definitive market-size figure.View Result →
Its Google AI Overview answered the 2025 global artificial intelligence market question with several estimates tied to report and market scope.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
No published Result yet (10): Exa, Firecrawl, GPT-5.6 terra, Linkup, OpenAI web search, Perplexity Sonar, Serper, Tavily, Valyu, You.com.
No published Result does not tell you whether a tool has been tested.Test design
- Pinned test cases
- 1
- Disclosure
- 0 public · 1 withheld
- Capabilities exercised here
- Answer Generation
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →