Scenario in benchmark · Web Search for AI · Version 1

The retrieved sources disagree with each other

When retrieved sources give different values for the same fact.

Capability: Answer Generation1 pinned test case per toolNeutral global definition →

What we check

  1. Whether the answer recognises that the retrieved sources conflict on the same fact.
  2. Whether the answer avoids presenting one source’s value as the fact.
  3. Whether the answer conveys that the sources disagree instead of choosing a side without support.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Passed · 2
Brave Search APIPassed1 of 1 test case passedSurfaced the disagreement rather than choosing one value.View Result →

Its answer gave a range for the 2025 global AI market and attributed nine estimates to their respective firms.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

SerpAPIPassed1 of 1 test case passedSurfaced disagreement between reports instead of choosing one definitive market-size figure.View Result →

Its Google AI Overview answered the 2025 global artificial intelligence market question with several estimates tied to report and market scope.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (10): Exa, Firecrawl, GPT-5.6 terra, Linkup, OpenAI web search, Perplexity Sonar, Serper, Tavily, Valyu, You.com.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Answer Generation

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

The retrieved sources disagree with each other in Web Search for AI | AI Demos