Scenario in benchmark · Web Search for AI · Version 1
The retrieved material supports an answer
The retrieved material contains support for the answer.
What we check
- Whether the retrieved material contains enough support for the claim being made.
- Whether the system can ground its response in the retrieved material rather than outside knowledge.
- Whether the case is treated as a supported baseline, not as an unsupported or missing-answer case.
Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
Perplexity SonarPassed1 of 1 test case passedAnswered with a source-supported limit of 512 MB per file for OpenAI’s Files API.View Result →
Its added note about a possible limit of 5 million tokens per file for retrieval or file search was also supported.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
You.comPassed1 of 1 test case passedAnswered the upload-limit question with the correct figure of 512 MB.View Result →
It cited a user’s forum question; OpenAI’s own Upload file reference stated the same limit and appeared in its results without being cited.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
No published Result yet (10): Brave Search API, Exa, Firecrawl, GPT-5.6 terra, Linkup, OpenAI web search, SerpAPI, Serper, Tavily, Valyu.
No published Result does not tell you whether a tool has been tested.Test design
- Pinned test cases
- 1
- Disclosure
- 0 public · 1 withheld
- Capabilities exercised here
- Answer Generation
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →