The retrieved material doesn't support an answer
Retrieved material is related to the topic but does not support an answer.
What this scenario means
This scenario checks whether a system can tell that relevant-looking material still fails to justify a conclusion. It is hard because the retrieved content may seem on-topic, so a good agent must not turn partial relevance into a grounded answer. Instead, it should withhold the answer or clearly indicate that the material does not support one.
What we evaluate
- Whether the response withholds a direct answer when the retrieved material is insufficient.
- Whether the response avoids presenting related material as if it supports the conclusion.
- Whether the response indicates that the retrieved material does not answer the question, rather than overclaiming confidence.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Answer Generation
Where it writes the answer, does it say only what its own sources support -- declining when they don't, flagging when they disagree, and attributing what it does claim? Judged against what THAT TOOL retrieved, never against the web. Out of scope for search-only products, which is not a zero and never drags them down a ranking. A citation that correctly points at the source of a wrong answer is a CORRECT citation.
Grounded Generation
The answer is faithful to the material the system itself retrieved, including declining when that material doesn't support one.
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
Managed RAG
Which managed RAG service turns a folder of private documents into a query API that finds the right passage, answers only from what it retrieved, cites the source that actually supports the claim — and keeps up when the source changes?
Web Search for AI
Also uses this scenario.