The request uses different wording than the source
The request uses different wording from the source, but still refers to the same underlying fact.
What this scenario means
This scenario tests whether a system can match meaning across a wording change. It is hard because exact-term matching can miss the right source when the request does not reuse the source’s own phrasing. A good system still recognises the paraphrase, finds the relevant source, and answers from that source’s meaning rather than from copied words.
What we evaluate
- Whether the system recognises the same fact when the request uses alternative wording.
- Whether it still uses the relevant source even when the request does not repeat the source’s terms.
- Whether it avoids treating a paraphrase mismatch as if the fact were absent.
- Whether it answers by meaning rather than by literal phrase matching.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Knowledge-grounded answering
Answers customer questions from the company's own knowledge base — not from what the model assumes
Retrieval
The passage that answers the question comes back when the documents hold it — and the absence is discoverable when they don't. Judged against the documents: did the supporting passage come back?
Web Retrieval
When the web holds material that supports the answer, does it come back? Judged against the WEB, not against what the tool retrieved: if the answer was out there and search missed it, that is a retrieval failure. The developer does not know which URL holds the answer -- the product has to find it, which is the whole category.
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
Managed RAG
Also uses this scenario.
AI Customer Support Chatbots
Automate customer support using an AI chatbot / AI customer support agent — which agents handle real customer support conversations for a business best?
Web Search for AI
Also uses this scenario.