Search only these websites
A search request is limited to a permitted set of websites.
What this scenario means
This scenario tests whether a search tool respects source restrictions when it is only allowed to use approved sites. A good agent searches within the allowed set, returns relevant material from those sites when it exists, and does not quietly broaden the search or narrow it so far that it misses available evidence.
What we evaluate
- Whether the tool honours the restriction rather than silently ignoring it.
- Whether the tool returns relevant material from the permitted sites when such material exists.
- Whether the tool avoids bringing in results from outside the permitted set.
- Whether the tool avoids applying the restriction so aggressively that it misses usable allowed-source material.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Web Retrieval
When the web holds material that supports the answer, does it come back? Judged against the WEB, not against what the tool retrieved: if the answer was out there and search missed it, that is a retrieval failure. The developer does not know which URL holds the answer -- the product has to find it, which is the whole category.
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
Web Search for AI
Which web search or answer API finds the live-web material that actually supports an answer, keeps up when that material changes, and -- where it writes the answer itself -- says only what its own sources support?