A new page has just gone live
A page has just gone live, and the tool should be able to find it.
What this scenario means
This tests whether freshness-sensitive search can surface newly published material instead of missing it as if it were old or absent. A good agent first confirms the page was not already findable, then finds the new page or answer once it exists, rather than treating an earlier miss as final.
What we evaluate
- Whether the required before-check confirms the page was not already findable.
- Whether the tool finds the newly published page once it is available.
- Whether a run that starts with the page already findable is treated as invalid, not as a freshness success.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Freshness
Does what it returns match the web as it is now — new pages found, changed pages not served stale, deleted pages not still handed to you? Three independent failure directions; a tool can be good at one and bad at another. Every tool must be tested against the same page or event. Every run is probe-before, the world changes, probe-after; a run whose before-probe comes out the wrong way is VOID, not failed.
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
Web Search for AI
Which web search or answer API finds the live-web material that actually supports an answer, keeps up when that material changes, and -- where it writes the answer itself -- says only what its own sources support?