A page the tool already finds has changed
A page the tool already returned is later updated.
What this scenario means
This scenario checks whether a tool keeps pace with a page it already returned when that page changes. A valid before-probe must show the tool already returned the page. After the change, the after-probe should show the updated live version rather than a stale copy. If the before-probe does not prove prior return, the run is void.
What we evaluate
- Whether the tool already returned the page before the change was introduced.
- Whether the after-probe shows the updated live version of that same page.
- Whether the tool avoids serving a stale copy after the page changes.
- Whether the run is void when the before-probe does not establish prior return.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Freshness
Does what it returns match the web as it is now — new pages found, changed pages not served stale, deleted pages not still handed to you? Three independent failure directions; a tool can be good at one and bad at another. Every tool must be tested against the same page or event. Every run is probe-before, the world changes, probe-after; a run whose before-probe comes out the wrong way is VOID, not failed.
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
Web Search for AI
Which web search or answer API finds the live-web material that actually supports an answer, keeps up when that material changes, and -- where it writes the answer itself -- says only what its own sources support?