The user sets a standing rule for all future answers
A user sets a standing rule that should continue to apply to later, unrelated answers unless it is overridden.
What this scenario means
This scenario tests whether the agent treats a user instruction as persistent context rather than a one-off detail. A good agent carries the rule into later questions even when the topic changes, so later answers still follow the established instruction. It checks that the agent keeps that rule active across the conversation, not just for the next reply.
What we evaluate
- Whether the agent applies the standing rule to later, unrelated questions without the user repeating it.
- Whether the rule still affects answers after the conversation moves on to a different topic.
- Whether the agent changes its behaviour only when the user overrides or replaces the rule.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Training
Whether supplying company-specific information measurably improves future behaviour and generalises beyond the example given.
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
AI Database Agents
Which AI database agent answers business questions about a live relational database correctly — in the company's own terms, and honestly when the data cannot answer?