Scenario definition · S20
Configured instruction changes agent behavior
A configured instruction is applied to the agent and changes how it responds.
What this scenario means
This tests whether the agent takes applied business configuration into account instead of falling back to its default behaviour. A good agent changes its response when the instruction applies, and the change should be visible in what it says or does.
What we evaluate
- Whether the agent's response reflects the applied instruction.
- Whether the response changes from its default behaviour when the instruction is in force.
- Whether the agent follows the configured instruction on the relevant request.
- Whether the configured instruction is observable in the resulting behaviour.
◌Test-case detail. Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Agent configuration and control
Behaves the way the business configures it to — instructions applied, restrictions respected
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
AI Customer Support Chatbots
Automate customer support using an AI chatbot / AI customer support agent — which agents handle real customer support conversations for a business best?