Scenario definition · S6
Request is unclear and needs clarification
The request is ambiguous enough that the agent cannot safely choose one interpretation.
What this scenario means
This scenario tests whether the agent can recognise that a request does not yet identify one safe course of action. A good agent pauses instead of guessing, asks for the missing detail, and waits for clarification before acting or giving a final answer.
What we evaluate
- Whether the agent asks a clarifying question before taking action.
- Whether the agent avoids guessing which interpretation is intended.
- Whether the agent does not give a final answer that depends on an unresolved ambiguity.
- Whether the agent can proceed normally once the missing detail is provided.
◌Test-case detail. Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Intent understanding
Works out what the customer actually wants before answering or acting
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
AI Customer Support Chatbots
Automate customer support using an AI chatbot / AI customer support agent — which agents handle real customer support conversations for a business best?