Scenario definition · S23
Agent actions/tool calls can be inspected
A prior conversation contains an agent tool or action call that can be inspected.
What this scenario means
This checks whether a product preserves enough conversational trace to review what the agent did, not just what it said. A good product makes the relevant action visible and attributable in the earlier conversation, so later review can confirm the call happened and understand it in context.
What we evaluate
- Whether the earlier conversation can be found and opened.
- Whether an agent action or tool call in that conversation is visible for inspection.
- Whether the inspection shows that the call occurred, not only the final reply.
- Whether the call can be linked back to the conversation where it happened.
◌Test-case detail. Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Analytics and observability
Lets the operator see what the agent did, inspect any conversation, and trust the reported numbers
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
AI Customer Support Chatbots
Automate customer support using an AI chatbot / AI customer support agent — which agents handle real customer support conversations for a business best?