Scenario definition · S7
Information from earlier in the conversation is needed later
A later exchange depends on information introduced earlier in the same conversation or session.
What this scenario means
This tests whether the agent keeps relevant context and reuses it when a later step only makes sense if earlier information is still available. A good agent links the later turn to the earlier one, carries the thread forward, and uses the stored context without forcing repetition.
What we evaluate
- Whether the agent uses information provided earlier when answering or acting later.
- Whether the agent avoids asking for information again when it is already available in the conversation history.
- Whether the agent preserves the earlier context when the same thread continues across turns or sessions.
◌Test-case detail. Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Conversation continuity
Keeps track of what has already been said — across turns and across sessions
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
AI Customer Support Chatbots
Automate customer support using an AI chatbot / AI customer support agent — which agents handle real customer support conversations for a business best?