Customer corrects information given earlier
A later correction supersedes earlier information for the rest of the interaction.
What this scenario means
This scenario checks whether the agent updates its working understanding when a later statement changes an earlier detail. A good agent treats the correction as authoritative, discards the superseded information, and uses only the revised detail in subsequent reasoning and answer selection.
What we evaluate
- Whether the later correction replaces the earlier detail in later turns.
- Whether the response reflects only the corrected information, not a mix of old and new details.
- Whether the agent does not continue reasoning from the superseded detail after the correction.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Conversation continuity
Keeps track of what has already been said — across turns and across sessions
Conversational Interaction
Holds a working session rather than a series of isolated queries: context carried, questions refined, direction changed, ambiguity raised.
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
AI Customer Support Chatbots
Automate customer support using an AI chatbot / AI customer support agent — which agents handle real customer support conversations for a business best?
AI Database Agents
Also uses this scenario.