Scenario definition · S18
Customer communicates in another supported language
A support question is asked in a supported non-English language.
What this scenario means
This scenario tests whether the agent can understand and answer a support request when the language changes but the task does not. A good agent still gives the grounded answer, in that language, without losing context or requiring the request to be restated in English. It checks multilingual support as part of normal customer support, not as a separate language exercise.
What we evaluate
- Whether the agent understands the request when it is written in a supported non-English language.
- Whether the agent answers in the same supported language.
- Whether the response stays grounded in the benchmark’s approved source of truth.
- Whether the agent preserves the support meaning and does not force a language switch before responding.
◌Test-case detail. Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Multilingual support
Handles customers who don't write in English
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
AI Customer Support Chatbots
Automate customer support using an AI chatbot / AI customer support agent — which agents handle real customer support conversations for a business best?