Scenario definition · S45

The agent offers what to ask next

After an answer, the product suggests follow-up questions grounded in the just-completed analysis.

What this scenario means

This scenario tests whether the product can move from answering to useful next-step guidance without losing the thread of the conversation. A good agent does not give generic prompts; it offers follow-up questions that fit the answer it has just given and reflect the same line of analysis. The hard part is staying relevant and concrete while being proactive, so the suggestions feel earned rather than bolted on.

What we evaluate

  • Whether it offers follow-up questions after answering.
  • Whether the suggestions are grounded in the answer just given and the analysis behind it.
  • Whether the follow-up suggestions are specific to the conversation rather than generic prompts.
◌Test-case detail. Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

Capabilities this scenario exercises

A scenario may exercise one or more capabilities.

Capability

Conversational Interaction

Holds a working session rather than a series of isolated queries: context carried, questions refined, direction changed, ambiguity raised.

Benchmarks that use this scenario

A scenario has global identity and may be reused across benchmarks.

The agent offers what to ask next — Scenario definition | AI Demos