Scenario definition · S98

The user asks the quiz to test a particular learning goal

The user asks for a quiz to test a stated learning goal that requires explaining an idea or relationship.

What this scenario means

This scenario checks whether the product can turn a stated goal into an assessment that asks for explanation, not just recall. A good system recognises the required relationship or idea, builds the quiz around that target, and provides answer or evaluation criteria that cover the elements needed for a complete explanation.

What we evaluate

  • Whether the assessment is centred on explaining the specified idea or relationship rather than only recognising facts.
  • Whether the product provides answer or evaluation criteria that cover the required elements of a complete explanation.
  • Whether the response can be judged against those required elements in a way that matches the stated learning goal.
◌Test-case detail. Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

Capabilities this scenario exercises

A scenario may exercise one or more capabilities.

Capability

Quiz Generation

Creates valid quizzes from supplied material that test the intended knowledge and include sufficient information for answers to be evaluated

Benchmarks that use this scenario

A scenario has global identity and may be reused across benchmarks.

The user asks the quiz to test a particular learning goal — Scenario definition | AI Demos