Scenario definition · S102
The learner answers some fixed-answer questions correctly and others incorrectly
A fixed-answer quiz attempt contains both correct and incorrect responses.
What this scenario means
This situation checks whether grading can handle mixed outcomes within one attempt. A good grader matches each response to the answer key, recognises which answers are correct and which are not, and reflects that mix accurately.
What we evaluate
- Whether each response is checked against the correct answer key.
- Whether correct and incorrect responses are distinguished within the same attempt.
- Whether the grading outcome reflects a mix of right and wrong responses rather than treating them the same.
◌Test-case detail. Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Grading
Determines how correctly a learner answered, including fixed-answer scoring, semantically equivalent wording, and partial understanding
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
Quiz Generation
Which AI quiz tool turns study material into valid quizzes, and then grades, explains and tracks what the learner actually understood?