Scenario definition · S108

The learner checks whether they are improving

A learner reviews progress after earlier and later attempts show improvement in one topic and continued errors in another.

What this scenario means

This scenario tests whether a progress view can reflect change over time without hiding weaker areas. A good system agrees with the recorded attempts, shows the improved topic as doing better than before, keeps the other topic marked as needing work, and avoids claiming complete mastery while weakness remains.

What we evaluate

  • Whether the progress view agrees with the recorded attempts over time.
  • Whether it presents the improved topic more positively than before.
  • Whether it still shows the weaker topic as needing work.
  • Whether it avoids claiming complete mastery while weakness remains.
◌Test-case detail. Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

Capabilities this scenario exercises

A scenario may exercise one or more capabilities.

Capability

Learner Progress

Shows learners how their performance changes over time and which topics still need work

Benchmarks that use this scenario

A scenario has global identity and may be reused across benchmarks.

The learner checks whether they are improving — Scenario definition | AI Demos