Scenario in benchmark · Quiz Generation · Version 1

The teacher checks what the class struggled with

A teacher reviews class performance to see which topic or question area the learners struggled with.

Capability: Class Reports1 pinned test case per toolNeutral global definition →

What we check

  1. Whether the report matches the underlying learner attempts and completion records.
  2. Whether the weaker topic or question area is shown as performing worse than the stronger area.
  3. Whether the performance difference stays visible instead of being erased or inverted.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Failed · 2
PDF to QuizFailed1 of 1 test case failedDid not provide a class report showing what learners struggled with.View Result →

A completed 47-question attempt scored 11%—5 correct and 42 incorrect—yet the teacher's five analytics panels remained empty after a reload.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

QuizgeckoFailed1 of 1 test case failedDoes not surface the class's shared weakness in the teacher report.View Result →

The class view shows 28 completed sessions and a 16% average, while the difficulty/ease panels have blank question rows; both learners' own results are 81% on 53 questions.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Passed · 2
Edcafe AIPassed1 of 1 test case passedMade the class's shared weakness in parallel circuits visible alongside stronger performance on series circuits.View Result →

Answer-level feedback flagged off-topic parallel responses, and learner summaries pointed to parallel circuits as the area to improve.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

QuestionWellPassed1 of 1 test case passedShowed the class’s shared weakness in electrical measurement without hiding it behind overall scores.View Result →

Both learners’ ammeter outcome was red while circuit basics were green, making the stronger and weaker areas visible together.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (6): DoctoQuiz, GPT-5.6 terra, Knowt, Quizizz, QuizRise, Studyglen.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Class Reports

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

The teacher checks what the class struggled with in Quiz Generation | AI Demos