Scenario in benchmark · Quiz Generation · Version 1
The teacher checks what the class struggled with
A teacher reviews class performance to see which topic or question area the learners struggled with.
What we check
- Whether the report matches the underlying learner attempts and completion records.
- Whether the weaker topic or question area is shown as performing worse than the stronger area.
- Whether the performance difference stays visible instead of being erased or inverted.
Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
PDF to QuizFailed1 of 1 test case failedDid not provide a class report showing what learners struggled with.View Result →
A completed 47-question attempt scored 11%—5 correct and 42 incorrect—yet the teacher's five analytics panels remained empty after a reload.
0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
QuizgeckoFailed1 of 1 test case failedDoes not surface the class's shared weakness in the teacher report.View Result →
The class view shows 28 completed sessions and a 16% average, while the difficulty/ease panels have blank question rows; both learners' own results are 81% on 53 questions.
0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Edcafe AIPassed1 of 1 test case passedMade the class's shared weakness in parallel circuits visible alongside stronger performance on series circuits.View Result →
Answer-level feedback flagged off-topic parallel responses, and learner summaries pointed to parallel circuits as the area to improve.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
QuestionWellPassed1 of 1 test case passedShowed the class’s shared weakness in electrical measurement without hiding it behind overall scores.View Result →
Both learners’ ammeter outcome was red while circuit basics were green, making the stronger and weaker areas visible together.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
Test design
- Pinned test cases
- 1
- Disclosure
- 0 public · 1 withheld
- Capabilities exercised here
- Class Reports
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →