Scenario in benchmark · Quiz Generation · Version 1

One topic is weaker than the others

Two otherwise equivalent learner histories differ only in which topic is weaker.

Capability: Personalized Practice1 pinned test case per toolNeutral global definition →

What we check

  1. Whether the practice focus changes when the performance history changes.
  2. Whether each learner is directed toward the topic they were weaker on.
  3. Whether the system avoids giving both histories the same fixed focus despite opposite strengths and weaknesses.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Failed · 4
Edcafe AIFailed1 of 1 test case failedDid not turn opposite learner histories into personalized practice.View Result →

It correctly identified opposite weak topics for the two learners, but the finished result cards only offered Restart and the report surface had no practice control.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

PDF to QuizFailed1 of 1 test case failedIdentified parallel circuits and meter placement as weak areas, then recommended the full original quiz rather than topic-directed practice.View Result →

Its 47-question practice pool still included questions the learner had already answered correctly.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

QuestionWellFailed1 of 1 test case failedDid not personalize practice from learners' stored performance histories.View Result →

Its Teaching Assistant first asked for results to be shared, then said it could not access the assignment or class results.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

StudyglenFailed1 of 1 test case failedDid not focus practice on either learner’s weaker topic.View Result →

Learner A received a whole-chapter deck, while Learner B received a mixed deck that included ammeter and voltmeter cards.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Passed · 1
QuizgeckoPassed1 of 1 test case passedFocused practice on the weaker topic for each learner.View Result →

Lavi's review card and first question point to parallel-circuit practice, while Anshika's card and review questions point to series-circuit practice; one Anshika item is shown wrong.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (5): DoctoQuiz, GPT-5.6 terra, Knowt, Quizizz, QuizRise.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Personalized Practice

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

One topic is weaker than the others in Quiz Generation | AI Demos