Scenario in benchmark · Quiz Generation · Version 1

The user turns study material into a quiz

Study material is turned into a quiz.

Capability: Quiz Generation2 pinned test cases per toolNeutral global definition →

What we check

  1. Whether the quiz is grounded in the supplied study material rather than outside knowledge.
  2. Whether the questions are answerable from the material and focus on the core ideas, not just supporting detail.
  3. Whether the answer or evaluation information matches the source material.
  4. Whether the quiz includes enough information for responses to be judged against the material.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Failed · 3
DoctoQuizFailed2 of 2 test cases failedProduced correctly keyed multiple-choice quizzes in English and Hindi, but both omitted core material while repeating other topics.View Result →

Each ten-question quiz left out home circuits from a chapter with six core sections; the Hindi quiz kept every question, option and answer in Hindi.

0 Pass · 2 Fail · 0 Not gradable · 2 of 2 assessed · 2 of 2 gradable

PDF to QuizFailed1 of 2 test cases passed · 1 failedPassed the English chapter quiz task and failed the Hindi one.View Result →

In the English run, it produced multiple-choice questions with correct keys and later chapter coverage added in the second run. In the Hindi run, the language stayed Hindi, but two checked keys were wrong and one item had no correct option.

1 Pass · 1 Fail · 0 Not gradable · 2 of 2 assessed · 2 of 2 gradable

QuestionWellFailed2 of 2 test cases failedProduced multiple-choice quizzes from both chapters, with answer-quality problems in each.View Result →

One quiz had a defensible unkeyed option and repeated a core idea. The Hindi quiz used Hindi throughout, but keyed equal branch currents despite the chapter’s 0.30 A and 0.18 A examples; another item’s statement did not fit its answer choices.

0 Pass · 2 Fail · 0 Not gradable · 2 of 2 assessed · 2 of 2 gradable

Passed · 1
QuizgeckoPassed2 of 2 test cases passedGenerated English and Hindi multiple-choice quizzes covering all six core ideas, with correct keys and clearly incorrect alternatives.View Result →

Both displayed formatting code around numbers, and the Hindi content stayed in Hindi. Repetition involved one confirmed pair and two further reported pairs in English, and one pair in Hindi.

2 Pass · 0 Fail · 0 Not gradable · 2 of 2 assessed · 2 of 2 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (6): Edcafe AI, GPT-5.6 terra, Knowt, Quizizz, QuizRise, Studyglen.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
2
Disclosure
0 public · 2 withheld
Capabilities exercised here
Quiz Generation

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

The user turns study material into a quiz in Quiz Generation | AI Demos