Scenario in benchmark · Quiz Generation · Version 1

The user asks the quiz to test a particular learning goal

The user asks for a quiz to test a stated learning goal that requires explaining an idea or relationship.

Capability: Quiz Generation1 pinned test case per toolNeutral global definition →

What we check

  1. Whether the assessment is centred on explaining the specified idea or relationship rather than only recognising facts.
  2. Whether the product provides answer or evaluation criteria that cover the required elements of a complete explanation.
  3. Whether the response can be judged against those required elements in a way that matches the stated learning goal.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Failed · 2
DoctoQuizFailed1 of 1 test case failedReturned ten multiple-choice questions when asked for one open-ended question testing an explanation goal.View Result →

It provided no model answer or key points.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

StudyglenFailed1 of 1 test case failedGenerated no question requiring an explanation of why other parallel lamps stay lit when one fails.View Result →

Question 15 asked what happens, while Question 1 asked broadly why household lamps use parallel wiring.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Passed · 2
PDF to QuizPassed1 of 1 test case passedGenerated a question on the learning goal with enough answer guidance to judge a response.View Result →

It asks learners to explain lamp failures in series and parallel circuits, and supplies an accurate model answer.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

QuizgeckoPassed1 of 1 test case passedPassed this learning-goal scenario.View Result →

The uploaded electric-circuits chapter is shown, along with settings and generated questions that ask for explanation rather than recall; the visible model answers cover both required ideas: current is not used up or shared out, and a series loop has one path.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (6): Edcafe AI, GPT-5.6 terra, Knowt, QuestionWell, Quizizz, QuizRise.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Quiz Generation

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

The user asks the quiz to test a particular learning goal in Quiz Generation | AI Demos