Scenario in benchmark · Quiz Generation · Version 1

The learner answers some fixed-answer questions correctly and others incorrectly

A fixed-answer quiz attempt contains both correct and incorrect responses.

Capability: Grading1 pinned test case per toolNeutral global definition →

What we check

  1. Whether each response is checked against the correct answer key.
  2. Whether correct and incorrect responses are distinguished within the same attempt.
  3. Whether the grading outcome reflects a mix of right and wrong responses rather than treating them the same.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Passed · 5
Edcafe AIPassed1 of 1 test case passedAccurately graded the mixed-answer attempt, marking six answers correct and four incorrect.View Result →

The learner result and teacher report both gave 60% (6/10), the total required by the answer key.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

PDF to QuizPassed1 of 1 test case passedPasses this mixed-answer grading case.View Result →

Live marking, the review cards, and the final tally show correct answers marked green and wrong answers marked red, ending at 7 correct, 7 incorrect, and 50%.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

QuestionWellPassed1 of 1 test case passedCorrectly distinguished right and wrong fixed answers and produced a consistent total.View Result →

It marked 9 answers correct and 11 incorrect, awarded no partial credit, and scored the attempt 9/20.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

QuizgeckoPassed1 of 1 test case passedGraded the mixed fixed-answer attempt correctly.View Result →

Live question views and the results table distinguished correct from incorrect answers, and the final score matched the row-by-row judgments at 17 of 53 correct, or 32%.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

StudyglenPassed1 of 1 test case passedCorrectly graded the learner’s mix of correct and incorrect fixed answers.View Result →

It reported 10 correct and 10 incorrect answers, giving 50% for 10 correct out of 20.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (5): DoctoQuiz, GPT-5.6 terra, Knowt, Quizizz, QuizRise.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Grading

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

The learner answers some fixed-answer questions correctly and others incorrectly in Quiz Generation | AI Demos