Scenario in benchmark · Quiz Generation · Version 1

The learner answers correctly in their own words

A response is judged by the meaning it conveys, not by matching the reference wording exactly.

Capability: Grading1 pinned test case per toolNeutral global definition →

What we check

  1. Whether the response is accepted when it expresses all required meaning in different wording.
  2. Whether wording differences alone do not change a correct response into an incorrect one.
  3. Whether a response that omits required meaning is not treated as correct.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Passed · 5
Edcafe AIPassed1 of 1 test case passedPasses this own-words meaning-grading case.View Result →

It marked Learner D Excellent Understanding Demonstrated, and the report showed a 0% average across four submissions while three learners were labelled excellent, so the average does not reflect this answer's wording.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

PDF to QuizPassed1 of 1 test case passedMarked the learner’s answer fully correct.View Result →

The review screen gave it 10/10 for a response that kept the required meaning in different wording, and the brief pre-grading incorrect counter did not affect the score.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

QuestionWellPassed1 of 1 test case passedMarked a learner's answer fully correct even though it was worded differently from the quiz's model answer.View Result →

It returned an unqualified “Correct!”, and nothing on the result screen marked the answer down or flagged the wording.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

QuizgeckoPassed1 of 1 test case passedMarks a correct written answer in the learner's own words as correct.View Result →

The graded screen shows 'Correct' with +15 XP, and the explanation credits the response's meaning rather than matching its phrasing; the related-material panel points to the same electric-circuits chapter as the input file.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

StudyglenPassed1 of 1 test case passedGave full credit to a correct answer written in the learner’s own words.View Result →

For the circuit question, the 98% score measured similarity, not reduced credit, and the feedback recognized the required meaning.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (5): DoctoQuiz, GPT-5.6 terra, Knowt, Quizizz, QuizRise.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Grading

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

The learner answers correctly in their own words in Quiz Generation | AI Demos