Scenario in benchmark · Quiz Generation · Version 1

Important information appears in both text and a diagram

The material contains essential information in written text and a diagram, and both parts matter.

Capability: Quiz Generation1 pinned test case per toolNeutral global definition →

What we check

  1. Whether the quiz assesses core information from the written text and the diagram.
  2. Whether the questions stay supported when the needed content is split across both forms.
  3. Whether essential information shown only in one form is not omitted from coverage.
  4. Whether the answer or evaluation information aligns with both parts of the material.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Failed · 6
DoctoQuizFailed1 of 1 test case failedGenerated a quiz that covered text-based concepts but omitted the diagram-only core idea.View Result →

None of its ten questions assessed the Fenmoor Ditch food web’s feeding relationships.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Edcafe AIFailed1 of 1 test case failedCovered text ideas but omitted core information taught only through the diagram.View Result →

Its 30-question quiz never asked who eats whom in the food web.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

PDF to QuizFailed1 of 1 test case failedGenerated 17 questions covering text-based ideas but left the diagram-only food web unassessed.View Result →

The questions addressed energy transfer, decomposers and other prose-based topics, with answers that agreed with the chapter.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

QuestionWellFailed1 of 1 test case failedWrote 30 questions from the food-web chapter, including items on animals that appear only in its diagram, but some keys disagree with the chapter.View Result →

The one question on the diagram keys the stickleback as getting food from kingfishers, the reverse of the figure, and a question on energy keys new leaves where the chapter says energy leaves the chain as heat.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

QuizgeckoFailed1 of 1 test case failedFailed this scenario: it returned a 39-question quiz that covered the chapter’s text-based ideas, but it missed the diagram-only food web.View Result →

The food-web questions stayed abstract and never named the diagram’s unique organisms.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

StudyglenFailed1 of 1 test case failedProduced a quiz covering prose-based ideas but omitted the diagram-only feeding relationships.View Result →

Its answer keys agreed with the chapter, yet none of its questions assessed who eats whom in the Fenmoor Ditch food web.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (4): GPT-5.6 terra, Knowt, Quizizz, QuizRise.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Quiz Generation

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

Important information appears in both text and a diagram in Quiz Generation | AI Demos