Scenario in benchmark · Quiz Generation · Version 1
A cleanly scanned document
A document is scanned and image-only, so its visible text must be recovered without relying on a text layer.
What we check
- Whether the visible text from the scanned page is recovered accurately.
- Whether the page is not silently skipped or treated as empty because no text layer exists.
- Whether the output stays faithful to the visible content without inventing text.
- Whether scanned pages are handled as real input, not hidden by omission.
Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
Edcafe AIFailed1 of 1 test case failedProduced no quiz questions from the scanned chapter.View Result →
It reported that it could not extract text from the PDF.
0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
QuestionWellFailed1 of 1 test case failedRead the image-only scan of a science chapter faithfully, but one of the 20 questions it wrote has two correct answers.View Result →
Asked the total current leaving a battery that pushes 0.60 A through two parallel lamps, question 13 keys exactly 0.60 A, while another option, the sum of both branch currents, is equally correct.
0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
PDF to QuizPassed1 of 1 test case passedGenerated valid questions and correct answer information from the image-only chapter, without omitting material for lack of a text layer.View Result →
It reproduced the chapter’s 0.20 A and 0.14 A readings and used its ammeter, fuse and parallel-circuit material accurately.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
QuizgeckoPassed1 of 1 test case passedPassed this clean-scan quiz-generation scenario.View Result →
It read an image-only PDF with no text layer and generated keyed questions that matched figure-only circuit symbols, table values, and other chapter facts.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
StudyglenPassed1 of 1 test case passedGenerated a quiz from the scanned chapter without materially omitting or misreading its content.View Result →
For example, its lamp-failure question gave the chapter’s answer: the working lamp stays lit at the same brightness.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
No published Result yet (5): DoctoQuiz, GPT-5.6 terra, Knowt, Quizizz, QuizRise.
No published Result does not tell you whether a tool has been tested.Test design
- Pinned test cases
- 1
- Disclosure
- 0 public · 1 withheld
- Capabilities exercised here
- Quiz Generation
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →