Scenario in benchmark · Converting a complex PDF into clean Markdown with an open-source library · Version 1
A document mixing digital and scanned pages
A document contains both digital text pages and scanned image-only pages.
What we check
- Whether the scanned page inside a mixed document is processed rather than silently skipped.
- Whether page-by-page handling keeps content from both digital text pages and scanned image-only pages.
- Whether the system treats the document as mixed, not as a fully digital file with no special handling needed.
Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
DoclingPassed1 of 1 test case passedIncluded the scanned worksheet's text and values in the Markdown output instead of skipping the page.View Result →
The worksheet stayed in its original position between the surrounding pages.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
LiteParsePassed1 of 1 test case passedKept the scanned page in the mixed PDF instead of skipping it.View Result →
The Markdown places the scanned page between the digital pages and recovers its text, though labels and values come out as separate groups; OCR completed.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
PyMuPDF4LLMPassed1 of 1 test case passedPassed this mixed-PDF scenario: the scanned page was OCR'd and kept between the surrounding digital pages.View Result →
The worksheet values were recovered, and one OCR sentence is visibly garbled without the page being skipped.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
No published Result yet (7): doc2mark, GPT-5.6 terra, Marker, MarkItDown, MinerU, olmOCR, PaddleOCR-VL.
No published Result does not tell you whether a tool has been tested.Test design
- Pinned test cases
- 1
- Disclosure
- 1 public · 0 withheld
- Capabilities exercised here
- Scanned Document OCR
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →