Scenario in benchmark · Converting a complex PDF into clean Markdown with an open-source library · Version 1
A page laid out in multiple columns
A PDF page laid out in multiple columns.
What we check
- Whether text is emitted in the natural reading order down each column.
- Whether content from different columns is kept separate rather than interleaved.
- Whether the page's reading sequence is preserved even when all text is present.
Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
LiteParseFailed1 of 1 test case failedFailed to keep the page’s text in natural column order.View Result →
The output kept all source text, but three sentence continuations appeared before the text they completed, with unrelated material between them.
0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
DoclingPassed1 of 1 test case passedPreserved the natural reading order in a multi-column document.View Result →
Across three two-column pages, it read the left column before the right without mixing their text, and kept the sentences spanning column breaks together.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
No published Result yet (8): doc2mark, GPT-5.6 terra, Marker, MarkItDown, MinerU, olmOCR, PaddleOCR-VL, PyMuPDF4LLM.
No published Result does not tell you whether a tool has been tested.Test design
- Pinned test cases
- 1
- Disclosure
- 1 public · 0 withheld
- Capabilities exercised here
- Reading Order & Layout
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →