Scenario in benchmark · Converting a complex PDF into clean Markdown with an open-source library · Version 1

A simple, clearly formatted table

A simple, clearly formatted table is provided for processing.

Capability: Table Extraction1 pinned test case per toolNeutral global definition →

What we check

  1. Whether the rows and columns remain aligned after processing.
  2. Whether each value stays under the correct header.
  3. Whether the output still reads as a table rather than a flattened or split block of text.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Passed · 3
DoclingPassed1 of 1 test case passedKept the simple table intact in Markdown.View Result →

The output has one four-column table with eight data rows, and every value stayed under the right header.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

LiteParsePassed1 of 1 test case passedKept the simple table structured correctly in Markdown.View Result →

A cell-by-cell comparison showed all four headers, eight rows, and thirty-two values in place, and the only other visible changes were outside the table.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

PyMuPDF4LLMPassed1 of 1 test case passedKept the simple table intact as a Markdown table.View Result →

The source and output both show one header row, eight data rows, and all 36 cells in the same order, with nothing merged, split, reordered, or dropped.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (7): doc2mark, GPT-5.6 terra, Marker, MarkItDown, MinerU, olmOCR, PaddleOCR-VL.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
1 public · 0 withheld
Capabilities exercised here
Table Extraction

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

A simple, clearly formatted table in Converting a complex PDF into clean Markdown with an open-source library | AI Demos