Scenario in benchmark · Converting a complex PDF into clean Markdown with an open-source library · Version 1

A document with figures and captions

A document contains a figure with its caption attached.

Capability: Figures & Charts1 pinned test case per toolNeutral global definition →

What we check

  1. Whether the figure is preserved rather than omitted or replaced by surrounding prose.
  2. Whether the figure remains positioned in the output at the point where it belongs in the document flow.
  3. Whether the caption stays attached to the figure and remains associated with it.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Failed · 3
DoclingFailed1 of 1 test case failedDocling’s default Markdown export omitted the figure images.View Result →

It kept the transect drawing’s caption and the bar chart’s title in their reading positions, with image placeholders rather than embedded pictures or image references.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

LiteParseFailed1 of 1 test case failedDid not preserve the captioned figure in Markdown.View Result →

It kept the caption unchanged and the reference in the right position, but embedded no image and wrote no image file.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

PyMuPDF4LLMFailed1 of 1 test case failedPyMuPDF4LLM’s default Markdown conversion omitted the figure image while keeping its caption in the right place.View Result →

A block of the figure’s text labels appeared where the image belonged, without an image or image link.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (7): doc2mark, GPT-5.6 terra, Marker, MarkItDown, MinerU, olmOCR, PaddleOCR-VL.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
1 public · 0 withheld
Capabilities exercised here
Figures & Charts

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

A document with figures and captions in Converting a complex PDF into clean Markdown with an open-source library | AI Demos