Scenario in benchmark · Converting a complex PDF into clean Markdown with an open-source library · Version 1

Footnotes at the bottom of the page

A footnote stays separate from the main text and remains linked to the content it annotates.

Capability: Heading & Section Structure1 pinned test case per toolNeutral global definition →

What we check

  1. Whether the footnote is kept outside the main body flow.
  2. Whether the footnote remains present.
  3. Whether the footnote stays associated with the content it annotates.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Failed · 2
DoclingFailed1 of 1 test case failedDid not preserve the relationship between footnotes and their references in Markdown.View Result →

It included all 13 footnote texts, but placed them in the body flow without associating them with their markers.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

LiteParseFailed1 of 1 test case failedFailed to keep the footnote both present and attached while leaving it out of the body flow.View Result →

The output kept footnotes in page-end blocks, but footnote 5 disappeared while its marker stayed in the body; other footnotes were also merged or detached.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Passed · 3
MarkItDownPassed1 of 1 test case passedKept all 13 footnotes separate from the body text and associated with their in-text references.View Result →

The notes followed their pages’ body text, with all 13 reference numbers preserved.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

MinerUPassed1 of 1 test case passedKept bottom-of-page footnotes out of the body text and associated with their numbered references.View Result →

All 13 footnotes remained in order, with their source wording unchanged.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

PyMuPDF4LLMPassed1 of 1 test case passedPreserved the PDF’s footnotes without mixing them into body paragraphs.View Result →

All 13 notes remained present and associated with the corresponding numbered references.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (5): doc2mark, GPT-5.6 terra, Marker, olmOCR, PaddleOCR-VL.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
1 public · 0 withheld
Capabilities exercised here
Heading & Section Structure

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

Footnotes at the bottom of the page in Converting a complex PDF into clean Markdown with an open-source library | AI Demos