Scenario in benchmark · Converting a complex PDF into clean Markdown with a hosted API · Version 1
Footnotes at the bottom of the page
A footnote stays separate from the main text and remains linked to the content it annotates.
What we check
- Whether the footnote is kept outside the main body flow.
- Whether the footnote remains present.
- Whether the footnote stays associated with the content it annotates.
Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
PDF VectorFailed1 of 1 test case failedKept all 13 footnotes but did not keep them out of the body or link them to their references.View Result →
In the returned Markdown, each page's notes come before that page's text in one undivided block, and the reference marks are plain digits attached to the preceding word. Choosing higher model tiers by hand moved the notes to the end of each page, but the marks stayed unlinked.
0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Adobe PDF Extract APIPassed1 of 1 test case passedPreserved the footnotes and their links to the text they annotate.View Result →
The structured output kept all 13 numbered notes outside the body flow, with their associations retained by number.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Landing AIPassed1 of 1 test case passedKept footnotes outside body paragraphs while preserving their references.View Result →
It retained all thirteen notes intact, grouping each page’s notes after its body text.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
LlamaParsePassed1 of 1 test case passedKept all 13 footnotes out of the body text and tied each one to its reference.View Result →
Each page's notes come after that page's body, every note on its own line, and every reference mark in the body is a superscript that matches a note, with every note matching a mark.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Mistral AIPassed1 of 1 test case passedKeeps footnotes out of the body flow and links each marker to its note.View Result →
The Markdown output places 13 footnotes after the page bodies in source order, and the 26 marker occurrences each appear once inline and once at their notes, so the association is preserved.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Nutrient.ioPassed1 of 1 test case passedPassed this footnote-handling case.View Result →
The Markdown kept all thirteen notes separate from body paragraphs across all four pages, with matching digits still in the body. The free-tier run returned 10,842 characters of Markdown.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Upstage AIPassed1 of 1 test case passedKept footnotes out of the body flow and attached them to their references in this test.View Result →
The source PDF’s thirteen notes all appear in the Markdown, set as separate blocks after the relevant page body. Each note keeps its leading number, and the matching reference digit remains in the body text. One reference digit has a space before it.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
No published Result yet (6): Extend AI, Firecrawl, GPT-5.6 terra, Reducto, Tensorlake, Unstructured.
No published Result does not tell you whether a tool has been tested.Test design
- Pinned test cases
- 1
- Disclosure
- 1 public · 0 withheld
- Capabilities exercised here
- Heading & Section Structure
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →