Scenario in benchmark · Converting a complex PDF into clean Markdown with a hosted API · Version 1
A table that continues across a page break
A table continues across a page break and is still treated as one table.
What we check
- Whether the table is recognised as one continuous table rather than split into separate tables.
- Whether the continuation keeps the table header context intact across the page break.
- Whether rows and columns remain aligned as a single structure across both pages.
Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
Mistral AIDifferent test setHandled one cross-page table case and failed another.View Result →
In the passing case, all 22 rows were recovered across two table blocks, with the repeated page-4 header outside the data and row order preserved. In the failing case, the second page used the first data row as its header, so 25 rows were returned and later rows lost their column labels.
Published on a different test set, so its counts are not compared here.
Upstage AIDifferent test setPassed one page-break table case and failed the other.View Result →
In the pass case, it returned all 22 rows in order as two Markdown tables, with the page-4 part labelled "Table 1 (continued)" under the same headers. In the fail case, it turned the first continuation row into the second table's header, so later rows lost their real column labels.
Published on a different test set, so its counts are not compared here.
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
No published Result yet (11): Adobe PDF Extract API, Extend AI, Firecrawl, GPT-5.6 terra, Landing AI, LlamaParse, Nutrient.io, PDF Vector, Reducto, Tensorlake, Unstructured.
No published Result does not tell you whether a tool has been tested.Test design
- Pinned test cases
- 2
- Disclosure
- 1 public · 1 withheld
- Capabilities exercised here
- Table Extraction
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 pins a scenario rubric for this scenario. Benchmark methodology →