Scenario in benchmark · Structured Document Extraction · Version 1

The set continues across a page break

A repeated record set continues across a page boundary and should stay one array.

Capability: Nested & Repeated Fields1 pinned test case per toolNeutral global definition →

What we check

  1. Whether the tool returns the whole repeated set as one array.
  2. Whether it avoids splitting the set into separate arrays at the boundary.
  3. Whether it avoids dropping members that appear after the boundary.
  4. Whether it avoids duplicating members at the boundary.
  5. Whether each member remains associated with the correct record across the full set.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Passed · 5
DocsumoPassed1 of 1 test case passedReturned the invoice's ten line items, six on page one and four on page two, as a single array, with no item repeated at the page break and no second array.View Result →

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

FutureSmart Document IntelligencePassed1 of 1 test case passedKept one array across the page break.View Result →

The invoice splits across two pages, and the output keeps ten entries in order with item 6 followed by item 7 and no missing or repeated member.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Landing AIPassed1 of 1 test case passedKept the repeated set in one array when the data crossed a page break.View Result →

In the checked case, ten line items appeared in document order with no duplicate at the boundary, and the results panel showed the array opening once and closing once. No other case is reported here.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

LlamaParsePassed1 of 1 test case passedKept all ten invoice line items together in one array across the page break.View Result →

It retained the six items from page one and four from page two, with no boundary duplicate and all 40 values correct.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

NanonetsPassed1 of 1 test case passedKept one array across the page break.View Result →

The output has ten objects in one top-level array, with no second array and no repeated member at the boundary. The invoice shows items 1-6 on page 1 and 7-10 on page 2, and the result screen shows the same single table numbered 1 to 10.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (5): Datalab, Extend AI, Reducto, Retab, Unstract.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Nested & Repeated Fields

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

The set continues across a page break in Structured Document Extraction | AI Demos