Scenario in benchmark · Structured Document Extraction · Version 1
The set continues across a page break
A repeated record set continues across a page boundary and should stay one array.
What we check
- Whether the tool returns the whole repeated set as one array.
- Whether it avoids splitting the set into separate arrays at the boundary.
- Whether it avoids dropping members that appear after the boundary.
- Whether it avoids duplicating members at the boundary.
- Whether each member remains associated with the correct record across the full set.
Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
DocsumoPassed1 of 1 test case passedReturned the invoice's ten line items, six on page one and four on page two, as a single array, with no item repeated at the page break and no second array.View Result →
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
FutureSmart Document IntelligencePassed1 of 1 test case passedKept one array across the page break.View Result →
The invoice splits across two pages, and the output keeps ten entries in order with item 6 followed by item 7 and no missing or repeated member.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Landing AIPassed1 of 1 test case passedKept the repeated set in one array when the data crossed a page break.View Result →
In the checked case, ten line items appeared in document order with no duplicate at the boundary, and the results panel showed the array opening once and closing once. No other case is reported here.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
LlamaParsePassed1 of 1 test case passedKept all ten invoice line items together in one array across the page break.View Result →
It retained the six items from page one and four from page two, with no boundary duplicate and all 40 values correct.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
NanonetsPassed1 of 1 test case passedKept one array across the page break.View Result →
The output has ten objects in one top-level array, with no second array and no repeated member at the boundary. The invoice shows items 1-6 on page 1 and 7-10 on page 2, and the result screen shows the same single table numbered 1 to 10.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
Test design
- Pinned test cases
- 1
- Disclosure
- 0 public · 1 withheld
- Capabilities exercised here
- Nested & Repeated Fields
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →