Scenario in benchmark · Structured Document Extraction · Version 1
The same field across layouts
The same requested field appears in documents laid out differently.
What we check
- Whether the same field is extracted correctly from documents with different layouts.
- Whether layout changes do not change the field’s identity or the value returned.
- Whether the returned value remains correctly typed across layouts.
- Whether the tool avoids treating a layout change as a missing or different field.
Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
Extend AIPassed1 of 1 test case passedReturned the correct value for the same field across documents laid out differently.View Result →
The shown runs returned $90.10, $227.33, and USD 157.50, each matching its source document.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
FutureSmart Document IntelligencePassed1 of 1 test case passedExtracted the same field correctly across three differently laid-out documents.View Result →
The baseline receipt, the invoice, and the stacked receipt all returned the right identifiers and amounts; the tax field format varied, and the invoice was labeled as an invoice while the other two were receipts.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Landing AIPassed1 of 1 test case passedPassed this scenario.View Result →
It returned the correct owed amount from three documents with different layouts: $90.10 from the receipt, $227.33 from the invoice, and USD 157.50 from the other receipt. The screenshots show each value highlighted in the viewer and the same single String field in the schema.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
LlamaParsePassed1 of 1 test case passedPassed this scenario.View Result →
It returned the correct identifier, subtotal, tax, and total_due values for all three differently laid out documents, and the three outputs matched the values read from the pages while keeping the same four-field schema.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
NanonetsPassed1 of 1 test case passedReturned the correct document number, subtotal, tax and total due from all three differently laid-out documents with one unchanged prompt: Riverside RHS-70234 ($90.10), GreenLeaf GLR-98213 ($227.33) and Sunrise SPD-2026-0714 (USD 157.50).View Result →
It read the GreenLeaf number from a boxed work-order block rather than a receipt-number label.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
Test design
- Pinned test cases
- 1
- Disclosure
- 0 public · 1 withheld
- Capabilities exercised here
- Field Extraction
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →