Scenario in benchmark · Structured Document Extraction · Version 1

The same field across layouts

The same requested field appears in documents laid out differently.

Capability: Field Extraction1 pinned test case per toolNeutral global definition →

What we check

  1. Whether the same field is extracted correctly from documents with different layouts.
  2. Whether layout changes do not change the field’s identity or the value returned.
  3. Whether the returned value remains correctly typed across layouts.
  4. Whether the tool avoids treating a layout change as a missing or different field.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Passed · 5
Extend AIPassed1 of 1 test case passedReturned the correct value for the same field across documents laid out differently.View Result →

The shown runs returned $90.10, $227.33, and USD 157.50, each matching its source document.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

FutureSmart Document IntelligencePassed1 of 1 test case passedExtracted the same field correctly across three differently laid-out documents.View Result →

The baseline receipt, the invoice, and the stacked receipt all returned the right identifiers and amounts; the tax field format varied, and the invoice was labeled as an invoice while the other two were receipts.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Landing AIPassed1 of 1 test case passedPassed this scenario.View Result →

It returned the correct owed amount from three documents with different layouts: $90.10 from the receipt, $227.33 from the invoice, and USD 157.50 from the other receipt. The screenshots show each value highlighted in the viewer and the same single String field in the schema.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

LlamaParsePassed1 of 1 test case passedPassed this scenario.View Result →

It returned the correct identifier, subtotal, tax, and total_due values for all three differently laid out documents, and the three outputs matched the values read from the pages while keeping the same four-field schema.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

NanonetsPassed1 of 1 test case passedReturned the correct document number, subtotal, tax and total due from all three differently laid-out documents with one unchanged prompt: Riverside RHS-70234 ($90.10), GreenLeaf GLR-98213 ($227.33) and Sunrise SPD-2026-0714 (USD 157.50).View Result →

It read the GreenLeaf number from a boxed work-order block rather than a receipt-number label.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (5): Datalab, Docsumo, Reducto, Retab, Unstract.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Field Extraction

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

The same field across layouts in Structured Document Extraction | AI Demos