Scenario in benchmark · Structured Document Extraction · Version 1

The value is directly available

A document contains one plainly labelled scalar field, and the task is to extract and return that value.

Capability: Field Extraction1 pinned test case per toolNeutral global definition →

What we check

  1. Whether the tool returns the value attached to the plainly labelled field.
  2. Whether it ignores unrelated text and does not invent a different answer.
  3. Whether it treats the task as direct extraction rather than inference or correction.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Passed · 6
DocsumoPassed1 of 1 test case passedReturned the plainly labelled receipt number correctly and as text, and the three totals requested alongside it matched the receipt's totals block.View Result →

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Extend AIPassed1 of 1 test case passedPassed the directly available value scenario.View Result →

It returned the receipt number once under its label, and the JSON result matched that value as a string.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

FutureSmart Document IntelligencePassed1 of 1 test case passedPassed this plain-field extraction case.View Result →

The output and the document overview both returned the receipt number as RHS-70234, and the receipt text shows a different nearby identifier.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Landing AIPassed1 of 1 test case passedPassed this plain-field extraction case.View Result →

It returned the receipt number as a string, and the source span was highlighted on the page; the input showed one receipt number and no competing same-type value nearby.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

LlamaParsePassed1 of 1 test case passedCorrectly extracted the directly available receipt number as text.View Result →

It also returned subtotal, tax and total due as $85.00, $5.10 and $90.10, respectively—the same amounts printed on the receipt.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

NanonetsPassed1 of 1 test case passedPassed this single-field extraction task.View Result →

The receipt number appeared once on the document and returned as RHS-70234 in both the preview table and exported JSON; the other returned fields also matched the document.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (4): Datalab, Reducto, Retab, Unstract.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Field Extraction

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

The value is directly available in Structured Document Extraction | AI Demos