Scenario in benchmark · Structured Document Extraction · Version 1

A cleanly scanned document

A document is scanned and image-only, so its visible text must be recovered without relying on a text layer.

Capability: OCR1 pinned test case per toolNeutral global definition →

What we check

  1. Whether the visible text from the scanned page is recovered accurately.
  2. Whether the page is not silently skipped or treated as empty because no text layer exists.
  3. Whether the output stays faithful to the visible content without inventing text.
  4. Whether scanned pages are handled as real input, not hidden by omission.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Passed · 5
DatalabPassed1 of 1 test case passedExtracted accurate invoice values from the image-only scan.View Result →

The scan and digital original both returned the source invoice's £1,754.40 total due and the same three line items.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

FutureSmart Document IntelligencePassed1 of 1 test case passedPasses this clean-scan OCR check.View Result →

The image-only twin has no text layer, and the digital and scanned runs match on every graded field.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Landing AIPassed1 of 1 test case passedExtracted the same values from a scanned invoice as from its digital original.View Result →

Eight of nine fields were identical, including three line items; the currency was expressed differently but meant pounds sterling in both.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

LlamaParsePassed1 of 1 test case passedPassed this OCR baseline.View Result →

It returned the same invoice values for the digital file and the image-only twin, including the page 2 totals, and the twin had no text layer. The two runs differed only in job details such as timestamps and confidence flags.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

ReductoPassed1 of 1 test case passedRead the clean scan without changing the values extracted from the digital original.View Result →

Both versions returned all 20 requested values, preserving currency formatting and hire-period punctuation.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (5): Docsumo, Extend AI, Nanonets, Retab, Unstract.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
OCR

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

A cleanly scanned document in Structured Document Extraction | AI Demos