Scenario in benchmark · Structured Document Extraction · Version 1
Correct a field value
A human-applied correction changes a returned field value in the stored record.
What we check
- Whether a human-applied change becomes the stored field value.
- Whether the corrected value is still present when the record is read again.
- Whether the original returned value is replaced rather than kept as the stored value.
Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
FutureSmart Document IntelligenceFailed1 of 1 test case failedDid not let the corrected value reach the stored record.View Result →
The overview and document view showed plain-text fields with no edit control, and the stored receipt number stayed RHS-70234 while RHS-99999 never appeared.
0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
LlamaParseFailed1 of 1 test case failedOffered no way to correct a returned field.View Result →
No edit control was found in the Results view, JSON view, History menu or Details drawer, and the saved receipt number stayed RHS-70234.
0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
DocsumoPassed1 of 1 test case passedKept a corrected value in the stored record: after the receipt number was changed from RHS-70234 to RHS-99999 in its review screen, confirmed and approved, the record reopened after a reload and Docsumo's own export both showed RHS-99999.View Result →
The subtotal, tax and total were unchanged.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
Test design
- Pinned test cases
- 1
- Disclosure
- 0 public · 1 withheld
- Capabilities exercised here
- Review and Correction
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →