Tool in benchmark · Version 1

FutureSmart Document Intelligence in Structured Document Extraction

Scenario-level performance from current published Results.

19 scenarios with published Results · 19 scenarios in the benchmark

How FutureSmart Document Intelligence performed

Open a capability to explore its scenarios. Each row reports the test set in its published Result; counts are not combined into an overall score.

Field Extraction5 scenarios · 5 with published Results
ScenarioPublished outcomesTest coverageResult
The field is absent
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

FutureSmart Document Intelligence returned an honest null for the absent field. The structured data row shows the loyalty cell empty while sibling fields are filled, and the document details output carries only document-level fields; the source text has no loyalty or member mention.

1/1 assessed1/1 gradablePublished test setView Result →
The same field across layouts
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

FutureSmart Document Intelligence extracted the same field correctly across three differently laid-out documents. The baseline receipt, the invoice, and the stacked receipt all returned the right identifiers and amounts; the tax field format varied, and the invoice was labeled as an invoice while the other two were receipts.

1/1 assessed1/1 gradablePublished test setView Result →
The value is directly available
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

FutureSmart Document Intelligence passed this plain-field extraction case. The output and the document overview both returned the receipt number as RHS-70234, and the receipt text shows a different nearby identifier.

1/1 assessed1/1 gradablePublished test setView Result →
The value must be derived
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

FutureSmart Document Intelligence derived the missing fuel cost for this invoice. The invoice shows 450 miles and £0.18 per mile, and the product returned 81 in both the JSON output and the overview card.

1/1 assessed1/1 gradablePublished test setView Result →
The value needs a supplied definition
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

FutureSmart Document Intelligence passed this definition-driven extraction case. It returned the consignment reference, not the more prominent invoice number, and the field definition named the consignment reference and excluded the invoice number.

1/1 assessed1/1 gradablePublished test setView Result →
Nested & Repeated Fields4 scenarios · 4 with published Results
ScenarioPublished outcomesTest coverageResult
A repeated set of records
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

FutureSmart Document Intelligence passed this repeated-set case. It returned three line items with the right descriptions, quantities, unit prices, and amounts in the right rows; item code was requested but missing from every record.

1/1 assessed1/1 gradablePublished test setView Result →
The set contains a non-record
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

FutureSmart Document Intelligence passed this non-member scenario. It kept the five genuine rows, omitted the Total row, showed the same five items on screen, and treated the 1,982.00 GBP total as document-level information.

1/1 assessed1/1 gradablePublished test setView Result →
The set continues across a page break
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

FutureSmart Document Intelligence kept one array across the page break. The invoice splits across two pages, and the output keeps ten entries in order with item 6 followed by item 7 and no missing or repeated member.

1/1 assessed1/1 gradablePublished test setView Result →
The set is legitimately empty
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

FutureSmart Document Intelligence passed this empty-set extraction case. It returned damage_exceptions as an empty array, and the document says no damage or exceptions were reported; the overview card omits that field.

1/1 assessed1/1 gradablePublished test setView Result →
OCR2 scenarios · 2 with published Results
ScenarioPublished outcomesTest coverageResult
A cleanly scanned document
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

FutureSmart Document Intelligence passes this clean-scan OCR check. The image-only twin has no text layer, and the digital and scanned runs match on every graded field.

1/1 assessed1/1 gradablePublished test setView Result →
A document mixing digital and scanned pages
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

FutureSmart Document Intelligence handled the mixed-page document successfully. It returned all five requested values exactly, including the two that existed only on the scanned page, and the three digital-page values matched unchanged.

1/1 assessed1/1 gradablePublished test setView Result →
Querying3 scenarios · 3 with published Results
ScenarioPublished outcomesTest coverageResult
Aggregate across documents
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

FutureSmart Document Intelligence aggregated across the three named invoices and returned the exact sum, £10,327.64. The Coppergate invoice's £1,982.00 total matched its line items, and the answer itemized the three document totals without any document missing or extra value added.

1/1 assessed1/1 gradablePublished test setView Result →
Filter records across documents
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

FutureSmart Document Intelligence passes this cross-document filter task. It returned Meridian Retail Group invoice MRG-INV-20481 at £6,591.24. The run covered 17 documents, and a later 20-document totals table still shows that Blackwood invoice was the only record above £2,500.

1/1 assessed1/1 gradablePublished test setView Result →
The answer was never in the schema
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

FutureSmart Document Intelligence passed this scenario. It gave the IFSC code and then said it came from reading the document directly, not from a separate extracted metadata field.

1/1 assessed1/1 gradablePublished test setView Result →
Review and Correction3 scenarios · 3 with published Results
ScenarioPublished outcomesTest coverageResult
Correct a field value
0 Pass1 Fail0 Not gradable
1 of 1 test case failed
What happened

FutureSmart Document Intelligence did not let the corrected value reach the stored record. The overview and document view showed plain-text fields with no edit control, and the stored receipt number stayed RHS-70234 while RHS-99999 never appeared.

1/1 assessed1/1 gradablePublished test setView Result →
Corrected data goes downstream
0 Pass1 Fail0 Not gradable
1 of 1 test case failed
What happened

FutureSmart Document Intelligence did not keep corrected data downstream. The export path existed, but it carried only the original extraction, and the receipt row in the downloaded CSV still held RHS-70234.

1/1 assessed1/1 gradablePublished test setView Result →
Repair a record
0 Pass1 Fail0 Not gradable
1 of 1 test case failed
What happened

FutureSmart Document Intelligence fails this repair task. It could not add a new line item or find a delete option, and the read-back stayed at the same ten Blackwood items.

1/1 assessed1/1 gradablePublished test setView Result →
Source Grounding2 scenarios · 2 with published Results
ScenarioPublished outcomesTest coverageResult
The field is absent
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

FutureSmart Document Intelligence returned an honest null for the absent field. The structured data row shows the loyalty cell empty while sibling fields are filled, and the document details output carries only document-level fields; the source text has no loyalty or member mention.

1/1 assessed1/1 gradablePublished test setView Result →
Trace a value to its location
0 Pass1 Fail0 Not gradable
1 of 1 test case failed
What happened

FutureSmart Document Intelligence failed to trace the subtotal to its document location. The subtotal was on page 2, but the saved result had no location field, the chat only named a chunk, and the magnifier highlighted text in the extracted-text panel rather than a returned pointer.

1/1 assessed1/1 gradablePublished test setView Result →
Training1 scenario · 1 with published Results
ScenarioPublished outcomesTest coverageResult
A field the tool can only get right from examplesDifferent test setPublished Result available, but not included in this comparison.View Result →

Reading these Results

Published evidence and test coverage answer different questions.

Publication availability

Which scenarios have a Result?

A published Result is public evidence for this tool on one scenario. “No published result” does not say whether testing has taken place.

Test coverage

What does each Result cover?

Assessed includes Pass, Fail and Not gradable. Gradable includes Pass and Fail. Both use the pinned test count in that published Result.

Scenario scope

Inventory is not testing progress

The 19 scenarios describe this benchmark’s scope. They are not an assumed applicability or test-coverage denominator for FutureSmart Document Intelligence.

FutureSmart Document Intelligence in Structured Document Extraction | AI Demos