Graded 23 October 2026
Can FutureSmart Document Intelligence filter records across documents?
FutureSmart Document Intelligence passes this cross-document filter task. It returned Meridian Retail Group invoice MRG-INV-20481 at £6,591.24. The run covered 17 documents, and a later 20-document totals table still shows that Blackwood invoice was the only record above £2,500.
1 of 1 test case passed
Every test case in this scenario has a result.
Pass rate100%1 of 1 gradable
Coverage1 of 1test case with a result
1PassDid everything it was expected to do.
0FailNo test case failed.
0Not gradableEvery result here could be graded.
0UntestedEvery test case in this scenario has a result.
The pass rate is a summary. The evidence is the test case below: what we sent, what we checked, what the tool returned, and the proof.
The test case
Each test case is judged on its own: Pass, Fail, Not gradable, or Untested. The scenario result above counts this row.
Filter across the extracted setPassEvidence
What we sent
“Which of these documents have a total due over £2,500?”
Input 1Document (PDF)
Input 2Document (PDF)
Input 3Document (PDF)
Input 4Document (PDF)
Input 5Document (PDF)
Input 6Document (PDF)
Input 7Document (PDF)
Input 8Document (PDF)
Input 9Document (PDF)
What the tool returned
Output file (JSON)
Cited values8 values from this file
- message
- The document with a total due over £2,500 is: - **Meridian Retail Group invoice MRG-INV-20481** — **£6,591.24**
- data › di_run › model
- gpt-5.4-mini
- data › knowledge › knowledge_evidence 1 › query
- invoice receipt total due amount over £2,500 £6,591.24
- data › knowledge › knowledge_evidence 2 › query
- "Total Due" OR "TOTAL" invoice receipt amount due £
- data › knowledge › knowledge_evidence 1 › documents 1 › score
- 1
- data › knowledge › detailed_chunks › 45a86024-1893-4ef0-b83e-b65858eebb1d 1 › file_name
- main_invoice_multipage_blackwood.pdf
- data › knowledge › detailed_chunks › 80746437-7de4-44d0-8e4f-640198986339 1 › file_name
- SiblingA1_Thistlewood_PlantHire.pdf
- data › knowledge › detailed_chunks › e6290e0c-d06b-4c63-8b9d-89fd26aeea3e 1 › file_name
- TC8_total_row_lookalike.pdf
Show the whole file (13 KB)
JSON
{ "chat_id": "95c8ce6e-f313-4cc9-9e7e-7f567fb42d6e", "role": "assistant", "session_id": "88720eb1-9562-4435-89b1-acff0c8ccd97", "message": "The document with a total due over £2,500 is:\n\n- **Meridian Retail Group invoice MRG-INV-20481** — **£6,591.24**", "sources": null, "chart_output": null, "user_feedback": null, "data": { "knowledge": { "knowledge_evidence": [ { "tool": "retrieve_chunks", "query": "invoice receipt total due amount over £2,500 £6,591.24", "source": "global_chunk_scan", "documents": [ { "score": 1, "chunk_ids": ["af60f0d1-25a8-54af-8d6b-fd3cf64aca87", "79bab586-d69a-5d9a-b228-1f70d929da80"], "object_id": "45a86024-1893-4ef0-b83e-b65858eebb1d" }, { "score": 0.6667, "chunk_ids": ["606e65b4-d08f-5ffb-82fd-7fd29279cfb3", "105a5933-c357-5362-8fbf-3e9bf5b1c67c"], "object_id": "75285524-15d3-4d21-9000-cf4c1bc8e326" }, { "score": 0.35, "chunk_ids": ["3ff1ba22-4a8d-57ee-ad6c-189c09400a9a"], "object_id": "26066a63-9267-48df-8e81-a8097218724b" }, { "score": 0.3111, "chunk_ids": ["a3cd3d85-bf0b-59d8-8056-6ef1e351b6ca"], "object_id": "fb667677-cc22-487a-99d6-e013ef1bdec8" }, { "score": 0.3095, "chunk_ids": ["5b64f540-ea08-57df-a3fd-7437d5b9759c"], "object_id": "80746437-7de4-44d0-8e4f-640198986339" }, { "score": 0.2679, "chunk_ids": ["fd4ca825-4853-53cd-9348-020cd198adf2"], "object_id": "d2a9652f-76c9-4666-aae6-d84e6c188497" }, { "score": 0.2576, "chunk_ids": ["ff2e009e-7658-59aa-ad2f-4992a47ee59f"], "object_id": "e738012c-60a4-4b14-9c11-9d2ab98c32ab" }, { "score": 0.2361, "chunk_ids": ["b0747967-1fcb-5a32-a753-6ce7e7958b50"], "object_id": "4daa8753-c6f9-49a9-bfaf-03c9be8656ed" } ], "question_shape": "predicate" }, { "tool": "retrieve_chunks", "query": "\"Total Due\" OR \"TOTAL\" invoice receipt amount due £", "source": "global_chunk_scan", "documents": [ { "score": 0.5159, "chunk_ids": ["606e65b4-d08f-5ffb-82fd-7fd29279cfb3", "105a5933-c357-5362-8fbf-3e9bf5b1c67c"], "object_id": "75285524-15d3-4d21-9000-cf4c1bc8e326" }, { "score": 0.5159, "chunk_ids": ["af60f0d1-25a8-54af-8d6b-fd3cf64aca87", "79bab586-d69a-5d9a-b228-1f70d929da80"], "object_id": "45a86024-1893-4ef0-b83e-b65858eebb1d" }, { "score": 0.4986, "chunk_ids": ["5b64f540-ea08-57df-a3fd-7437d5b9759c"], "object_id": "80746437-7de4-44d0-8e4f-640198986339" }, { "score": 0.4984, "chunk_ids": ["fd4ca825-4853-53cd-9348-020cd198adf2"], "object_id": "d2a9652f-76c9-4666-aae6-d84e6c188497" }, { "score": 0.4984, "chunk_ids": ["b0747967-1fcb-5a32-a753-6ce7e7958b50"], "object_id": "4daa8753-c6f9-49a9-bfaf-03c9be8656ed" }, { "score": 0.494, "chunk_ids": ["528c8719-02dd-5245-8d15-9e45b53e577a"], "object_id": "e6290e0c-d06b-4c63-8b9d-89fd26aeea3e" }, { "score": 0.4913, "chunk_ids": ["3ff1ba22-4a8d-57ee-ad6c-189c09400a9a"], "object_id": "26066a63-9267-48df-8e81-a8097218724b" }, { "score": 0.4913, "chunk_ids": ["ff2e009e-7658-59aa-ad2f-4992a47ee59f"], "object_id": "e738012c-60a4-4b14-9c11-9d2ab98c32ab" } ], "question_shape": "predicate" } ], "relevant_sources": { "data": [], "tool_name": "rag" }, "summaries": {}, "detailed_chunks": { "45a86024-1893-4ef0-b83e-b65858eebb1d": [ { "id": "af60f0d1-25a8-54af-8d6b-fd3cf64aca87", "file_name": "main_invoice_multipage_blackwood.pdf", "object_id": "45a86024-1893-4ef0-b83e-b65858eebb1d", "chunk_text": "---\n\n\n\n# **Meridian Retail Group**\nRETAIL FIXTURES & STORE EQUIPMENT SUPPLY\n\n# INVOICE\nPage 2 of 2 — MRG-INV-20481\n\n<table>\n<thead>\n<tr>\n<th>Sl.</th>\n<th>Item Description</th>\n<th>Qty</th>\n<th>Unit Price (£)</th>\n<th>Amount (£)</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>7</td>\n<td>Barcode Scanner - Wireless</td>\n<td>10</td>\n<td>58.00</td>\n<td>580.00</td>\n</tr>\n<tr>\n<td>8</td>\n<td>Receipt Paper Roll - Case of 50</td>\n<td>15</td>\n<td>22.00</td>\n<td>330.00</td>\n</tr>\n<tr>\n<td>9</td>\n<td>Mannequin - Adjustable, Full Body</td>\n<td>6</td>\n<td>95.00</td>\n<td>570.00</td>\n</tr>\n<tr>\n<td>10</td>\n<td>Display Signage Kit - Window Set</td>\n<td>5</td>\n<td>76.00</td>\n<td>380.00</td>\n</tr>\n<tr>\n<td></td>\n<td></td>\n<td></td>\n<td>Subtotal</td>\n<td>£6,103.00</td>\n</tr>\n<tr>\n<td></td>\n<td></td>\n<td></td>\n<td>VAT (8%)</td>\n<td>£488.24</td>\n</tr>\n<tr>\n<td></td>\n<td></td>\n<td></td>\n<td><b>TOTAL</b></td>\n<td><b>£6,591.24</b></td>\n</tr>\n</tbody>\n</table>\n\nPayment due within 30 days to Meridian Retail Group, Continental Trust Bank, Acc. 5021 8834 7712, Sort Code CTBK0002211. Meridian Retail Group | Registered in England No. 08847721 | accounts@meridianretailgroup.example", "document_id": "45a86024-1893-4ef0-b83e-b65858eebb1d", "knowledge_id": "d76e5c59-55b2-467c-af7a-646a233dc9e1", "search_method": "hybrid_knowledge_base", "relevance_score": 1 }, { "id": "79bab586-d69a-5d9a-b228-1f70d929da80", "file_name": "main_invoice_multipage_blackwood.pdf", "object_id": "45a86024-1893-4ef0-b83e-b65858eebb1d", "chunk_text": "File Name: main_invoice_multipage_blackwood.pdf\n\nExtracted Text: \n\n# **Meridian Retail Group**\nRETAIL FIXTURES & STORE EQUIPMENT SUPPLY\n\n# INVOICE\nPage 1 of 2\n\nFROM\n**Meridian Retail Group**\n18 Market Square, Camden\nLondon NW1 8QP, United Kingdom\nVAT No. GB 402 8817 55\n\nBILL TO\n**Blackwood Distribution LLC**\n402 Harborfront Way, Suite 210\nBristol BS1 6XN, United Kingdom\nContact: Mr. Desmond Ayotte\n\nINVOICE NO. **MRG-INV-20481**\nINVOICE DATE **06 Jun 2020**\nDUE DATE **06 Jul 2020**\nPO NUMBER **BW-PO-9982**\n\n<table>\n<thead>\n<tr>\n<th>Sl.</th>\n<th>Item Description</th>\n<th>Qty</th>\n<th>Unit Price (£)</th>\n<th>Amount (£)</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1</td>\n<td>Retail Shelving Unit - Steel, 5-Tier</td>\n<td>12</td>\n<td>84.00</td>\n<td>1,008.00</td>\n</tr>\n<tr>\n<td>2</td>\n<td>Checkout Counter - Laminate Top</td>\n<td>3</td>\n<td>310.00</td>\n<td>930.00</td>\n</tr>\n<tr>\n<td>3</td>\n<td>Security Tag Deactivator - Handheld</td>\n<td>6</td>\n<td>45.50</td>\n<td>273.00</td>\n</tr>\n<tr>\n<td>4</td>\n<td>Price Label Printer - Thermal</td>\n<td>4</td>\n<td>162.00</td>\n<td>648.00</td>\n</tr>\n<tr>\n<td>5</td>\n<td>Shopping Basket - 20L, Set of 10</td>\n<td>8</td>\n<td>38.00</td>\n<td>304.00</td>\n</tr>\n<tr>\n<td>6</td>\n<td>Storeroom Pallet Racking - 4-Bay</td>\n<td>2</td>\n<td>540.00</td>\n<td>1,080.00</td>\n</tr>\n</tbody>\n</table>\n\n*Continued on page 2 of 2 — line items 7 through 10 and invoice totals follow.*\n\nMeridian Retail Group | Registered in England No. 08847721 | accounts@meridianretailgroup.example\n\n---\n\n\n\n# **Meridian Retail Group**\nRETAIL FIXTURES & STORE EQUIPMENT SUPPLY\n\n# INVOICE\nPage 2 of 2 — MRG-INV-20481", "document_id": "45a86024-1893-4ef0-b83e-b65858eebb1d", "knowledge_id": "d76e5c59-55b2-467c-af7a-646a233dc9e1", "search_method": "hybrid_knowledge_base", "relevance_score": 0.2909091 } ], "80746437-7de4-44d0-8e4f-640198986339": [ { "id": "5b64f540-ea08-57df-a3fd-7437d5b9759c", "file_name": "SiblingA1_Thistlewood_PlantHire.pdf", "object_id": "80746437-7de4-44d0-8e4f-640198986339", "chunk_text": "File Name: SiblingA1_Thistlewood_PlantHire.pdf\n\nExtracted Text: \n\n# **Thistlewood Plant & Tool Hire**\nGROUNDWORKS & SITE EQUIPMENT\n\n# **INVOICE**\n\nUnit 4, Sedgefield Trading Estate, Darrowby DL9 3PQ\n01325 771 4402 · accounts@thistlewood-hire.example · VAT reg. GB 559 1120 74\n\n**BILL TO**\nOakbridge Construction Ltd\nSite Office, Oakbridge Depot Extension\nDarrowby DL4 7HN\n\nInvoice number **TPH-3301**\nInvoice date **10 March 2026**\nPayment due **09 April 2026**\n\n**SITE**\nOakbridge Depot Extension\n\n**HIRE PERIOD**\n03 Mar - 10 Mar 2026\n\n<table>\n <thead>\n <tr>\n <th>Description</th>\n <th>Qty</th>\n <th>Unit price</th>\n <th>Amount</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <td>Telehandler, 4-tonne, weekly hire</td>\n<td>1</td>\n<td>£850.00</td>\n<td>£850.00</td>\n </tr>\n<tr>\n <td>Generator, 20kVA, weekly hire</td>\n<td>2</td>\n<td>£210.00</td>\n<td>£420.00</td>\n </tr>\n<tr>\n <td>Site fencing, per 10m panel, weekly</td>\n<td>8</td>\n<td>£24.00</td>\n<td>£192.00</td>\n </tr>\n </tbody>\n</table>\n\n*Continued on page 2 — totals and remittance details follow.*\n\nTPH-3301\nPage 1 of 2\n\n---\n\n\n\nThistlewood Plant & Tool Hire\nInvoice TPH-3301\n\n<table>\n<thead>\n<tr>\n<th>Item</th>\n<th>Amount</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Subtotal</td>\n<td>£1,462.00</td>\n</tr>\n<tr>\n<td>VAT at 20%</td>\n<td>£292.40</td>\n</tr>\n<tr>\n<td><b>Total Due</b></td>\n<td><b>£1,754.40</b></td>\n</tr>\n</tbody>\n</table>\n\n## **Remittance**\n\nPayment is due within 30 days of invoice date to Thistlewood Plant & Tool Hire, Darrowby & Dales Bank, Acc. 4471 0092 6614, Sort Code 20-55-71. Equipment remains the property of Thistlewood Plant & Tool Hire and is insured by the hirer for the duration of the hire period.\n\nTPH-3301\nPage 2 of 2", "document_id": "80746437-7de4-44d0-8e4f-640198986339", "knowledge_id": "d76e5c59-55b2-467c-af7a-646a233dc9e1", "search_method": "hybrid_knowledge_base", "relevance_score": 0.30952382 } ], "e6290e0c-d06b-4c63-8b9d-89fd26aeea3e": [ { "id": "528c8719-02dd-5245-8d15-9e45b53e577a", "file_name": "TC8_total_row_lookalike.pdf", "object_id": "e6290e0c-d06b-4c63-8b9d-89fd26aeea3e", "chunk_text": "File Name: TC8_total_row_lookalike.pdf\n\nExtracted Text: \n\n# Meridian Retail Group\nRETAIL FIXTURES & STORE EQUIPMENT SUPPLY\n\n# INVOICE\n\nFROM\n**Meridian Retail Group**\n18 Market Square, Camden\nLondon NW1 8QP, United Kingdom\n\nBILL TO\n**Coppergate Wholesale Ltd**\n29 Kilm Lane, Nottingham NG1 5FQ\nUnited Kingdom\n\nINVOICE NO. **MRG-INV-20517**\nINVOICE DATE **02 Sep 2020**\nDUE DATE **02 Oct 2020**\n\n<table>\n <tr>\n <th>Sl.</th>\n <th>Item Description</th>\n <th>Qty</th>\n <th>Unit Price (£)</th>\n <th>Amount (£)</th>\n </tr>\n<tr>\n <td>1</td>\n<td>Retail Shelving Unit - Steel, 5-Tier</td>\n<td>6</td>\n<td>84.00</td>\n<td>504.00</td>\n </tr>\n<tr>\n <td>2</td>\n<td>Checkout Counter - Laminate Top</td>\n<td>2</td>\n<td>310.00</td>\n<td>620.00</td>\n </tr>\n<tr>\n <td>3</td>\n<td>Security Tag Deactivator - Handheld</td>\n<td>4</td>\n<td>45.50</td>\n<td>182.00</td>\n </tr>\n<tr>\n <td>4</td>\n<td>Price Label Printer - Thermal</td>\n<td>3</td>\n<td>162.00</td>\n<td>486.00</td>\n </tr>\n<tr>\n <td>5</td>\n<td>Shopping Basket - 20L, Set of 10</td>\n<td>5</td>\n<td>38.00</td>\n<td>190.00</td>\n </tr>\n<tr>\n <td>—</td>\n<td><b>Total</b></td>\n<td>—</td>\n<td>—</td>\n<td><b>1,982.00</b></td>\n </tr>\n</table>\n\nMeridian Retail Group | Registered in England No. 08847721 | accounts@meridianretailgroup.example", "document_id": "e6290e0c-d06b-4c63-8b9d-89fd26aeea3e", "knowledge_id": "d76e5c59-55b2-467c-af7a-646a233dc9e1", "search_method": "semantic_only", "relevance_score": 0.49397212 } ] }, "chart_output": { "type": "direct_response", "message": "Below is a table of the documents where total_due_gbp exceeds £2,500:\n\n| document_id | document_name | due_date | file_name | invoice_date | invoice_number | total_due_gbp |\n|---|---|---|---|---|---|---|\n| (list the matching rows from the dataframe) |", "reasoning": null, "graph_generated": false } }, "di_run": { "refined_query": "Which of these documents have a total due over £2,500?", "llm_provider": "openai", "model": "gpt-5.4-mini", "is_answered": true, "stored": true } } }
Expected vs. Found
| ✓ | Expected: Only the matching records from the whole collection.Found: It returned the single matching invoice at £6,591.24 and did not include any extra record. |
Supporting proof
Proof 1Screenshot
Proof 2Screenshot
Proof 3Captured file (text)
Text46 lines
TC80 — total due for EVERY document in the "Plant & Freight Hire Invoices" collection (20 documents as of 2026-10-06) Column "Source PDF total due" = read directly from the source PDF (ground truth). Currency as printed on the document. Threshold in the question: total due over 2,500. # Doc id (first 8) File name in collection Added Source PDF total due Over 2,500? 1 661d269b Calderwood_Freight_S66_S67_S72.pdf Sep 24 2026 £822.00 no 2 d45aa7f4 Calderwood_Freight_S66_S67_S72.pdf Sep 21 2026 £822.00 no 3 fef35859 Calderwood_Freight_S66_S67_S72.pdf Sep 21 2026 £822.00 no 4 f7ddaf58 Calderwood_Freight_S66_S67_S72.pdf Sep 21 2026 £822.00 no 5 29c942d0 Calderwood_Freight_S66_S67_S72.pdf Sep 21 2026 £822.00 no 6 0b0bd090 Baseline_receipt.pdf Sep 21 2026 $90.10 no 7 135310f9 SiblingA1_Thistlewood_PlantHire.pdf Sep 21 2026 £1,754.40 no 8 886070d1 SiblingA1_Thistlewood_PlantHire_-_Scanned_Twin.pdf Sep 21 2026 £1,754.40 (scanned copy of #7) no 9 129f1ba9 invoice_HIS-4402_-_Digital_Scanned_Digital.pdf Sep 21 2026 £1,246.80 no 10 3721a82a TC1_TC4_TC5A_baseline_receipt.pdf Sep 21 2026 $90.10 no 11 45a86024 main_invoice_multipage_blackwood.pdf Sep 19 2026 £6,591.24 YES 12 5b3bc71c Calderwood_Freight_S66_S67_S72.pdf Sep 19 2026 £822.00 no 13 e6290e0c TC8_total_row_lookalike.pdf Sep 19 2026 £1,982.00 no 14 75285524 main_invoice_multipage_blackwood.pdf Sep 19 2026 £6,591.24 YES 15 fb667677 SiblingA1_Thistlewood_PlantHire_-_Scanned_Twin.pdf Sep 19 2026 £1,754.40 (scanned copy of #7) no 16 80746437 SiblingA1_Thistlewood_PlantHire.pdf Sep 19 2026 £1,754.40 no 17 a8bbd31e TC5B_altlabel_receipt.pdf Sep 18 2026 $227.33 no 18 073a4399 TC5C_altlayout_receipt.pdf Sep 18 2026 $157.50 no 19 cbb19486 TC1_TC4_TC5A_baseline_receipt_1.pdf Sep 18 2026 $90.10 no 20 dd1200e5 Calderwood_Freight_S66_S67_S72.pdf Sep 18 2026 £822.00 no Result: 20 documents (11 distinct source files). The only documents over 2,500 are the two uploads of main_invoice_multipage_blackwood.pdf (#11 and #14), total £6,591.24. All 18 others are at or below £1,982.00. Nothing over 2,500 can have been missed. Source files read: Calderwood_Freight_S66_S67_S72.pdf (Total Due £822.00), SiblingA1_Thistlewood_PlantHire.pdf (Total Due £1,754.40), main_invoice_multipage_blackwood.pdf (TOTAL £6,591.24), TC8_total_row_lookalike.pdf (Total £1,982.00), invoice_HIS-4402 (Total due £1,246.80), TC5B (Amount Due $227.33), TC5C (Total Due USD 157.50), baseline receipt (Total $90.10). The scanned twin is an image-only copy of the Thistlewood invoice (same content, £1,754.40). --- Tool's own answer, same collection, new chat, 2026-10-06 ("List the total due for every document in this collection, one line per document (file name and total due).") --- Read 8 documents (of 20), 8 sources: Calderwood_Freight_S66_S67_S72.pdf — Total due not surfaced SiblingA1_Thistlewood_PlantHire.pdf — £1,754.40 SiblingA1_Thistlewood_PlantHire_-Scanned_Twin.pdf — £1,754.40 TC1_TC4_TC5A_baseline_receipt.pdf — Total due not surfaced TC1_TC4_TC5A_baseline_receipt_1.pdf — $90.10 Baseline_receipt.pdf — $90.10 main_invoice_multipage_blackwood.pdf — £6,591.24 invoice_HIS-4402-_Digital_Scanned_Digital.pdf — £1,246.80 TC5B_altlabel_receipt.pdf — $227.33 TC8_total_row_lookalike.pdf — £1,982.00 TC5C_altlayout_receipt.pdf — $157.50 Every value the tool did report matches the source PDFs. It folded the repeated uploads into one line per file name, and reported "not surfaced" for two files whose totals are in the PDFs (£822.00 and $90.10).
Why this result
The task asked for the collection-wide set above the threshold. The response named the Meridian Retail Group invoice at £6,591.24, and the totals table shows no other document above £2,500, so the returned set is exact.
Also observed on this row
Run log mismatch. The run log said it searched 0 documents, while the native JSON shows two retrieval searches over the collection.
Follow-up gaps. In a later list of every document's total, repeated uploads were folded into one line per file name and two totals were left unsurfaced.
Tested by Rugved Nichite · evidence dated 21 September 2026
Limitations of this result
The totals table was read from the source PDFs after the answer run, when the collection had grown to 20 documents.
Configuration and setup
How this tool was set up for the run and what the test needed in place. Each row is a fact from the run's records; a fact the records do not hold is left out, not guessed.
Software that produced the output
FutureSmart Document Intelligence
Build or version
Not exposed by the hosted product
Model
gpt-5.4-mini
Surface
Web app
Set up before the run
A collection of one-class documents, one comparable scalar per document, with both matching and non-matching records; ask which files meet one plain condition across the whole extracted set.
Tested
21 September 2026 · Rugved Nichite
How this scenario is graded
How we decide Pass, Fail and Not gradable. The same rules apply to every tool tested on this scenario.
How results are decided
Each test case gets one result per tool: Pass, Fail or Not gradable. A test case we haven't run yet shows Untested. There are no partial results.
The rules
- Pass: the tool did everything the test expected, and nothing it said contradicts the correct answer.
- Fail: at least one expected behaviour clearly didn't happen; the row says which and quotes the tool.
- Not gradable: our evidence couldn't settle the outcome (for example a record we needed is missing). It is never counted as a fail, and the row says what's missing.
Where this sits in the benchmark
This page is one cell of a larger study: one tool, one scenario. Only this benchmark's frame appears here.
| Level | Name | Scope |
|---|---|---|
| Benchmark | Structured Document Extraction → | 19 scenarios · 10 tools |
| Capability | Querying → | |
| Scenario | Filter records across documents → | |
| Tool | FutureSmart Document Intelligence → |
Global scenario definition → · Global capability definition →
History of this result
What has happened to this result since it was first published. Runs and grades are never overwritten: a retest or a re-grade publishes a new result and keeps the earlier one readable.
23 October 2026First publishedStructured Document Extraction v1
27 October 2026Updated — the page was revised
The result is unchanged: 1 pass · 0 fail. The earlier result stays readable. Read the earlier result →
Act on this result
Nothing filed here edits the run or the grade. A challenge opens a review, and a review can produce a new run or a re-grade — which becomes the current result and leaves this one in the history.
This matches what I see
You run the same kind of test against your own setup and get the same behaviour.
Agree →This does not match
Yours behaves differently. Tell us what you got, with a screenshot if you have one.
Disagree →Point out an issue
Something here is wrong — a reference value, a transcription, a grade.
Report an issue →We have fixed this
Tell us what changed and we schedule a rerun of the failing test case. The old result stays as history.
Vendor notice →Filed against this evidenceNothing yet. Challenges, counter-evidence and fix notices appear here with their outcome, and stay on the page after they are resolved.
Cite this result
aidemos.com/benchmarks/structured-document-extraction/results/futuresmart-document-intelligence/filter-records-across-documents · 1 pass · 0 fail · coverage 1/1 · graded 2026-10-23
The same record is available as structured data through the AI Demos MCP server, with the counts, the coverage and every per-test-case reason carried as fields.
Verify the proof files
These files support this result. Open a file to inspect the original evidence.
Input 1 · Document (PDF) ↗application/pdf · 54 KB
Input 2 · Document (PDF) ↗application/pdf · 7 KB
Input 3 · Document (PDF) ↗application/pdf · 8 KB
Input 4 · Document (PDF) ↗application/pdf · 55 KB
Input 5 · Document (PDF) ↗application/pdf · 118 KB
Input 6 · Document (PDF) ↗application/pdf · 176 KB
Input 7 · Document (PDF) ↗application/pdf · 232 KB
Input 8 · Document (PDF) ↗application/pdf · 78 KB
Input 9 · Document (PDF) ↗application/pdf · 82 KB
Output file (JSON) ↗application/json · 13 KB
Proof 1 · Screenshot ↗image/png · 145 KB
Proof 2 · Screenshot ↗image/png · 122 KB
Proof 3 · Captured file (text) ↗text/plain; charset=utf-8 · 4 KB
File fingerprints (SHA-256)
A fingerprint identifies the exact file used for this result.
Input 1 · Document (PDF)a96bbe1c5b22336e7b54169baf72b85e45eabf1312cd293a1821017b52fa85f7
Input 2 · Document (PDF)0276d246d69482340b53797f4eaa21831977cede22c8c8f14fc48dbf84757660
Input 3 · Document (PDF)4a0b1754da9f52ce35a783d58f8e8a1b5e1353b5b992d5553122886f1cd4a638
Input 4 · Document (PDF)791590c9d422110ebebf3c1e434eebac20aff3c6728024693becc870818c26d8
Input 5 · Document (PDF)1bcc6c7b945956c5b6f5a87d223ed3d578a62aa2c56c5fc1e3abedb4cc4de72e
Input 6 · Document (PDF)89fb14d7c0897bbfef47ffa3aa3930ee7eccf62c8a9430710d1906598c9bdfec
Input 7 · Document (PDF)6f1cc2a5f5e5e57494671341557d7a53d318b0f62bb8f8536165e1dac1b7a805
Input 8 · Document (PDF)fdcf03ac5ee53b083da58166f17fd41e7c72de79a9f3a3f6b67daa1360ddbca9
Input 9 · Document (PDF)d1f3282702260730267278b1071ee30cf79918fa8489d9411c98d8f6e5209f18
Output file (JSON)6c2a0e7227d5db1a8c07be568bcb563f5269c477b641174befef46694fbc24cb
Proof 1 · Screenshote3c938bf8a3b39a6e34afac55343903137601f8fff5571b7a29346c68538d4d4
Proof 2 · Screenshotd236c92466c55f74550676f2290317bcf2b04f8600034710b9ac4bc8b0ec6208
Proof 3 · Captured file (text)774c9ff2b70f1d57eb55fda2c79192096de56dd947519a1de93b8df77ec85000