Does not consistently enrich transaction rows with schema-level fields such as transaction_type and transaction_id; the visible extracted rows remain partially populated and omit those fields.
What was measured
Semantic Field Enrichment
Are derived fields such as transaction_type, transaction_id, cheque_number, day patterns, and ad codes correctly classified or extracted beyond raw OCR?
decisive for this rankingtransformation
For this ranking, correctly identifying derived business fields is part of producing usable structured data, not just a nice extra. (3 of 3 judges)
What was given, what came back
Test input: Bank Statement PDF · pdf · group: business-document-extraction
Input — what we sent
A four-page bank statement PDF with dense transaction tables, balance-forward bridges, account metadata, rewards data, and disclaimer text. It was used to stress schema-driven extraction, multi-page continuity, row completeness, and financial numerical accuracy.
Why this input is hard
- · Table extraction across 50+ transaction rows
- · Multi-page continuity with balance-forward bridges
- · Parsing structured account metadata alongside unstructured transaction descriptions
- · Numerical accuracy for deposits, withdrawals, running balances, and summaries
- · Extraction of nested rewards and disclaimer sections
Also checked on this input — same tool, 5 other criteria
Extraction Accuracy✓ WorkedCorrectly extracts the bank statement currency as INR and attaches high-confidence citation metadata, with extract confidence shown as 0.984.Extraction Accuracy✓ WorkedCorrectly extracts the account-holder/customer name from the bank statement and attaches a citation bounding box to the source text.Extraction Accuracy✗ FailedMiscomputes the bank summary aggregation by reporting total_transactions as 70 even though the report says the correct count is 40 after exclusions, so the financial summary is not reliable for reconciliation.Schema Adherence✓ WorkedPreserves a nested, schema-shaped JSON structure for the bank statement instead of flattening the document into raw OCR, including top-level objects like metadata, account_holder, account, branch, statement_period, transactions, and summary.Table & Record Completeness✗ FailedOver-produces the repeating transaction table, returning 53 transaction records when the bank statement contains 51, indicating row-count drift in line-item extraction.
Provenance
- Observation
- b786a10c-15a5-4f31-9a78-3508cf20cdf5
- Evidence run
- a061b9e7-a9c5-443d-a171-b296aaf51b8c
- Study
- Extract and query structured data from documents using natural language
- Research task
- 86b9y25e5
- Tested at
- not recorded
- Source
- first-party
- Evidence state
- verified
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "reducto",
scenario: "business-document-extraction"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 7 other tools
measured on Semantic Field Enrichment
Datalab◐ MixedClassifies bank-statement transaction_type inconsistently on merged rows: the 28 Jun record is labeled Deposit even though the visible row shows only a 399 withdrawal amount, and another merged row is labeled Deposit/Withdrawal.Extend AI✓ WorkedThe tool infers transaction direction from free text, classifying an ATM entry as `Withdrawal` and populating `withdrawal_amount: 1000`.Landing AI✓ WorkedClassifies extracted bank rows with semantic transaction_type labels such as Withdrawal and Deposit, showing derived type tagging beyond raw transaction text.LlamaParse✗ FailedThe tool does not reliably enrich bank transactions with derived identifiers or types: transaction_id and transaction_type are left empty across extracted records even when the descriptions contain transaction clues.Nanonets✗ FailedDoes not derive transaction_type on the bank statement at all; the report states that transaction_type is null for every extracted transaction.Retab✓ WorkedDerives transaction_type and transaction_id from the transaction narration, labeling the sample record as UPI with transaction_id 917615251879 instead of leaving only raw description text.Unstract✓ WorkedCorrectly enriches bank rows with transaction_type classification, with the report stating Withdrawal/Deposit was assigned correctly on all 51 transaction rows.
This evidence is published in
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com