Nanonets in Structured Document Extraction
Scenario-level performance from current published Results.
12 of 20 scenarios published
Where Nanonets failed
1 of 12 graded scenarios. Open a Result to see the conversation and the evidence.
The answer was never in the schema
Nanonets presented a document answer as extracted data without distinguishing its source. The first request correctly returned the receipt number, subtotal, tax and total due. The follow-up used the same extraction tool, placing the technician in a separate, similarly styled extraction block; the numbered source chip appeared only in the answer text below, which did not say the technician was outside the requested fields.
View Result →How Nanonets performed
Open a capability to explore its scenarios. Each row reports the test set in its published Result.
Field Extraction3 pass2 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The value is directly available | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedNanonets passed this single-field extraction task. The receipt number appeared once on the document and returned as RHS-70234 in both the preview table and exported JSON; the other returned fields also matched the document. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The value must be derived | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedNanonets derived the requested value from the two present inputs and returned 81 for total_fuel_cost. The two operand rows carried source-link icons in the screenshot and the total row did not, matching the JSON output and showing the value as a computed field. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The value needs a supplied definition | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedNanonets chose the consignment reference, not the invoice number, for this field. The result row and exported JSON both show CFS-CN-40821, matching the supplied definition. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The field is absent | No published result | ||
| The same field across layouts | No published result | ||
Nested & Repeated Fields3 pass1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The set contains a non-record | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedNanonets passed this lookalike-row scenario. It returned five genuine line items and left out the lookalike Total row; the input table shows that row in the same layout, and the output has no 'Total' entry or 1,982 amount. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The set continues across a page break | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedNanonets kept one array across the page break. The output has ten objects in one top-level array, with no second array and no repeated member at the boundary. The invoice shows items 1-6 on page 1 and 7-10 on page 2, and the result screen shows the same single table numbered 1 to 10. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The set is legitimately empty | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedNanonets returned the empty set the document called for. The input says no damage or exceptions were reported, and the output is an empty array; the capture shows the same field as List [0] with no rows. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A repeated set of records | No published result | ||
OCR1 pass1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A document mixing digital and scanned pages | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedNanonets handled the mixed digital-and-scanned document in this test. It returned the five requested values, including the two that appear only in the scanned page image, and kept the digital-page values unchanged. The JSON output itself does not show per-value page provenance. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A cleanly scanned document | No published result | ||
Querying1 pass1 fail1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| Filter records across documents | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedNanonets returned the matching documents for the tested cross-document filter. It produced the two receipts above $150 and left out the one below that threshold. The screenshot also shows a Data Extraction step over three files before the final JSON answer. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The answer was never in the schema | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedNanonets presented a document answer as extracted data without distinguishing its source. The first request correctly returned the receipt number, subtotal, tax and total due. The follow-up used the same extraction tool, placing the technician in a separate, similarly styled extraction block; the numbered source chip appeared only in the answer text below, which did not say the technician was outside the requested fields. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Aggregate across documents | No published result | ||
Review and Correction2 pass1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| Corrected data goes downstream | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedNanonets passes this scenario. In the tested case, the corrected receipt number appears in the product’s delivery path: the result grid shows RHS-99999, the Download menu is open on that grid, and the exported JSON carries RHS-99999 with the other receipt values. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Repair a record | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedNanonets let a user add one line item and remove another while preserving the other two rows. Both corrections persisted after a full page reload and appeared in the JSON export. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Correct a field value | No published result | ||
Source Grounding1 pass1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| Trace a value to its location | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedNanonets traced the returned subtotal to page 2, where GBP 6,103.00 is highlighted in the totals block. The exported page number was 2, and the preview highlight sat on the returned value rather than the neighboring totals. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The field is absent | No published result | ||
Training1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A field the tool can only get right from examples | No published result | ||
Reading these Results
Published evidence and test coverage answer different questions.
Which scenarios have a Result?
A published Result is public evidence for this tool on one scenario. “No published result” does not say whether testing has taken place.
What does each Result cover?
Assessed includes Pass, Fail and Not gradable. Gradable includes Pass and Fail. Both use the pinned test count in that published Result.
Out of 19 scenarios
The summary counts each of the 19 scenarios once. A scenario fails when any of its published test cases failed. “No published Result yet” means nothing is public for that scenario; it does not say whether Nanonets was tested on it.