LlamaParse in Structured Document Extraction
Scenario-level performance from current published Results.
11 of 20 scenarios published
Where LlamaParse failed
1 of 11 graded scenarios. Open a Result to see the conversation and the evidence.
Corrected data goes downstream
LlamaParse did not carry the corrected receipt_number RHS-99999 downstream on the Extract screen. On the Free plan, the agentic four-field setup stayed read-only, and the downloaded JSON and opened export still showed receipt_number RHS-70234 with subtotal, tax, and total_due.
View Result →How LlamaParse performed
Open a capability to explore its scenarios. Each row reports the test set in its published Result.
Field Extraction3 pass2 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The field is absent | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedLlamaParse passed this absent-field case by leaving the missing loyalty field empty and ungrounded. The receipt has no loyalty wording, the extractor returned an empty value for that field, and the job record gave that field no citation metadata while the present receipt number had a citation. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The same field across layouts | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedLlamaParse passed this scenario. It returned the correct identifier, subtotal, tax, and total_due values for all three differently laid out documents, and the three outputs matched the values read from the pages while keeping the same four-field schema. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The value must be derived | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedLlamaParse passed this derived-value task. It read the two page-1 operands and returned £81.00; the derived field had no citation, and the source contains no 81.00. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The value is directly available | No published result | ||
| The value needs a supplied definition | No published result | ||
Nested & Repeated Fields3 pass1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A repeated set of records | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedLlamaParse handled the repeated set correctly. It returned all three records with the right values, and the per-value citations stayed in separate rows, so nothing was bound to the wrong record. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The set continues across a page break | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedLlamaParse kept all ten invoice line items together in one array across the page break. It retained the six items from page one and four from page two, with no boundary duplicate and all 40 values correct. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The set is legitimately empty | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedLlamaParse passed this empty-set extraction case. It returned an empty required array for the damage-exceptions field, with the source page saying nothing was reported and no invented member or marker text added. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The set contains a non-record | No published result | ||
OCR2 pass
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A cleanly scanned document | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedLlamaParse passed this OCR baseline. It returned the same invoice values for the digital file and the image-only twin, including the page 2 totals, and the twin had no text layer. The two runs differed only in job details such as timestamps and confidence flags. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A document mixing digital and scanned pages | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedLlamaParse extracted the requested values from a document mixing digital and scanned pages. It returned all five values correctly, including two found only on the scanned page. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Review and Correction1 fail2 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| Corrected data goes downstream | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedLlamaParse did not carry the corrected receipt_number RHS-99999 downstream on the Extract screen. On the Free plan, the agentic four-field setup stayed read-only, and the downloaded JSON and opened export still showed receipt_number RHS-70234 with subtotal, tax, and total_due. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Correct a field value | No published result | ||
| Repair a record | No published result | ||
Source Grounding2 pass
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The field is absent | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedLlamaParse passed this absent-field case by leaving the missing loyalty field empty and ungrounded. The receipt has no loyalty wording, the extractor returned an empty value for that field, and the job record gave that field no citation metadata while the present receipt number had a citation. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Trace a value to its location | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedLlamaParse traced the value to the right place in the document. The value appears once in the two-page invoice; the citation names page 2 and the viewer box lands on the value. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Querying3 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| Aggregate across documents | No published result | ||
| Filter records across documents | No published result | ||
| The answer was never in the schema | No published result | ||
Training1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A field the tool can only get right from examples | No published result | ||
Reading these Results
Published evidence and test coverage answer different questions.
Which scenarios have a Result?
A published Result is public evidence for this tool on one scenario. “No published result” does not say whether testing has taken place.
What does each Result cover?
Assessed includes Pass, Fail and Not gradable. Gradable includes Pass and Fail. Both use the pinned test count in that published Result.
Out of 19 scenarios
The summary counts each of the 19 scenarios once. A scenario fails when any of its published test cases failed. “No published Result yet” means nothing is public for that scenario; it does not say whether LlamaParse was tested on it.