AI Demos Research — the structured-intelligence platform. Every verdict on these pages opens to the execution behind it.
Graded 16 November 2026

Does LlamaParse preserve styled heading hierarchy in Markdown?

LlamaParse turned every heading into a Markdown heading but kept only two of the source's three heading levels. On page 1, the subsections Chain of custody and Condition on arrival came out at the same level as their parent, Reception, and the third-tier headings shared a level with the second. A separate run with a custom prompt asking for three heading levels did not preserve the hierarchy either.

1 of 1 test case failed

Every test case in this scenario has a result.

Pass rate0%0 of 1 gradable
Coverage1 of 1test case with a result
0PassNo test case passed.
1FailMissed at least one thing it was expected to do; the reason and proof are with each test case below.
0Not gradableEvery result here could be graded.
0UntestedEvery test case in this scenario has a result.
The pass rate is a summary. The evidence is the test case below: what we sent, what we checked, what the tool returned, and the proof.

The test case

Each test case is judged on its own: Pass, Fail, Not gradable, or Untested. The scenario result above is based on this test case.

Styled headings — hierarchy preservedFailEvidence
What we sent

We uploaded the complete three-page laboratory procedure to LlamaParse's conversion API in one request.

InputDocument (PDF)
Input PDF: the three-page lab procedure with headings set at three sizes
Open the PDF ↗
What the tool returned

Markdown output: headings in two levels; Chain of custody and Condition on arrival share Reception's level

Copied from Output file (Markdown), lines 9–17

Markdown
## Reception

Samples are received at reception between 08:00 and 17:30 on weekdays and between 09:00 and 12:00 on Saturdays. Outside those hours clients who hold a key may use the refrigerated drop box at the rear entrance.

## Chain of custody

Every delivery arrives with a completed submission form. Reception staff check that the form is signed, that the number of containers matches the number listed on it, and that each container carries a legible client reference. The form is signed and timed on receipt and the top copy is returned to the person making the delivery.

## Condition on arrival
Open the Markdown file ↗

API response: the completed job at the default tier, with all three pages processed

Copied from Captured file (JSON), job › status

Text
COMPLETED
Open the whole JSON file ↗
Expected vs. Found
✗Expected: Headings become Markdown headings with the source's heading hierarchy kept, not turned into plain or bold text.Found: All headings became Markdown headings, but three source levels became two: Chain of custody and Condition on arrival sit at Reception's level; both third-tier headings share the second tier's level.
Supporting proof
Proof 1Screenshot
Page 1 of 3, source beside output: two subsections come out at Reception's level
Open original ↗
Proof 2Screenshot
Page 2 of 3, source beside output: subsections nest correctly under their parents
Open original ↗
Proof 3Screenshot
Page 3 of 3, source beside output: five same-size headings at one level
Open original ↗
Proof 4Screenshot
Run details card: the default setup, the complete PDF sent and the completed run
Open original ↗
Cited values2 values from this file
job › tier
agentic
job › name
procedure_KAL-SP-06_sample_reception.pdf
Why this result

Every heading became a Markdown heading, but three source levels came back as two: two subsections sit at their parent's level, while the same construction under Storage nests correctly. A custom-prompt run asking for three levels still did not keep them and promoted the title block to a heading.

Also observed in this test case

Title lines left as text. The three page-1 title lines, including Sample Reception and Handling, are set in body type in the source and come out as plain text, not as headings.

Body text complete. Every one of the source's 1,255 body words comes back in order; the only additions are two page-break rules.

Tested by Chandresh Bisht · evidence dated 10 September 2026

Configuration and setup

How this tool was set up for the run and what the test needed in place. Each row is a fact from the run's records; a fact the records do not hold is left out, not guessed.

Software that produced the output
LlamaParse
Build or version
Parse API v2, tier agentic, version latest (build not exposed)
Plan or tier
free tier
Surface
API
Set up before the run
A three-page laboratory procedure with headings at three sizes over body text; on page 1, Reception has two smaller subsections, Chain of custody and Condition on arrival. The complete PDF was to be converted in one default pass.
Tested
10 September 2026 · Chandresh Bisht

How this scenario is graded

How we decide Pass, Fail and Not gradable. The same rules apply to every tool tested on this scenario.

How results are decided

Each test case gets one result per tool: Pass, Fail or Not gradable. A test case we haven't run yet shows Untested. There are no partial results.

The rules
  • Pass: the tool did everything the test expected, and nothing it said contradicts the correct answer.
  • Fail: at least one expected behaviour clearly didn't happen; the test case's result says which one and quotes the tool.
  • Not gradable: our evidence couldn't settle the outcome (for example a record we needed is missing). It is never counted as a fail, and the test case's result says what's missing.

Where this sits in the benchmark

This page is one cell of a larger study: one tool, one scenario. Only this benchmark's frame appears here.

History of this result

What has happened to this result since it was first published. Runs and grades are never overwritten: a retest or a re-grade publishes a new result and keeps the earlier one readable.

from the publication record
17 November 2026First publishedConverting a complex PDF into clean Markdown with a hosted API v1

Act on this result

Nothing filed here edits the run or the grade. A challenge opens a review, and a review can produce a new run or a re-grade — which becomes the current result and leaves this one in the history.

This matches what I see

You run the same kind of test against your own setup and get the same behaviour.

Agree →
This does not match

Yours behaves differently. Tell us what you got, with a screenshot if you have one.

Disagree →
Point out an issue

Something here is wrong — a reference value, a transcription, a grade.

Report an issue →
Request a retest

On a newer build, a larger dataset, or your own setup.

Request a retest →
We have fixed this

Tell us what changed and we schedule a rerun of the failing test case. The old result stays as history.

Vendor notice →
Filed against this evidenceNothing yet. Challenges, counter-evidence and fix notices appear here with their outcome, and stay on the page after they are resolved.
Cite this result
aidemos.com/benchmarks/pdf-to-markdown-apis/results/llamaparse/a-document-with-styled-headings-and-subheadings · 0 pass · 1 fail · coverage 1/1 · graded 2026-11-16

The same record is available as structured data through the AI Demos MCP server, with the counts, the coverage and every per-test-case reason carried as fields.

Verify the proof files

These files support this result. Open a file to inspect the original evidence.

Input · Document (PDF) ↗application/pdf · 43 KB
Output file (Markdown) ↗text/markdown · 7 KB
Proof 1 · Screenshot ↗image/png · 547 KB
Proof 2 · Screenshot ↗image/png · 613 KB
Proof 3 · Screenshot ↗image/png · 347 KB
Captured file (JSON) ↗application/json · 18 KB
Proof 4 · Screenshot ↗image/png · 67 KB
File fingerprints (SHA-256)

A fingerprint identifies the exact file used for this result.

Input · Document (PDF)1d53151023b44fca82503e6e748e2a1bed0b2374f54d2dc2d10279009401f34d
Output file (Markdown)aeaa9c2e9ccde8de2687a24542f37c80546f86fb03bfea59566062c28547ac29
Proof 1 · Screenshot41a8cca2b1582aa3d20ba538ece1347da07a91d313e6ec01a601aa316dc18f43
Proof 2 · Screenshot5e1ef61cf0a7a4b112761ce836a2790c2177e8df66a785bc5e7566bdeb0091c5
Proof 3 · Screenshot227bc07a2c27f1964c998afd416559bae3fc50337dbc43bb0cdb5708f66e5d4b
Captured file (JSON)61452ca7857428cbd531ee49e5731e5e2f66491eeb4cbd9b00f4e5aa96dee0ae
Proof 4 · Screenshot8a95215576710c6d371ace24d69ca27a6f91e95a85ae804258e6ffa2ae01070f