Firecrawl in Converting a complex PDF into clean Markdown with a hosted API
Scenario-level performance from current published Results.
8 of 12 scenarios published
Where Firecrawl failed
5 of 8 graded scenarios. Open a Result to see the conversation and the evidence.
A document containing a code block
Firecrawl did not preserve the full structure of the document’s code blocks. It fenced both snippets and retained their code characters, but removed their indentation.
View Result →A document with a data chart
Firecrawl did not preserve the data chart in this PDF-to-Markdown conversion. Its default output omitted both the chart image and its title.
View Result →A document with figures and captions
Firecrawl did not preserve the figure with its caption in the converted document. It kept the survey-plan caption in position but returned no image.
View Result →A page laid out in multiple columns
Firecrawl did not consistently preserve column reading order. In a three-page bulletin, it merged a left-column heading into a right-column heading on page 1; the remaining text followed the intended column order.
View Result →A cleanly scanned document
Firecrawl did not recover visible text accurately from the cleanly scanned document. The output dropped the letterhead and footer, split words apart, interleaved the opening prose, kept the table values correct, and used full-width punctuation for the sampling time.
View Result →How Firecrawl performed
Open a capability to explore its scenarios. Each row reports the test set in its published Result.
Code Extraction1 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A document containing a code block | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedFirecrawl did not preserve the full structure of the document’s code blocks. It fenced both snippets and retained their code characters, but removed their indentation. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Figures & Charts2 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A document with a data chart | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedFirecrawl did not preserve the data chart in this PDF-to-Markdown conversion. Its default output omitted both the chart image and its title. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A document with figures and captions | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedFirecrawl did not preserve the figure with its caption in the converted document. It kept the survey-plan caption in position but returned no image. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Heading & Section Structure1 pass1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A document with styled headings and subheadings | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedFirecrawl passes this styled-heading conversion case. The returned Markdown keeps the three heading tiers as H1, H2 and H3 in the same order and nesting, and the letterhead stays plain text. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| Footnotes at the bottom of the page | No published result | ||
Reading Order & Layout1 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A page laid out in multiple columns | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedFirecrawl did not consistently preserve column reading order. In a three-page bulletin, it merged a left-column heading into a right-column heading on page 1; the remaining text followed the intended column order. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Scanned Document OCR1 pass1 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A cleanly scanned document | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedFirecrawl did not recover visible text accurately from the cleanly scanned document. The output dropped the letterhead and footer, split words apart, interleaved the opening prose, kept the table values correct, and used full-width punctuation for the sampling time. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A document mixing digital and scanned pages | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedFirecrawl processed the scanned page in a mixed digital and scanned PDF. It returned the worksheet in document order rather than skipping it. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Table Extraction1 pass1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A simple, clearly formatted table | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedFirecrawl preserved the structure of the simple table in this test. It returned all four headers and eight rows, with the values under the correct headers. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A table that continues across a page break | No published result | ||
Equations & Mathematical Notation1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A document containing mathematical equations | No published result | ||
Text Fidelity1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| An ordinary digital text document | No published result | ||
Reading these Results
Published evidence and test coverage answer different questions.
Which scenarios have a Result?
A published Result is public evidence for this tool on one scenario. “No published result” does not say whether testing has taken place.
What does each Result cover?
Assessed includes Pass, Fail and Not gradable. Gradable includes Pass and Fail. Both use the pinned test count in that published Result.
Out of 12 scenarios
The summary counts each of the 12 scenarios once. A scenario fails when any of its published test cases failed. “No published Result yet” means nothing is public for that scenario; it does not say whether Firecrawl was tested on it.