Processes the full 84-page mixed-content report in 25.5 seconds with no truncation and keeps the hardest nested table’s 27 values exact, but 10 footer numbers are corrupted in the back two-thirds of the file.
What was measured
Complex Document Handling
Maintains quality across long, multi-section, and mixed-content documents without degradation.
decisive for this rankingtransformation
The subject explicitly says complex PDF, so sustained quality across long, mixed-content documents is central to the tool’s ability to do the job. (3 of 3 judges)
What was given, what came back
Test input: Hybrid Earnings Report · pdf · group: hybrid-earnings-report
Input — what we sent
Hybrid Earnings Report
A long, real-world hybrid annual report used to benchmark end-to-end PDF-to-markdown conversion. It combines native text, financial tables, charts/graphics, and scanned signature/stamp regions, stressing preservation of document structure, reading order, and embedded visual content across a multi-section report.
Why this input is hard
- · Native digital text extraction
- · Complex financial table preservation
- · Chart and graphic retention
- · Image-based signatures and stamps
- · Reading order across a long multi-section report
- · Markdown quality and consistency
Also checked on this input — same tool, 11 other criteria
Advanced Features✗ FailedOn the bar-chart schema, extract.run gets only 3 of 5 Sales values right: 2011 and 2012 are silently pulled from a different table ($69.9B and $73.3B), with no confidence or warning signal.Advanced Features◐ MixedThe parse pipeline’s confidence field catches all 10 footer-number corruptions and also produces one false alarm on a correct securities table, so the signal is real but not perfectly calibrated.Advanced Features✓ WorkedThe separate schema-driven extract.run endpoint cleanly recovers both a financial table and a donut chart: all requested table rows and all five donut percentages come back exactly.Markdown Quality⚠ StruggledProduces usable but not clean markdown: the output contains 12 <signature> tags, 2 <empty> placeholders, and literal HTML tags such as <b>, <i>, and <u> instead of pure CommonMark.Reading Order & Structure✓ WorkedLinearizes a two-column page correctly, reading the full left column, including all four bullets, before the right-column continuation appears, with no interleaving or sequence inversion.Reading Order & Structure⚠ StruggledDrops heading structure on most of the filing: the FORM 10-K opening area receives no # or ## markup, and the report says 80 of 84 pages stay as plain paragraphs rather than navigable section headings.Table Preservation✓ WorkedPreserves a genuinely row-paired 52-row state table as a clean grid, keeping the 1,792 stores total and 239,539 retail-square-footage total intact.Table Preservation✗ FailedMis-merges two independent side-by-side lists into one table, forcing unrelated cost-of-sales and SG&A bullets into fake row pairs where no row correspondence exists in the source.Text & OCR Completeness✓ WorkedConverts the full 84-page native-digital annual report with no skipped pages; the output contains all 84 page-marker pairs, and checked high-risk numbers survive exactly, including the $51,550,988,273 market-value figure and 599,982,121 shares outstanding.Visual Content Retention✓ WorkedRetains visual content as real image outputs when requested: the portrait, donut chart, and bar chart are all returned as preserved figure/image crops rather than being dropped.Visual Content Retention✗ FailedDoes not retain handwritten signature graphics as images: a signature crop returns only the <signature> placeholder plus the printed name/title, omitting the cursive mark itself.
Provenance
- Observation
- 1961fa7a-2d5a-4ece-83fe-23b4c0399568
- Evidence run
- 6e3160de-fe46-4b45-b071-72560b5c5d0e
- Study
- Convert a Complex PDF into Clean Markdown with an API
- Research task
- 86b9h7t37
- Tested at
- not recorded
- Source
- first-party
- Evidence state
- verified
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "reducto",
scenario: "hybrid-earnings-report"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 8 other tools
measured on Complex Document Handling
Adobe API✓ WorkedProcesses a long mixed-content annual report in one automated run, keeping tables, charts, and scanned signature pages in the extracted output.Extend AI✓ WorkedProcesses an 84-page hybrid annual report end-to-end across native text, tables, charts, and scan-only signature regions without manual cleanup.LlamaParse✓ WorkedHandles an 84-page mixed-content annual report end-to-end and reaches SUCCESS in the Results view without manual correction.Mistral AI✓ WorkedThe tool handles an 84-page mixed-content annual report end-to-end, including tables, charts, and scanned signature/stamp regions, and returns usable markdown output without manual correction.Nutrient.io✓ WorkedHandles an 84-page mixed-content annual report end to end, including native text, tables, charts, and scanned signature pages, and returns a markdown export.PDFVector✓ WorkedHandled an 84-page hybrid annual report without errors.Tensorlake✓ WorkedProcesses an 84-page mixed-content annual report without collapsing the hierarchy, keeping text, tables, charts, and scanned signatures usable within the extracted workflow.Upstage AI◐ MixedHandles an 84-page hybrid report end to end, but quality degrades on signatures and multicolumn pages rather than staying uniform across the document.
This evidence is published in
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com