Graded 7 November 2026
Does Extend AI keep equations as math markup or a clear fallback in Markdown?
Extend AI failed to keep equations as math markup or an honest fallback in Markdown. The returned Markdown turned equations (1) to (4) into garbled prose and never showed math markup or a stated fallback, even though the source equations were available in the PDF.
1 of 1 test case failed
Every test case in this scenario has a result.
Pass rate0%0 of 1 gradable
Coverage1 of 1test case with a result
0PassNo test case passed.
1FailMissed at least one thing it was expected to do; the reason and proof are with each test case below.
0Not gradableEvery result here could be graded.
0UntestedEvery test case in this scenario has a result.
The pass rate is a summary. The evidence is the test case below: what we sent, what we checked, what the tool returned, and the proof.
The test case
Each test case is judged on its own: Pass, Fail, Not gradable, or Untested. The scenario result above is based on this test case.
Equations — math markup or honest fallbackFailEvidence
What we sent
InputDocument (PDF)
What the tool returned
Markdownthe whole file · cited lines 21–27 and 31–49 marked
Tarnbeck Institute of Hydrology TIH/TN/18 # A one-dimensional model of tidal attenuation in the Braithe channel Technical note TIH/TN/18 . January 2026 R. K. Sandiman and H. Vale ### 1 Introduction The tide entering the Braithe is attenuated over the eleven kilometres between the bar and Braithe Bridge, and the attenuation is large enough to matter to anyone predicting water levels in the upper estuary. This note sets out the one-dimensional model the Institute has used since 2018, states the approximation on which the working formula rests, and compares the result with the gauge record at three stations. Nothing here is new. The intention is to have the derivation, the assumptions and the coefficients in one place, because the formula has been quoted in Partnership papers without them. ### 2 Governing equations Take the channel as a single reach of slowly varying cross-section and neglect lateral inflow. Conservation of mass gives JA OQ 2+ == 0 ot dx (1) where A is the cross-sectional area below the free surface and Q the discharge through it. Conservation of momentum, with friction represented in the Manning form, gives 2 C + +(A) + 84 On + 812010=0 A 0x AR 4/3 (2) in which n is the surface elevation above mean sea level, n is Manning's coefficient, R the hydraulic radius R = A/P for wetted perimeter P, and g the acceleration due to gravity. The friction term is the only nonlinear one that matters over the range of interest. ### 3 Attenuation of the leading harmonic For a channel of nearly uniform depth the leading semidiurnal harmonic decays close to exponentially with distance upstream, so that n(x, t) = noe-Hx cos (@t - kx) (3) with no the amplitude at the bar, @ the angular frequency of the harmonic, k the wavenumber and µ the attenuation coefficient. Linearising the friction term by the Lorentz method and collecting terms gives <page_number>Page 1 of 3</page_number> Tarnbeck Institute of Hydrology TIH/TN/18 ω χ 1/2 8 n2 c no , 4/3 C 2(1+ x2 x = 3Th (4) where c is the frictionless wave speed and h the mean depth. The second expression is the friction parameter; it is small in the lower reach and approaches unity above Harrowby Reach, which is where the exponential form begins to fail. ### 4 Application to the Braithe Taking the mean depth as eleven metres below the bar and four metres at the bridge, with Manning's coefficient at the value fitted in 2018, the formula reproduces the observed amplitude at Vardenne Quay to within four per cent and at Harrowby Reach to within nine. At Braithe Bridge it overestimates the amplitude, and the discrepancy grows through the spring tides, which is the behaviour the friction parameter predicts. A two-reach treatment with separate depths would probably remove most of the error at the bridge. It has not been attempted here because the gauge record above Harrowby Reach is too short to fit a second coefficient with any confidence. ### 5 Coefficients Manning's coefficient is not constant with depth in a channel of this kind, and the single fitted value used above is a compromise. Fitting the 2018 record with a depth-dependent form gives b n(h) = n% + 13 h 1/3 (5) with no the deep-water value and b a bed constant. The fit is better in the lower reach and no better at the bridge, which suggests that the error there is not in the friction term at all. The two constants were fitted to the 2018 record and have not been refitted since. Refitting on the four years now available would be worth doing; the Institute's expectation is that the deep-water value will move very little and the bed constant appreciably, because it is carrying the shallow reaches where the record has improved most. ### 6 Limitations Three limitations should be stated plainly. The first is the one-dimensional assumption, which fails where the channel divides above Harrowby Reach. The second is the neglect of the freshwater inflow, which is small in summer and not small after rain. The third is the linearisation itself, which is a poor approximation once the friction parameter approaches unity - that is, in exactly the reach where the model is least accurate. The error measure quoted in the previous section is <page_number>Page 2 of 3</page_number> Tarnbeck Institute of Hydrology TIH/TN/18 Mo - 1ml no (6) taken over the spring-neap cycle at each gauge, with no the observed amplitude and nm the modelled one. It is a crude measure and it flatters the model at the bar, where the amplitude is large and the absolute error is not. None of this is an argument against using the formula, which is quick, transparent and good enough for the purposes the Partnership puts it to. It is an argument for quoting it with the reach and the tidal range attached, which the papers that quote it have not always done. ### References Sandiman, R. K. (2018). Tidal propagation in the Braithe: a first fit. Tarnbeck Institute internal report TIH/IR/09. Vale, H. and Croyde, P. (2021). The Harrowby Reach gauge, 2015 to 2020. Journal of Estuarine Hydrology 44, 218-31. Prosser, A. (2009). Friction coefficients for the Braithe channel. Braithe Harbour Board, unpublished. <page_number>Page 3 of 3</page_number>
Markdown output: equations rendered as prose
Copied from Output file (Markdown), lines 21–27, 31–49
Open the Markdown file ↗Expected vs. Found
| ✗ | Expected: Equations must appear as math markup or a clear fallback, not garbled prose.Found: It rendered equations (1) to (4) as garbled prose, with no math markup or fallback. |
Supporting proof
Proof 1Output file (JSON)
Cited values3 values from this file
- parse_run › config
- { "engine": "parse_performance", "engineVersion": "2.0.0" }
- parse_run › status
- PROCESSED
- parse_run › file
- { "object": "file", "id": "file_uKfbtFEODIMrHQbycCfnD", "name": "technical_note_TIH-TN-18.pdf", "type": "PDF", "parentFileId": null, "metadata": { "pageCount": 3 }, "dataRetention": { "mode": "workspace_default", "status": "available" }, "createdAt": "2026-09-09T12:41:02.241Z", "updatedAt": "2026-09-09T12:41:11.355Z" }
Show the file, long text shortened (73 KB)
JSONlong text shortened; the file holds every character
{ "parse_run": { "object": "parse_run", "id": "pr_Zm6I3XPWOxLzEmVPnRBX1", "file": { "object": "file", "id": "file_uKfbtFEODIMrHQbycCfnD", "name": "technical_note_TIH-TN-18.pdf", "type": "PDF", "parentFileId": null, "metadata": { "pageCount": 3 }, "dataRetention": { "mode": "workspace_default", "status": "available" }, "createdAt": "2026-09-09T12:41:02.241Z", "updatedAt": "2026-09-09T12:41:11.355Z" }, "status": "PROCESSED", "failureReason": null, "failureMessage": null, "metadata": null, "dataRetention": { "mode": "workspace_default", "status": "available" }, "output": { "chunks": [ { "object": "chunk", "id": "chunk_1_URs01B", "type": "page", "content": "Tarnbeck Institute of Hydrology\n\nTIH/TN/18\n\n# A one-dimensional model of tidal attenuation in the Braithe channel\n\nTechnical note TIH/TN/18 . January 2026\n\nR. K. Sandiman and H. Vale\n\n### 1 Introduction\n\nThe tide entering the Braithe is att … [1,926 characters in the file]", "metadata": { "pageRange": "{… 2 fields in the file}", "minOcrConfidence": 0.061, "avgOcrConfidence": 0.954 }, "blocks": [ "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}" ] }, { "object": "chunk", "id": "chunk_2_3a1ptV", "type": "page", "content": "Tarnbeck Institute of Hydrology\n\nTIH/TN/18\n\nω χ 1/2 8 n2 c no , 4/3 C 2(1+ x2 x = 3Th\n\n(4)\n\nwhere c is the frictionless wave speed and h the mean depth. The second expression is the friction parameter; it is small in the lower reach and app … [2,339 characters in the file]", "metadata": { "pageRange": "{… 2 fields in the file}", "minOcrConfidence": 0.101, "avgOcrConfidence": 0.984 }, "blocks": [ "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}" ] }, { "object": "chunk", "id": "chunk_3_jSz1Pv", "type": "page", "content": "Tarnbeck Institute of Hydrology\n\nTIH/TN/18\n\nMo - 1ml no\n\n(6)\n\ntaken over the spring-neap cycle at each gauge, with no the observed amplitude and nm the modelled one. It is a crude measure and it flatters the model at the bar, where the ampl … [954 characters in the file]", "metadata": { "pageRange": "{… 2 fields in the file}", "minOcrConfidence": 0.296, "avgOcrConfidence": 0.977 }, "blocks": [ "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}", "{… 8 fields in the file}" ] } ], "metadata": { "originalMimeType": "application/pdf", "finalMimeType": "application/pdf", "pages": [ { "number": 1, "rotationApplied": 0, "originalPageWidth": 596, "originalPageHeight": 842, "dpi": 150 }, { "number": 2, "rotationApplied": 0, "originalPageWidth": 596, "originalPageHeight": 842, "dpi": 150 }, { "number": 3, "rotationApplied": 0, "originalPageWidth": 596, "originalPageHeight": 842, "dpi": 150 } ] } }, "outputUrl": null, "metrics": { "processingTimeMs": 8113, "pageCount": 3 }, "config": { "engine": "parse_performance", "engineVersion": "2.0.0" }, "batchId": null, "usage": { "credits": 6, "totalCredits": 6, "breakdown": [ { "object": "parse_run", "id": "pr_Zm6I3XPWOxLzEmVPnRBX1", "credits": 6, "charges": ["{… 4 fields in the file}"] } ] } }, "file": { "object": "file", "id": "file_uKfbtFEODIMrHQbycCfnD", "name": "technical_note_TIH-TN-18.pdf", "type": "PDF", "parentFileId": null, "presignedUrl": null, "contents": null, "metadata": {}, "dataRetention": { "mode": "workspace_default", "status": "available" }, "createdAt": "2026-09-09T12:41:02.241Z", "updatedAt": "2026-09-09T12:41:02.283Z" }, "figure_assets": [] }
The whole file (73 KB) is too large to print here; open it from its link below.
Open the JSON file ↗Proof 2Screenshot
Why this result
The default parse had to preserve equations as markup or a clear fallback. Its Markdown instead turned equations (1) to (4) into garbled prose with no markup or fallback, so the required behaviour was not met.
Tested by Chandresh Bisht · evidence dated 9 September 2026
Configuration and setup
How this tool was set up for the run and what the test needed in place. Each row is a fact from the run's records; a fact the records do not hold is left out, not guessed.
Software that produced the output
Extend AI
Build or version
x-extend-api-version 2026-02-09; engine parse_performance 2.0.0
Plan or tier
free tier (10,000 starter credits, Pay As You Go)
Surface
API
Set up before the run
A PDF with mathematical equations had to be converted to Markdown in one default parse pass, with the complete file sent and no overrides.
Formula options
No block options or formula option set (the product’s defaults)
Tested
9 September 2026 · Chandresh Bisht
Connected to
How this scenario is graded
How we decide Pass, Fail and Not gradable. The same rules apply to every tool tested on this scenario.
How results are decided
Each test case gets one result per tool: Pass, Fail or Not gradable. A test case we haven't run yet shows Untested. There are no partial results.
The rules
- Pass: the tool did everything the test expected, and nothing it said contradicts the correct answer.
- Fail: at least one expected behaviour clearly didn't happen; the test case's result says which one and quotes the tool.
- Not gradable: our evidence couldn't settle the outcome (for example a record we needed is missing). It is never counted as a fail, and the test case's result says what's missing.
Where this sits in the benchmark
This page is one cell of a larger study: one tool, one scenario. Only this benchmark's frame appears here.
| Level | Name | Scope |
|---|---|---|
| Benchmark | Converting a complex PDF into clean Markdown with a hosted API → | 12 scenarios · 13 tools |
| Capability | Equations & Mathematical Notation → | |
| Scenario | A document containing mathematical equations → | |
| Tool | Extend AI → |
Global scenario definition → · Global capability definition → · Extend AI product page →
History of this result
What has happened to this result since it was first published. Runs and grades are never overwritten: a retest or a re-grade publishes a new result and keeps the earlier one readable.
11 November 2026First publishedConverting a complex PDF into clean Markdown with a hosted API v1
Act on this result
Nothing filed here edits the run or the grade. A challenge opens a review, and a review can produce a new run or a re-grade — which becomes the current result and leaves this one in the history.
This matches what I see
You run the same kind of test against your own setup and get the same behaviour.
Agree →This does not match
Yours behaves differently. Tell us what you got, with a screenshot if you have one.
Disagree →Point out an issue
Something here is wrong — a reference value, a transcription, a grade.
Report an issue →We have fixed this
Tell us what changed and we schedule a rerun of the failing test case. The old result stays as history.
Vendor notice →Filed against this evidenceNothing yet. Challenges, counter-evidence and fix notices appear here with their outcome, and stay on the page after they are resolved.
Cite this result
aidemos.com/benchmarks/pdf-to-markdown-apis/results/extend-ai/a-document-containing-mathematical-equations · 0 pass · 1 fail · coverage 1/1 · graded 2026-11-07
The same record is available as structured data through the AI Demos MCP server, with the counts, the coverage and every per-test-case reason carried as fields.
Verify the proof files
These files support this result. Open a file to inspect the original evidence.
Input · Document (PDF) ↗application/pdf · 236 KB
Proof 1 · Output file (JSON) ↗application/json · 73 KB
Output file (Markdown) ↗text/markdown · 5 KB
Proof 2 · Screenshot ↗image/png · 437 KB
File fingerprints (SHA-256)
A fingerprint identifies the exact file used for this result.
Input · Document (PDF)f3d69b6d9d2191c5806a426fd4ea5e209dcd7cf693ad6abe67a5d18a014b9fd6
Proof 1 · Output file (JSON)b3f30c18c68ab0cc01b4cad45b984fab8ecc04af28e27cd5f5001d18ccddc881
Output file (Markdown)2e40dc8c948e731a03e3f6d8a748488a2cc1dcdd855c20607b2a2ce2f5f9f16f
Proof 2 · Screenshotc5b9d6813637805e5ccc96e0fb34efac37ee700b75296cfc2e85829cdaa91f60