Evidence · first-party tested/Best AI Tools to Scrape Web Pages Into Clean Markdown or Structured Data
Captures hydrated product content but leaves substantial non-content noise in the output, including raw backend code artifacts and raw media-attachment matrices.
What was measured
JS DOM Hydration
How well the tool waits for and captures content rendered by client-side JavaScript on dynamic pages.
decisive for this rankingtransformation
Capturing JavaScript-rendered content is a core scraping capability for modern web pages. (3 of 3 judges)
What was given, what came back
Test input: Nike Air Force 1 '07 size options extraction · mixed
Input — what we sent
Input, verbatim
https://www.nike.com/t/air-force-1-07-mens-shoes-jBrhbr/CW2288-111 — Wait for the size selection options to fully render. Extract the product name, price, and a list of all available shoe sizes.
A Nike product page with client-side JavaScript hydration used to test whether a headless scraper waits for dynamic DOM content before extracting product details and all available shoe sizes.
Output — unretouched
Firecrawl scrape result for a Nike product page in Markdown format.
Also checked on this input — same tool, 5 other criteria
Input Handling✓ WorkedAccepted the dynamic Nike product URL through the standard scrape path and started extraction without routing or access errors.Interaction Stability✓ WorkedReliably waited for client-side hydration and captured the complete size set, from M 5 / W 6.5 through M 18 / W 19.5.Output Quality◐ MixedCaptures the title, pricing, and inventory layout, but the output still includes raw backend code artifacts, media-matrix noise, and uncleaned global link trees.Visual Spatial Awareness⚠ StruggledThe output kept the target product data but failed to clean surrounding page chrome, packing in global localization links, background asset tags, and raw image URL trees.Visual Spatial Awareness◐ MixedCan surface the hydrated inventory information, but does not clean surrounding layout noise such as localization links, background asset tags, and raw image URL trees.
Provenance
- Observation
- 539003d1-a873-4167-8a3e-5427f4a36e74
- Evidence run
- 06e1dbd6-5518-4af8-aa1a-735259a75b4f
- Study
- Scrape Web Pages Into Clean Markdown or Structured Data Using AI
- Research task
- 86b9jm3a3
- Tested at
- Jun 23, 2026
- Source
- first-party
- Evidence state
- verified
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "firecrawl"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 3 other tools
measured on JS DOM Hydration
Jina AI Reader✗ FailedIt did not capture the client-rendered size selector; after 7.0 s the extract still lacked the live size grid and showed only static page content.Skyvern✓ WorkedWaits for the client-rendered product page to hydrate and extracts the size grid instead of stopping at the initial shell.Spider✗ FailedDoes not wait for client-side hydration long enough; it can capture the title and price but misses the size-selection UI entirely, leaving zero available sizing attributes.
This evidence is published in
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com