Skills were returned as one unstructured, space-separated string with no array structure or delimiters, making them hard to split programmatically.
What was measured
Output quality
Is the parsed result clean, complete, and usable overall?
decisive for this rankingtransformation
The whole point is usable structured extraction; clean, complete, usable output is the core measure of success. (3 of 3 judges)
What was given, what came back
Test input: Messy real-world resume — John Kumar · pdf · group: resume-parsing
Input — what we sent
A poorly formatted, inconsistent real-world resume for John Kumar, used to test robustness against noisy structure, inconsistent dates, and mixed-content sections.
Why this input is hard
- · messy formatting robustness
- · section detection fallback
- · inconsistent date parsing
- · soft-skills extraction
- · noise and hallucination control
Output — unretouched
Also checked on this input — same tool, 3 other criteria
Accuracy✗ FailedThe education CGPA value was wrong: the tool returned 67, which the report says is the percentage score rather than the actual CGPA.Field coverage⚠ StruggledOnly one education record was extracted, and the 12th Science and 10th std entries were omitted from the output.Messy resume handling✓ WorkedThe parser still accepted a highly inconsistent resume with missing headers and mixed date formats, and it continued parsing through the messy structure.
Provenance
- Observation
- fcafc622-fe49-4c1a-88e4-ecbbfb5ef7d5
- Evidence run
- cbbef4db-964c-49fa-a57f-a2977822bdfc
- Study
- Parse resumes into structured data using an API
- Research task
- 86b9jm30n
- Tested at
- not recorded
- Source
- first-party
- Evidence state
- verified
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "parseur",
scenario: "resume-parsing"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 5 other tools
measured on Output quality
Airparser◐ MixedProduces usable JSON from noisy text, but the skills section collapses into one long string and one education record keeps an unnormalized marks value.Extracta.ai✓ WorkedProduces a usable JSON result even on the messy resume, with the important sections still readable and structured.HrFlow◐ MixedThe messy-resume output is moderate overall: contact and basic structure survive, but the result is incomplete for soft skills, certifications, and task detail.LlamaParse✓ WorkedThe messy-resume output was the richest and cleanest among the tested tools, with all 3 education entries, all 14 skills, both certifications, hobbies, and a boolean references field.Skima AI⚠ StruggledThe messy-resume output degraded sharply: responsibilities became a run-on string, skills collapsed into one concatenated line, and references/certifications/hobbies were missing.
This evidence is published in
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com