The messy-resume output degraded sharply: responsibilities became a run-on string, skills collapsed into one concatenated line, and references/certifications/hobbies were missing.
What was measured
Output quality
Is the parsed result clean, complete, and usable overall?
decisive for this rankingtransformation
The whole point is usable structured extraction; clean, complete, usable output is the core measure of success. (3 of 3 judges)
What was given, what came back
Test input: Messy real-world resume — John Kumar · pdf · group: resume-parsing
Input — what we sent
A poorly formatted, inconsistent real-world resume for John Kumar, used to test robustness against noisy structure, inconsistent dates, and mixed-content sections.
Why this input is hard
- · messy formatting robustness
- · section detection fallback
- · inconsistent date parsing
- · soft-skills extraction
- · noise and hallucination control
Output — unretouched
Also checked on this input — same tool, 3 other criteria
Field coverage✓ WorkedEven on the messy resume, the parser still returned the core coverage set of name, email, phone, work experience, education, and skills.Messy resume handling⚠ StruggledThe tool accepted the messy file, but it degraded badly in structure by merging bullets into a run-on responsibility line instead of preserving clean formatting.Noise in output✗ FailedThe messy-resume skills block was collapsed into a single concatenated string with no separators, making the field hard to reuse downstream.
Provenance
- Observation
- f3277ca4-0119-47a4-a0b2-fb948ba0dbd9
- Evidence run
- cbbef4db-964c-49fa-a57f-a2977822bdfc
- Study
- Parse resumes into structured data using an API
- Research task
- 86b9jm30n
- Tested at
- not recorded
- Source
- first-party
- Evidence state
- verified
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "skima-ai",
scenario: "resume-parsing"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 5 other tools
measured on Output quality
Airparser◐ MixedProduces usable JSON from noisy text, but the skills section collapses into one long string and one education record keeps an unnormalized marks value.Extracta.ai✓ WorkedProduces a usable JSON result even on the messy resume, with the important sections still readable and structured.HrFlow◐ MixedThe messy-resume output is moderate overall: contact and basic structure survive, but the result is incomplete for soft skills, certifications, and task detail.LlamaParse✓ WorkedThe messy-resume output was the richest and cleanest among the tested tools, with all 3 education entries, all 14 skills, both certifications, hobbies, and a boolean references field.Parseur⚠ StruggledSkills were returned as one unstructured, space-separated string with no array structure or delimiters, making them hard to split programmatically.
This evidence is published in
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com