The set contains a non-record
A repeated set contains an explicitly identified non-record that resembles a member but must be excluded.
What this scenario means
This scenario checks whether the tool can separate genuine members of a repeated set from an item that looks like one but is not part of the set. It is hard because shape and position can make a non-record look valid, so a weak extractor may keep it or lose a real member. A good agent returns only the genuine members and leaves the set membership correct.
What we evaluate
- Whether the tool excludes the explicitly marked non-record from the extracted set.
- Whether the tool returns only genuine members of the repeated set.
- Whether the tool avoids dropping a real member while removing the non-record.
- Whether set membership stays correct even when the non-record matches the surrounding row pattern.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Nested & Repeated Fields
Returns repeated sets of records completely, with each value under the right record: none dropped, none invented, none cross-bound. ON TRIAL: merges back into Field Extraction if precision and recall do not actually diverge on repeated sets.
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
Structured Document Extraction
Which document extraction platform turns a user-defined schema into correct structured records — across layouts, scans and repeated sets — and is honest about what it could not find?