A bad file in the upload
A bad file uploaded for ingestion cannot be processed.
What this scenario means
This scenario covers a file uploaded for ingestion that the service cannot process, such as a corrupt, password-protected, or too-large file. The correct behaviour is to report the failure for that file and not treat the upload as successfully ingested.
What we evaluate
- Whether the service reports failure for the uploaded file.
- Whether the file is marked as not processed or not ingested.
- Whether the upload is presented as unsuccessful rather than successful.
Capabilities this scenario exercises
A scenario may exercise one or more capabilities.
Data Ingestion
The knowledge in the files you hand over actually becomes reachable — and the service tells you when it didn't. Graded with the retrieval side held trivially easy (a query in the source's own wording over a small corpus), so a failure is attributable to ingestion rather than to search.
Benchmarks that use this scenario
A scenario has global identity and may be reused across benchmarks.
Managed RAG
Which managed RAG service turns a folder of private documents into a query API that finds the right passage, answers only from what it retrieved, cites the source that actually supports the claim — and keeps up when the source changes?