Scenario in benchmark · Managed RAG · Version 1
An ordinary digital text document
A plain digital text document used as the baseline text-only document condition.
What we check
- Whether every paragraph is preserved complete and unchanged.
- Whether nothing is silently missing, garbled, or invented.
- Whether the result remains recognisably plain text without unintended changes to the content.
Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
Denser AIPassed1 of 1 test case passedReturned the requested fact from the ordinary text document.View Result →
The 22-month interval came back as the top passage, and the answer repeated the same value; the other returned passages did not state that interval.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
DifyPassed1 of 1 test case passedReturned the passage that states the asked-for fact from a plain digital text document.View Result →
Asked for the Tolvan A3's recommended recalibration interval in the document's own words, it ranked the whole Calibration passage first, which gives the interval as 22 months from the date of the last certificate.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
FutureSmart Document IntelligencePassed1 of 1 test case passedReturned the calibration passage first, and the answer gave 22 months from the last calibration certificate.View Result →
A 36-month warranty passage also appeared at a lower score and did not replace the calibration passage.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
RAGFlowPassed1 of 1 test case passedPasses this plain-text document scenario.View Result →
The ingested document contains the target sentence, and the default retrieval call returns that chunk first with a clear score gap.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
VectaraPassed1 of 1 test case passedReturned the requested fact in the document's own words.View Result →
In both configurations, the top result was the passage stating a 22-month recalibration interval, and the next results were different warranty passages; both captures showed HTTP 200.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
No published Result yet (3): Llama Cloud, Onyx, Vectorize.io.
No published Result does not tell you whether a tool has been tested.Test design
- Pinned test cases
- 1
- Disclosure
- 0 public · 1 withheld
- Capabilities exercised here
- Data Ingestion
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →