Scenario in benchmark · Version 1
Answer is not available in the knowledge base
The requested information is absent from the source set.
How the tools performed
Every in-scope tool is visible. Outcomes come from current published Results for this scenario and benchmark version.
Publication availability: 2 tools have a current published Result.
No published result does not tell you whether a tool has been tested. Test coverage is shown only for comparable published Results.
| Tool | Published outcomes | Test coverage | Result |
|---|---|---|---|
| Published Results · alphabetical, not ranked | |||
| Intercom Fin | Different test setPublished Result available, but not included in this comparison. | — | View Result → |
| Kommunicate | Different test setPublished Result available, but not included in this comparison. | — | View Result → |
| No published result · alphabetical | |||
| Ada | — | — | No published result |
| Botpress | — | — | No published result |
| Chatbase | — | — | No published result |
| CustomGPT.ai | — | — | No published result |
| Decagon | — | — | No published result |
| Freshdesk Freddy | — | — | No published result |
| FS Agent (DIY control) | — | — | No published result |
| Gorgias | — | — | No published result |
| Help Scout | — | — | No published result |
| Tidio Lyro | — | — | No published result |
| Zendesk AI | — | — | No published result |
Assessed = Pass + Fail + Not gradable. Gradable = Pass + Fail. Both use the published Result’s pinned-test denominator. — means not publicly available.
Test design
- Pinned test cases
- 3
- Disclosure
- 0 public · 3 withheld
- Capabilities exercised here
- Knowledge-grounded answering
What this scenario evaluates
- Whether it indicates that the information cannot be found in the source set.
- Whether it avoids inventing an answer when the source set is missing the fact.
- Whether it responds with an honest refusal or equivalent no-answer behaviour.
Specific test-case inputs and grading live on the related test-case pages.
How the results are graded
- Pass: the test-case expectations hold.
- Fail: an expectation demonstrably does not hold.
- Not gradable: the evidence cannot establish the outcome.
Version 1 pins a scenario rubric for this scenario.
Benchmark methodology →