Scenario in benchmark · AI Customer Support Chatbots · Version 1
Retrieve information from an external system
A request requires information to be retrieved from an external connected system.
What we check
- Whether the response uses an external connected system to obtain the needed information.
- Whether the answer reflects the information returned by that system.
- Whether the agent does not rely on the knowledge base alone when the needed information lives elsewhere.
Specific test-case inputs and grading live on the related test-case pages.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
BotpressPassed1 of 1 test case passedPassed this external-order lookup scenario.View Result →
It ran an order-status tool call, and the reply matched the order record’s shipped status and tracking number.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Freshdesk FreddyPassed1 of 1 test case passedRetrieved information from an external system in this case.View Result →
It returned shipped, UPS, and a tracking number, and the reply repeated those values while no knowledge sources were used.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Intercom FinPassed1 of 1 test case passedRetrieved the shipped order's live status from the external system.View Result →
The reply and the logged 200 call match on shipped status, UPS, tracking number, shipped date, and estimated delivery, showing the answer came from that call.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Respond.ioPassed1 of 1 test case passedRetrieved the order’s status and correctly reported that it had shipped.View Result →
The order-status action ran, and the connected order system also listed shipped.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Zendesk AIPassed1 of 1 test case passedRetrieved information from an external system and reported the returned order status.View Result →
For order 41982, the conversation log shows the Check Order Status action firing successfully and returning shipped, UPS, tracking number 1Z999AA10123456784, and estimated delivery 2026-09-22; the customer reply repeats those same fields. The recording shows the action firing and returned fields, but not the HTTP request line or endpoint.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
No published Result yet (9): Ada, Chatbase, CustomGPT.ai, Decagon, FS Agent (DIY control), Gorgias, Help Scout, Kommunicate, Tidio Lyro.
No published Result does not tell you whether a tool has been tested.Test design
- Pinned test cases
- 1
- Disclosure
- 0 public · 1 withheld
- Capabilities exercised here
- Action execution
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →