Scenario in benchmark · AI Customer Support Chatbots · Version 1
User explicitly asks for a human
An explicit request for a human handoff is made.
What we check
- Whether the agent hands the conversation off to a human after an explicit request.
- Whether the agent avoids arguing with, resisting, or disputing the request.
Specific test-case inputs and grading live on the related test-case pages.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
BotpressPassed1 of 1 test case passedHandled the human request successfully in this scenario.View Result →
The first reply offered a human representative, the chat was transferred, and Desk then showed a human taking over and replying.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Freshdesk FreddyPassed1 of 1 test case passedHanded the user to a human without arguing.View Result →
The bot agreed in its next reply, and the user and agent then exchanged messages on the same ticket.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Intercom FinPassed1 of 1 test case passedHandled a direct request for a human.View Result →
It offered a human handoff on the first reply, showed a one-tap connect option, and the same conversation later appeared in the agent inbox with a teammate reply.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
KommunicatePassed1 of 1 test case passedHandled the request for a human by transferring the chat to a human without refusal or deflection.View Result →
The customer asked for a person, the assistant said it would connect the right person, and a transfer notice appeared before the human reply.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Respond.ioPassed1 of 1 test case passedHanded the chat to a human when the customer explicitly asked.View Result →
The agent agreed without asking for a reason or trying to answer instead.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Tidio LyroPassed1 of 1 test case passedPassed the human-handoff scenario.View Result →
The customer asked for a human, Lyro answered once with a direct transfer, and the same conversation then reached a human agent in Tidio's inbox.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Zendesk AIPassed1 of 1 test case passedHandled the request to speak to a human and passed it to a live agent.View Result →
The reply gave a callback-style answer without pushback, then ticket #5 showed a named agent responding.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
No published Result yet (7): Ada, Chatbase, CustomGPT.ai, Decagon, FS Agent (DIY control), Gorgias, Help Scout.
No published Result does not tell you whether a tool has been tested.Test design
- Pinned test cases
- 1
- Disclosure
- 0 public · 1 withheld
- Capabilities exercised here
- Escalation and human handoff
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →