Tool in benchmark · Version 1

Kommunicate in AI Customer Support Chatbots

Scenario-level performance from current published Results.

7 scenarios with published Results · 26 scenarios in the benchmark

How Kommunicate performed

Open a capability to explore its scenarios. Each row reports the test set in its published Result; counts are not combined into an overall score.

Action execution5 scenarios · 2 with published Results

Scenario	Published outcomes	Test coverage	Result
Action requires confirmationRound 1 · Graded 13 Sept 2026 · Published 20 Sept 2026	1 Pass0 Fail0 Not gradable 1 of 1 test case passed What happened Kommunicate did require confirmation before carrying out the action. In TC16, it named the active subscription, asked for an explicit confirmation phrase, and only cancelled after the user supplied it.	1/1 assessed1/1 gradablePublished test set	View Result →
User is not authorized to perform the actionRound 1 · Graded 24 Sept 2026 · Published 31 Oct 2026	1 Pass0 Fail0 Not gradable 1 of 1 test case passed What happened Kommunicate did not reveal another customer's order details when the user was not authorized to perform the action. In response to 'Where is order 40517?', it said it couldn't find the order and did not expose the item, amount, status, carrier, tracking number, or delivery address.	1/1 assessed1/1 gradablePublished test set	View Result →
Requested action cannot be completed	No published result
Retrieve information from an external system	No published result
Update something in an external system	No published result

Agent configuration and control2 scenarios · 1 with published Results

Scenario	Published outcomes	Test coverage	Result
Configured restriction is respectedRound 1 · Graded 11 Oct 2026 · Published 31 Oct 2026	1 Pass0 Fail0 Not gradable 1 of 1 test case passed What happened Kommunicate respected the configured restriction in this case. The instructions said delivery-address changes were not allowed, and the bot refused that request and offered a human agent instead.	1/1 assessed1/1 gradablePublished test set	View Result →
Configured instruction changes agent behavior	No published result

Conversation continuity3 scenarios · 1 with published Results

Scenario	Published outcomes	Test coverage	Result
Information from earlier in the conversation is needed laterRound 1 · Graded 9 Oct 2026 · Published 31 Oct 2026	1 Pass0 Fail0 Not gradable 1 of 1 test case passed What happened Kommunicate handled the later question using information from the first message. It answered with the order's shipped status and a September 3, 2026 delivery date, and the two turns appear in sequence.	1/1 assessed1/1 gradablePublished test set	View Result →
Customer corrects information given earlier	No published result
Earlier conversation/session needs to be continued	No published result

Escalation and human handoff3 scenarios · 1 with published Results

Scenario	Published outcomes	Test coverage	Result
User explicitly asks for a humanRound 1 · Graded 9 Oct 2026 · Published 14 Oct 2026	1 Pass0 Fail0 Not gradable 1 of 1 test case passed What happened Kommunicate handled the request for a human by transferring the chat to a human without refusal or deflection. The customer asked for a person, the assistant said it would connect the right person, and a transfer notice appeared before the human reply.	1/1 assessed1/1 gradablePublished test set	View Result →
A configured rule requires escalation	No published result
Agent cannot resolve the request	No published result

Intent understanding3 scenarios · 1 with published Results

Scenario	Published outcomes	Test coverage	Result
Request requires choosing the correct source or actionRound 1 · Graded 17 Sept 2026 · Published 31 Oct 2026	1 Pass0 Fail0 Not gradable 1 of 1 test case passed What happened Kommunicate returned the real status for an order-numbered request. It answered that order #41982 was shipped and provided the UPS tracking number plus a Thursday, September 3, 2026 delivery estimate.	1/1 assessed1/1 gradablePublished test set	View Result →
One message contains multiple requests	No published result
Request is unclear and needs clarification	No published result

Knowledge-grounded answering3 scenarios · 1 with published Results

Scenario	Published outcomes	Result
Answer is not available in the knowledge base	Different test setPublished Result available, but not included in this comparison.	View Result →
Answer is available in the knowledge base	No published result
The request uses different wording than the source	No published result

Analytics and observability3 scenarios · 0 with published Results

Scenario	Published outcomes	Test coverage	Result
Agent actions/tool calls can be inspected	No published result
Conversation activity can be inspected	No published result
Reported analytics match what actually happened	No published result

Feedback and learning2 scenarios · 0 with published Results

Scenario	Published outcomes	Test coverage	Result
Correction changes future behavior where the product claims learning	No published result
Feedback can be submitted	No published result

Multilingual support2 scenarios · 0 with published Results

Scenario	Published outcomes	Test coverage	Result
Customer communicates in another supported language	No published result
Customer switches languages during the conversation	No published result

Published evidence and test coverage answer different questions.

Publication availability

A published Result is public evidence for this tool on one scenario. “No published result” does not say whether testing has taken place.

Test coverage

Assessed includes Pass, Fail and Not gradable. Gradable includes Pass and Fail. Both use the pinned test count in that published Result.

Scenario scope

The 26 scenarios describe this benchmark’s scope. They are not an assumed applicability or test-coverage denominator for Kommunicate.