Scenario in benchmark · AI Customer Support Chatbots · Version 1

One message contains multiple requests

A customer message contains multiple requests in one interaction.

Capability: Intent understanding1 pinned test case per toolNeutral global definition →

What we check

  1. Whether the message contains more than one distinct request.
  2. Whether the response addresses each request separately and appropriately.
  3. Whether no request is ignored, merged, or left unanswered when another request is also present.

Specific test-case inputs and grading live on the related test-case pages.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Passed · 7
BotpressPassed1 of 1 test case passedAnswered both parts of a multi-request message in one reply.View Result →

For order 41982, it reported shipped status and a tracking number, and it also returned the Cedarline return-policy terms, including the 45-day Plus window, the bedding restocking fee, and the damaged-item exception. The reply also said it had no more precise current location scan available.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Freshdesk FreddyPassed1 of 1 test case passedAnswered both requests from one customer message in a single reply.View Result →

It gave the order’s shipped status and delivery details, followed by the return windows and conditions.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Intercom FinPassed1 of 1 test case passedAnswered both requests in one message.View Result →

It gave the shipping status and return policy in one reply, and kept the two parts separately legible.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

KommunicatePassed1 of 1 test case passedHandled one message with two requests correctly.View Result →

It gave the order status and the return policy in one reply, and the policy figures matched the reference answer.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Respond.ioPassed1 of 1 test case passedAnswered both requests in the one message.View Result →

It looked up the order and gave its live status, shipped with UPS and arriving October 27, 2026, then in the same reply stated the return policy: 45 days from delivery for Cedarline Plus members, items in resalable condition, and a 15% restocking fee on opened bedding.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Tidio LyroPassed1 of 1 test case passedHandled the one tested multi-request message successfully.View Result →

It answered the order status and return policy in one reply, and the order details matched the live order data while the policy block covered the requested rules.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Zendesk AIPassed1 of 1 test case passedAnswered both requests in one reply to a message that asked for order status and return policy.View Result →

It said order 41982 had shipped with UPS, and it gave the 21-day standard return window, the 45-day Cedarline Plus window, and the resalable-condition/original-packaging guidance.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (7): Ada, Chatbase, CustomGPT.ai, Decagon, FS Agent (DIY control), Gorgias, Help Scout.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Intent understanding

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

One message contains multiple requests in AI Customer Support Chatbots | AI Demos