Scenario in benchmark · AI Customer Support Chatbots · Version 1

Conversation activity can be inspected

A prior conversation or activity must be available to locate and open for inspection.

Capability: Analytics and observability1 pinned test case per toolNeutral global definition →

What we check

  1. Whether earlier conversation activity can be located in the product interface.
  2. Whether the located conversation can be opened for review.
  3. Whether the inspection view exposes the prior activity clearly enough to inspect what happened.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Passed · 6
BotpressPassed1 of 1 test case passedAllowed earlier conversation activity to be inspected in its own interface.View Result →

The shipped-order conversation could be found and opened, with its stored transcript visible.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Freshdesk FreddyPassed1 of 1 test case passedExposed the conversation's activity in enough detail to inspect what happened.View Result →

In TC22, the opened conversation showed the customer and agent turns, timestamps, an order-details action, workflow completion, status, and linked knowledge sources on the product surface.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Intercom FinPassed1 of 1 test case passedLet the user inspect an earlier shipped-order conversation in this case.View Result →

It showed the Fin-involved list, opened the matching thread, and displayed the order 41982 reply and tracking details.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

KommunicatePassed1 of 1 test case passedLet support staff inspect an earlier conversation in its own dashboard.View Result →

Both the conversation list and bot analytics provided routes to the same thread, which opened with its history intact.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Tidio LyroPassed1 of 1 test case passedLet an operator inspect an earlier customer conversation from start to finish.View Result →

The shipped-order chat appeared in its conversation log, with order details and the return policy available in the transcript.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Zendesk AIPassed1 of 1 test case passedListed an earlier shipped-order conversation in its own conversation list and opened it with the full transcript readable: the customer's order question, the order-status action marked Success, and the agent's shipping reply.View Result →

Its action details panel also showed the order data that action used.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (8): Ada, Chatbase, CustomGPT.ai, Decagon, FS Agent (DIY control), Gorgias, Help Scout, Respond.io.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Analytics and observability

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

Conversation activity can be inspected in AI Customer Support Chatbots | AI Demos