Scenario in benchmark · AI Customer Support Chatbots · Version 1

Feedback can be submitted

A user can submit feedback or a correction about a prior answer.

Capability: Feedback and learning1 pinned test case per toolNeutral global definition →

What we check

  1. Whether feedback or a correction about a prior answer can be submitted successfully.
  2. Whether the submission is linked to the earlier answer it refers to.
  3. Whether the system accepts the feedback without treating it as an unrelated new request.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Failed · 3
Freshdesk FreddyFailed1 of 1 test case failedOffered no way to submit a correction to an earlier wrong answer.View Result →

The conversation had no feedback option, while Improve promised future suggestions rather than accepting a correction to that answer.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

KommunicateFailed1 of 1 test case failedRecords binary feedback on answers, but this case did not show a place to submit a written correction.View Result →

The dashboard shows two thumbs-down marks tied to one knowledge document and two conversations, with no field for entering the corrected answer.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Zendesk AIFailed1 of 1 test case failedOffered no way to submit a correction to one of its earlier answers on the conversation screen where those answers are read.View Result →

That screen's only menu held an export option, the conversation overview offered only a field for adding labels, and no rating, flag or correction option appeared on the agent's replies.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Passed · 2
BotpressPassed1 of 1 test case passedTook a submitted correction in this scenario.View Result →

The correction was entered and submitted, and the returned draft fix said it would omit the AI disclosure and keep the response focused; the capture stops before any applied answer is shown.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Intercom FinPassed1 of 1 test case passedLets a customer submit feedback on a wrong answer.View Result →

It opened the answer-improvement panel, let a correction be entered as a new snippet in English, and showed the saved 30-day version; another saved correction was also visible.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (9): Ada, Chatbase, CustomGPT.ai, Decagon, FS Agent (DIY control), Gorgias, Help Scout, Respond.io, Tidio Lyro.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Feedback and learning

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

Feedback can be submitted in AI Customer Support Chatbots | AI Demos