Scenario in benchmark · AI Database Agents · Version 1

The agent offers what to ask next

After an answer, the product suggests follow-up questions grounded in the just-completed analysis.

Capability: Conversational Interaction1 pinned test case per toolNeutral global definition →

What we check

  1. Whether it offers follow-up questions after answering.
  2. Whether the suggestions are grounded in the answer just given and the analysis behind it.
  3. Whether the follow-up suggestions are specific to the conversation rather than generic prompts.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Failed · 3
AskYourDatabaseFailed1 of 1 test case failedDid not suggest a follow-up question after answering the query.View Result →

The answer ended the thread with no suggestion chips or follow-up list, and the only suggestion-like setting is for new chat sessions.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

BasedashFailed1 of 1 test case failedDid not offer a follow-up question after answering in web chat.View Result →

It answered the revenue question but left the space below the answer empty, even after eight minutes.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

DraxlrFailed1 of 1 test case failedAnswered the revenue question without suggesting what to ask next.View Result →

Its chat left the user to write a follow-up rather than offering a question based on the analysis.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Passed · 4
Anomaly AIPassed1 of 1 test case passedSuggested a relevant follow-up after answering a question.View Result →

For an August revenue answer, it offered a July comparison that the same database could support.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

BlazeSQLPassed1 of 1 test case passedOffered what to ask next after answering the revenue question.View Result →

It ended the August 2026 revenue reply with “Would you like a breakdown by category, or any other analysis?”, so the capture shows one specific follow-up rather than only a generic closing.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

DotPassed1 of 1 test case passedAfter answering last month's revenue question with $8,795, Dot showed one follow-up suggestion without being asked: "what about the order count".View Result →

It asks about the same August 2026 non-cancelled orders, and the result behind the answer already held that count, 22, so the suggestion can be answered from the same database.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

FutureSmart Database AgentPassed1 of 1 test case passedOffered two follow-up questions after its August 2026 revenue answer, and both were specific to that answer and answerable from the database.View Result →

The chips ask about August versus July and the highest-revenue month so far.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (4): AI for Database, camelAI, Definite, Querio.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Conversational Interaction

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

The agent offers what to ask next in AI Database Agents | AI Demos