Scenario in benchmark · AI Database Agents · Version 1

The user wants to keep an answer they just got

A person asks to save a just-produced answer so it can be reopened later without changing what it says.

Capability: Reporting1 pinned test case per toolNeutral global definition →

What we check

  1. Whether a recently produced answer can be saved as a durable artifact.
  2. Whether reopening the saved item later reproduces the same answer.
  3. Whether the reopened item is still recognisably the same saved result.

Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.

How the tools performed

One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.

Failed · 2
AskYourDatabaseFailed1 of 1 test case failedNo save control was found in AskYourDatabase for keeping the answer and reopening it in a new session.View Result →

Its bot settings, data source, billing, API key and settings pages offered no save control, and creating a dashboard page returned an error.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

FutureSmart Database AgentFailed1 of 1 test case failedDid not let a user save an answer and reopen it later.View Result →

The answer view showed viewing, filtering, charting, CSV export, thumbs, and copy controls, but no saved-answer destination.

0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Passed · 4
Anomaly AIPassed1 of 1 test case passedSaved the answer as a dashboard and reproduced it outside the original conversation.View Result →

Reopening retained the explanation and the order counts—18 for July 2026, 27 for August 2026 and 3 for September 1–2, 2026—with a heading identifying the question and cutoff.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

BasedashPassed1 of 1 test case passedSaved the monthly-order answer as a dashboard chart and reproduced it in another account's session.View Result →

The counts remained 11 for June 2026, 18 for July 2026 and 27 for August 2026, totalling 56.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

BlazeSQLPassed1 of 1 test case passedSaves an answer and its query, then reopens the saved item with the same three rows under the saved name.View Result →

The saved item appears in a separate Saved area, and the reopened table matches the earlier chat answer.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

DotPassed1 of 1 test case passedKept the answer available for reopening.View Result →

Saving it as an app and reopening it in a new session showed the same title, date range, and June/July/August values, and the saved app re-ran the same query.

1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable

Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.

No published Result yet (5): AI for Database, camelAI, Definite, Draxlr, Querio.

No published Result does not tell you whether a tool has been tested.

Test design

Pinned test cases
1
Disclosure
0 public · 1 withheld
Capabilities exercised here
Reporting

How the results are graded

  • Pass the test-case expectations hold.
  • Fail an expectation demonstrably does not hold.
  • Not gradable the evidence cannot establish the outcome.

Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →

The user wants to keep an answer they just got in AI Database Agents | AI Demos