Scenario in benchmark · AI Database Agents · Version 1
The answer is a plain fact or a short list
When an answer is a single value or short list, it should be stated plainly rather than shown as a chart or one-cell table.
What we check
- Whether the reply states a single value or short list directly in plain text.
- Whether the reply avoids over-presenting that answer as a chart or one-cell table.
- Whether the form used fits the answer without adding unnecessary structure.
Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
Anomaly AIPassed1 of 1 test case passedGave the answer in plain text: 308 total orders, matching the reference count of 308.View Result →
A single-bar chart accompanied the written answer.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
AskYourDatabasePassed1 of 1 test case passedMet this presentation requirement in the tested case.View Result →
It stated 308 total orders in a plain sentence, and the screen showed no chart, plot, result grid, or one-cell table.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
BasedashPassed1 of 1 test case passedHandled the single-value presentation scenario correctly.View Result →
It answered with 308 in prose, the one-row result panel supported that answer rather than replacing it, and no chart appeared.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
BlazeSQLPassed1 of 1 test case passedAnswered the plain fact in prose, so the evidence does not show the result being confined to a chart or a one-cell table.View Result →
In the captured run, the reply said there were 100 orders ever, and the same frame also showed a one-row result table and no chart.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
camelAIPassed1 of 1 test case passedStated the single value in its reply, 308 orders in total, with no chart or one-cell table.View Result →
Its own count query and a separate reference query on the same database both return 308.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
DraxlrPassed1 of 1 test case passedGave a single-value answer in plain language.View Result →
It stated that there were 308 orders in total.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
FutureSmart Database AgentPassed1 of 1 test case passedAnswered this case in plain prose rather than only as a chart or one-cell table.View Result →
It said there were 100 orders in total, and the nearby count grid repeated the same number.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
No published Result yet (4): AI for Database, Definite, Dot, Querio.
No published Result does not tell you whether a tool has been tested.Test design
- Pinned test cases
- 1
- Disclosure
- 0 public · 1 withheld
- Capabilities exercised here
- Answer Presentation
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →