Scenario in benchmark · AI Database Agents · Version 1
The answer is a trend or a comparison
The answer calls for a trend or comparison, so the product should present it in a form that makes the pattern visible.
What we check
- Whether the response uses a form that makes change or comparison easy to see.
- Whether the product avoids presenting the answer as an undifferentiated row dump when a structured view is more suitable.
- Whether the chosen presentation matches the kind of answer being asked for, rather than staying in a default format.
Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
AskYourDatabaseFailed1 of 1 test case failedReturned monthly order counts without volunteering a visual to show the trend.View Result →
Its answer used a text table and described the increase in words.
0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
camelAIFailed1 of 1 test case failedAnswered the monthly trend question with a table and no unprompted visual.View Result →
Its counts matched the database reference: June 2026 had 11 orders, July 18 and August 28.
0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
DraxlrFailed1 of 1 test case failedAnswered a monthly order-count question with a sentence and a three-row table, and drew no chart.View Result →
Its Visualization pane, shown by default, held a table of the three months with no axes, bars or line.
0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
FutureSmart Database AgentFailed1 of 1 test case failedDid not present the answer as a trend visual.View Result →
It showed monthly counts in Table view with Chart available but not selected, and the month labels were raw timestamps.
0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Anomaly AIPassed1 of 1 test case passedHandled this trend/comparison request with a visual, not just rows.View Result →
The request asked only for monthly counts, and the answer produced a month-axis bar chart with July 18, August 27, and September 3.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
BasedashPassed1 of 1 test case passedShowed the monthly order trend as both a table and an unasked bar chart.View Result →
The bars rose from June to August, and the answer used the three completed calendar months in Asia/Calcutta.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
BlazeSQLPassed1 of 1 test case passedPassed this scenario.View Result →
It showed an unasked bar chart for the monthly order count, then highlighted the August spike. The table showed June, July, and August counts of 20, 21, and 38.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
No published Result yet (4): AI for Database, Definite, Dot, Querio.
No published Result does not tell you whether a tool has been tested.Test design
- Pinned test cases
- 1
- Disclosure
- 0 public · 1 withheld
- Capabilities exercised here
- Answer Presentation
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →