Scenario in benchmark · AI Database Agents · Version 1
The user asks to see the same answer a different way
A request to re-present the same answer in a different form while the underlying content stays unchanged.
What we check
- Whether the agent changes the form of the response when asked to see the same answer differently.
- Whether the underlying answer content stays the same across the re-presentation.
- Whether no facts or values are added, removed, or changed when the answer is reformatted.
- Whether the agent preserves the original answer while using a new presentation style.
Exact wording, inputs, fixture state, expected output and detailed grading remain at the test-case level and may be withheld while the benchmark version is active. The scenario and its evaluation intent are public.
How the tools performed
One row per tool, from its current published Result for this scenario and benchmark version. Open a row for the full summary and the exact counts.
FutureSmart Database AgentFailed1 of 1 test case failedFailed to keep the same answer when asked for a different presentation.View Result →
The follow-up returned the same table and said it could not render a chart, while an earlier turn in the same conversation showed a bar chart for the same data; the later turn's values also differed from the earlier chart.
0 Pass · 1 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Anomaly AIPassed1 of 1 test case passedChanged the July-versus-August revenue comparison into a bar chart while preserving the underlying answer.View Result →
July remained $10,877.75 and August $8,795.45; the revenue change, average order values and category contributions also stayed unchanged.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
AskYourDatabasePassed1 of 1 test case passedPresented the same category revenue figures as a horizontal bar chart on request.View Result →
All seven amounts stayed the same to the cent, including outdoor and patio revenue of $7,744.59 in both forms.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
BasedashPassed1 of 1 test case passedTurned its revenue tables into a stacked bar chart on request without changing the figures.View Result →
The chart retained July’s $10,953.75 total, August’s $8,847.45 total and all fourteen category figures from the earlier answer.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
BlazeSQLPassed1 of 1 test case passedPassed this scenario: it showed the same revenue answer in a different form.View Result →
In the recorded conversation, the first result appeared as a table with seven categories, then the follow-up request switched that same result to a bar chart. The bar lengths matched the table values within the chart's scale.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
camelAIPassed1 of 1 test case passedChanged its revenue explanation into a chart on request while keeping the underlying answer intact.View Result →
Both forms showed a 19.1% decline and a $4,383 drop in furniture sales, with the chart using whole-dollar rounding.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
DotPassed1 of 1 test case passedAsked in a follow-up in the same saved chat to show its chart answer about August revenue as a table, Dot returned two tables with no chart, and every value matched the original: July $10,878, August $8,795 (down 19.1%), and all seven category figures, such as furniture falling from $5,483 to $1,099.View Result →
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
DraxlrPassed1 of 1 test case passedRedrew its revenue answer as a bar chart when asked, and the numbers stayed the same.View Result →
The chart's seven category changes, from furniture down $6,562.19 to bedding and rugs up $1,129.47, are the first answer's figures and the reference figures to the cent, and the chart's own table and query are unchanged.
1 Pass · 0 Fail · 0 Not gradable · 1 of 1 assessed · 1 of 1 gradable
Failures first, then A–Z within each group. Not a ranking. Assessed = Pass + Fail + Not gradable; Gradable = Pass + Fail, both out of the Result’s pinned test cases.
No published Result yet (3): AI for Database, Definite, Querio.
No published Result does not tell you whether a tool has been tested.Test design
- Pinned test cases
- 1
- Disclosure
- 0 public · 1 withheld
- Capabilities exercised here
- Answer Presentation
How the results are graded
- Pass the test-case expectations hold.
- Fail an expectation demonstrably does not hold.
- Not gradable the evidence cannot establish the outcome.
Version 1 uses test-case expectations; no scenario rubric is pinned. Benchmark methodology →