Tool in benchmark · Version 1

AskYourDatabase in AI Database Agents

Scenario-level performance from current published Results.

12 scenarios with published Results · 28 scenarios in the benchmark

How AskYourDatabase performed

Open a capability to explore its scenarios. Each row reports the test set in its published Result; counts are not combined into an overall score.

Analytics & Observability4 scenarios · 1 with published Results
ScenarioPublished outcomesTest coverageResult
Reported analytics match what actually happened
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase's reported totals matched what actually happened in this run. Person 1 showed 3 conversations and Person 2 showed 2, and both included unanswered questions; the five-message list in the capture matches the five questions that were run.

1/1 assessed1/1 gradablePublished test setView Result →
A user reports a bad answer and the operator has to find itNo published result
Someone needs to see what people have been askingNo published result
The operator needs to find the questions the agent is failing onNo published result
Answer Presentation4 scenarios · 3 with published Results
ScenarioPublished outcomesTest coverageResult
The answer is a plain fact or a short list
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase answered this case in a plain sentence. The chat shows "You've received 308 total orders to date." and the pane shows no chart or one-cell table. The answer matches the recorded total of 308 orders.

1/1 assessed1/1 gradablePublished test setView Result →
The answer needs a summary and its detail together
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase gave a headline figure and a breakdown in forms suited to each, so it met the scenario. It returned a revenue summary table for the total, a category table for the breakdown, and takeaways naming the driver; there was no chart.

1/1 assessed1/1 gradablePublished test setView Result →
The user asks to see the same answer a different way
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase changed the answer from a table to a chart without changing the values. The earlier reply shows a seven-category revenue table for August 2026. The charted reply prints the same seven category amounts beside the bars, and the code shown in the chat fetches saved data instead of running new SQL.

1/1 assessed1/1 gradablePublished test setView Result →
The answer is a trend or a comparisonNo published result
Conversational Interaction6 scenarios · 4 with published Results
ScenarioPublished outcomesTest coverageResult
Customer corrects information given earlier
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase handled a customer correction in this conversation. After the July 2026 question returned 23 orders, the follow-up “Sorry, I meant June 2026.” triggered fresh June-only SQL and a June answer of 10.

1/1 assessed1/1 gradablePublished test setView Result →
The follow-up is ambiguous
0 Pass1 Fail0 Not gradable
1 of 1 test case failed
What happened

AskYourDatabase did not ask for clarification when a follow-up was ambiguous. The first answer created two distinct customer lists, and the second turn answered with a side-by-side comparison instead of asking which list was meant.

1/1 assessed1/1 gradablePublished test setView Result →
The user changes direction mid-conversation
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase handles the mid-conversation change by dropping the earlier Texas filter and answering with the broader all-states breakdown. The second query has no state condition, and the 15-row table totals 308 orders and 67 cancellations; the Texas turn stays separate at 18 orders and 27.78%.

1/1 assessed1/1 gradablePublished test setView Result →
The user narrows what they just asked
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase handles this narrowing case: it keeps the earlier question’s date window and adds the new Texas constraint. In the first turn, the app returned 59 orders in a 10-row preview plus a full CSV. In the follow-up, it kept the same window, returned 3 Texas orders, and those three all appear in the first turn’s 59-row CSV.

1/1 assessed1/1 gradablePublished test setView Result →
The agent offers what to ask nextNo published result
The next question refers to the previous answerNo published result
Question Answering4 scenarios · 3 with published Results
ScenarioPublished outcomesTest coverageResult
The connected data cannot answer the question
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase could not answer the question from the connected data. It said the schema it could see had no review or rating data, asked where that data is stored, and returned no number.

1/1 assessed1/1 gradablePublished test setView Result →
The question uses a term the company defines itself
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase handled the company-defined term by stating the definition it used instead of giving a bare number. It named revenue as order totals, excluded cancelled orders, and offered other bases; the answer’s heading and SQL also showed a June 4–September 2 span under a 90-day label.

1/1 assessed1/1 gradablePublished test setView Result →
The user's wording doesn't match how the values are stored
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase handled a request whose wording differed from the stored value in the tested turn. It rewrote "gift voucher" as "gift card", queried the stored value, and returned 14 matching orders instead of an empty result. The answer view shows the table and a note that two orders were later cancelled.

1/1 assessed1/1 gradablePublished test setView Result →
The answer is available in the connected dataNo published result
Reporting3 scenarios · 1 with published Results
ScenarioPublished outcomesTest coverageResult
The user wants several findings in one place
1 Pass0 Fail0 Not gradable
1 of 1 test case passed
What happened

AskYourDatabase put the three findings into one combined chat reply, and the figures matched the earlier direct answers. The report was a formatted assistant reply, not a saved file. It reported revenue excluding cancelled orders, Home Decor as the best-selling category by units, and both order and subscription cancellation rates.

1/1 assessed1/1 gradablePublished test setView Result →
The report is run again after the data has changedNo published result
The user wants to keep an answer they just gotNo published result
Data Access Control3 scenarios · 0 with published Results
ScenarioPublished outcomesTest coverageResult
An excluded column is asked for directlyNo published result
An excluded table is needed to answer the questionNo published result
Two users are given different data scopeNo published result
Training4 scenarios · 0 with published Results
ScenarioPublished outcomesTest coverageResult
The agent used the wrong definition and the user corrects itNo published result
The database's names are cryptic and the user documents themNo published result
The user sets a standing rule for all future answersNo published result
The user supplies example questions and the queries they trustNo published result

Reading these Results

Published evidence and test coverage answer different questions.

Publication availability

Which scenarios have a Result?

A published Result is public evidence for this tool on one scenario. “No published result” does not say whether testing has taken place.

Test coverage

What does each Result cover?

Assessed includes Pass, Fail and Not gradable. Gradable includes Pass and Fail. Both use the pinned test count in that published Result.

Scenario scope

Inventory is not testing progress

The 28 scenarios describe this benchmark’s scope. They are not an assumed applicability or test-coverage denominator for AskYourDatabase.

AskYourDatabase in AI Database Agents | AI Demos