AskYourDatabase in AI Database Agents
Scenario-level performance from current published Results.
12 scenarios with published Results · 28 scenarios in the benchmark
How AskYourDatabase performed
Open a capability to explore its scenarios. Each row reports the test set in its published Result; counts are not combined into an overall score.
Analytics & Observability4 scenarios · 1 with published Results
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| Reported analytics match what actually happened | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase's reported totals matched what actually happened in this run. Person 1 showed 3 conversations and Person 2 showed 2, and both included unanswered questions; the five-message list in the capture matches the five questions that were run. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A user reports a bad answer and the operator has to find it | No published result | ||
| Someone needs to see what people have been asking | No published result | ||
| The operator needs to find the questions the agent is failing on | No published result | ||
Answer Presentation4 scenarios · 3 with published Results
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The answer is a plain fact or a short list | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase answered this case in a plain sentence. The chat shows "You've received 308 total orders to date." and the pane shows no chart or one-cell table. The answer matches the recorded total of 308 orders. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The answer needs a summary and its detail together | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase gave a headline figure and a breakdown in forms suited to each, so it met the scenario. It returned a revenue summary table for the total, a category table for the breakdown, and takeaways naming the driver; there was no chart. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The user asks to see the same answer a different way | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase changed the answer from a table to a chart without changing the values. The earlier reply shows a seven-category revenue table for August 2026. The charted reply prints the same seven category amounts beside the bars, and the code shown in the chat fetches saved data instead of running new SQL. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The answer is a trend or a comparison | No published result | ||
Conversational Interaction6 scenarios · 4 with published Results
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| Customer corrects information given earlier | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase handled a customer correction in this conversation. After the July 2026 question returned 23 orders, the follow-up “Sorry, I meant June 2026.” triggered fresh June-only SQL and a June answer of 10. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The follow-up is ambiguous | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedAskYourDatabase did not ask for clarification when a follow-up was ambiguous. The first answer created two distinct customer lists, and the second turn answered with a side-by-side comparison instead of asking which list was meant. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The user changes direction mid-conversation | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase handles the mid-conversation change by dropping the earlier Texas filter and answering with the broader all-states breakdown. The second query has no state condition, and the 15-row table totals 308 orders and 67 cancellations; the Texas turn stays separate at 18 orders and 27.78%. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The user narrows what they just asked | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase handles this narrowing case: it keeps the earlier question’s date window and adds the new Texas constraint. In the first turn, the app returned 59 orders in a 10-row preview plus a full CSV. In the follow-up, it kept the same window, returned 3 Texas orders, and those three all appear in the first turn’s 59-row CSV. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The agent offers what to ask next | No published result | ||
| The next question refers to the previous answer | No published result | ||
Question Answering4 scenarios · 3 with published Results
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The connected data cannot answer the question | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase could not answer the question from the connected data. It said the schema it could see had no review or rating data, asked where that data is stored, and returned no number. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The question uses a term the company defines itself | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase handled the company-defined term by stating the definition it used instead of giving a bare number. It named revenue as order totals, excluded cancelled orders, and offered other bases; the answer’s heading and SQL also showed a June 4–September 2 span under a 90-day label. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The user's wording doesn't match how the values are stored | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase handled a request whose wording differed from the stored value in the tested turn. It rewrote "gift voucher" as "gift card", queried the stored value, and returned 14 matching orders instead of an empty result. The answer view shows the table and a note that two orders were later cancelled. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The answer is available in the connected data | No published result | ||
Reporting3 scenarios · 1 with published Results
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The user wants several findings in one place | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedAskYourDatabase put the three findings into one combined chat reply, and the figures matched the earlier direct answers. The report was a formatted assistant reply, not a saved file. It reported revenue excluding cancelled orders, Home Decor as the best-selling category by units, and both order and subscription cancellation rates. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| The report is run again after the data has changed | No published result | ||
| The user wants to keep an answer they just got | No published result | ||
Data Access Control3 scenarios · 0 with published Results
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| An excluded column is asked for directly | No published result | ||
| An excluded table is needed to answer the question | No published result | ||
| Two users are given different data scope | No published result | ||
Training4 scenarios · 0 with published Results
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| The agent used the wrong definition and the user corrects it | No published result | ||
| The database's names are cryptic and the user documents them | No published result | ||
| The user sets a standing rule for all future answers | No published result | ||
| The user supplies example questions and the queries they trust | No published result | ||
Reading these Results
Published evidence and test coverage answer different questions.
Which scenarios have a Result?
A published Result is public evidence for this tool on one scenario. “No published result” does not say whether testing has taken place.
What does each Result cover?
Assessed includes Pass, Fail and Not gradable. Gradable includes Pass and Fail. Both use the pinned test count in that published Result.
Inventory is not testing progress
The 28 scenarios describe this benchmark’s scope. They are not an assumed applicability or test-coverage denominator for AskYourDatabase.