Tool in benchmark · Version 1

Dot in AI Database Agents

Scenario-level performance from current published Results.

3 scenarios with published Results · 28 scenarios in the benchmark

How Dot performed

Open a capability to explore its scenarios. Each row reports the test set in its published Result; counts are not combined into an overall score.

Answer Presentation4 scenarios · 1 with published Results

Scenario	Published outcomes	Test coverage	Result
The answer needs a summary and its detail togetherRound 1 · Graded 18 Oct 2026 · Published 31 Oct 2026	1 Pass0 Fail0 Not gradable 1 of 1 test case passed What happened Dot did combine a headline figure with its breakdown. It gave an August sales sentence with a monthly chart, then a contribution chart and driver sentences; Furniture was the main driver.	1/1 assessed1/1 gradablePublished test set	View Result →
The answer is a plain fact or a short list	No published result
The answer is a trend or a comparison	No published result
The user asks to see the same answer a different way	No published result

Data Access Control3 scenarios · 1 with published Results

Scenario	Published outcomes	Test coverage	Result
An excluded table is needed to answer the questionRound 1 · Graded 22 Oct 2026 · Published 31 Oct 2026	1 Pass0 Fail0 Not gradable 1 of 1 test case passed What happened Dot passed this scenario: it refused to answer because the returns table was turned off, and it did not use the table as the answer. The logs show only a search step, with no data fetch or SQL, and the reply pointed to Model for the switch.	1/1 assessed1/1 gradablePublished test set	View Result →
An excluded column is asked for directly	No published result
Two users are given different data scope	No published result

Reporting3 scenarios · 1 with published Results

Scenario	Published outcomes	Test coverage	Result
The user wants to keep an answer they just gotRound 1 · Graded 21 Oct 2026 · Published 31 Oct 2026	1 Pass0 Fail0 Not gradable 1 of 1 test case passed What happened Dot kept the answer available for reopening. Saving it as an app and reopening it in a new session showed the same title, date range, and June/July/August values, and the saved app re-ran the same query.	1/1 assessed1/1 gradablePublished test set	View Result →
The report is run again after the data has changed	No published result
The user wants several findings in one place	No published result

Analytics & Observability4 scenarios · 0 with published Results

Scenario	Published outcomes	Test coverage	Result
A user reports a bad answer and the operator has to find it	No published result
Reported analytics match what actually happened	No published result
Someone needs to see what people have been asking	No published result
The operator needs to find the questions the agent is failing on	No published result

Conversational Interaction6 scenarios · 0 with published Results

Scenario	Published outcomes	Test coverage	Result
Customer corrects information given earlier	No published result
The agent offers what to ask next	No published result
The follow-up is ambiguous	No published result
The next question refers to the previous answer	No published result
The user changes direction mid-conversation	No published result
The user narrows what they just asked	No published result

Question Answering4 scenarios · 0 with published Results

Scenario	Published outcomes	Test coverage	Result
The answer is available in the connected data	No published result
The connected data cannot answer the question	No published result
The question uses a term the company defines itself	No published result
The user's wording doesn't match how the values are stored	No published result

Training4 scenarios · 0 with published Results

Scenario	Published outcomes	Test coverage	Result
The agent used the wrong definition and the user corrects it	No published result
The database's names are cryptic and the user documents them	No published result
The user sets a standing rule for all future answers	No published result
The user supplies example questions and the queries they trust	No published result

Published evidence and test coverage answer different questions.

Publication availability

A published Result is public evidence for this tool on one scenario. “No published result” does not say whether testing has taken place.

Test coverage

Assessed includes Pass, Fail and Not gradable. Gradable includes Pass and Fail. Both use the pinned test count in that published Result.

Scenario scope

The 28 scenarios describe this benchmark’s scope. They are not an assumed applicability or test-coverage denominator for Dot.