AI Demos Research — the structured-intelligence platform. Every verdict on these pages opens to the execution behind it.
Graded 18 October 2026

Can AskYourDatabase answer questions from connected data?

AskYourDatabase answered three connected-data questions correctly, but missed the 30-day order count. It returned 24 instead of 25 for that lookup, while it got the month difference, August total, and top product by units right.

3 of 4 test cases passed · 1 failed

Every test case in this scenario has a result.

Pass rate75%3 of 4 gradable
Coverage4 of 4test cases with a result
3PassDid everything it was expected to do.
1FailMissed at least one thing it was expected to do; the reason and proof are with each test case below.
0Not gradableEvery result here could be graded.
0UntestedEvery test case in this scenario has a result.
The pass rate is a summary. The evidence is the 4 test cases below: what we sent, what we checked, what the tool returned, and the proof.

The 4 test cases

Each test case is judged on its own: Pass, Fail, Not gradable, or Untested. The scenario result above is based on these test cases.

Order count in a defined periodFailEvidence
What we sent

“As of September 2, 2026, how many orders were placed in the last 30 days?”

What the tool returned

As of September 2, 2026, 24 orders were placed in the last 30 days (from August 3 through September 1, 2026).

Transcribed from Proof 1 · Screenshot

Every message the tool sent, in order (3)
  1. Progress

    I'll query the database to find orders placed in the last 30 days before September 2, 2026.

  2. Generated SQL · not part of the answer
    SELECT COUNT(*) AS order_count
    FROM orders
    WHERE placed_at >= '2026-08-03' AND placed_at < '2026-09-02'
  3. Answer

    The final response, printed above.

Expected vs. Found
✗Expected: 25 orders in the 30 days ending 2 September 2026.Found: It gave 24 orders; the reference count for the connected data was 25.
✗Expected: The count should cover 3 August through 2 September 2026, including the as-of date.Found: It said the window ran from 3 August through 1 September, so 2 September was left out.
Supporting proof
Proof 1Screenshot
Chat screenshot: 30-day count query and 24-order answer
Open original ↗
Why this result

It counted through 1 September instead of through 2 September, so the 2 September order was left out and the answer came back one low.

Also observed in this test case

Window shown plainly. The answer states the date window in plain words, so the counted period is visible without opening the SQL.

Count label visible. The SQL block makes the count label visible.

Tested by Ayush Ghosh · evidence dated 6 October 2026
Order-count difference between two monthsPassEvidence
What we sent

“As of September 2, 2026, how many more orders were placed in August 2026 than in July 2026?”

What the tool returned

Answer

In August 2026, there were 3 more orders placed than in July 2026.

Order counts in the answer

Month | Orders Placed; July 2026 | 23; August 2026 | 26

Answer restatement

August had 26 orders compared to July's 23 — a difference of 3 orders.

Transcribed from Proof 1 · Screenshot

Every message the tool sent, in order (5)
  1. Progress

    I'll compare order counts between July and August 2026.

  2. Generated SQL · not part of the answer
    SELECT
      COUNT(*) FILTER (WHERE placed_at >= '2026-07-01' AND placed_at < '2026-08-01') AS july_orders,
      COUNT(*) FILTER (WHERE placed_at >= '2026-08-01' AND placed_at < '2026-09-01') AS august_orders
    FROM orders;
  3. Answer

    The final response, printed above.

  4. Order counts in the answer

    The final response, printed above.

  5. Answer restatement

    The final response, printed above.

Expected vs. Found
✓Expected: August 2026 should have 3 more orders than July 2026: 26 versus 23.Found: It gave 3 more orders, matching the reference difference of 3 (26 in August and 23 in July).
✓Expected: August 2026 should have more orders than July 2026: 26 versus 23.Found: It said August had more orders than July, and the table shows 26 versus 23.
Supporting proof
Proof 1Screenshot
Chat screenshot: month comparison query, table, and 3-order answer
Open original ↗
Why this result

It compared the two full months and gave August the higher count, with a difference of 3 orders.

Also observed in this test case

Three result forms. The answer shows the difference, both month counts, and the direction in a sentence, a table, and a closing line.

Tested by Ayush Ghosh · evidence dated 6 October 2026
Total order value in a calendar monthPassEvidence
What we sent

“As of September 2, 2026, what was the combined total amount of all orders placed in August 2026?”

What the tool returned

The combined total amount of all orders placed in August 2026 was $13,441.94.

Transcribed from Proof 1 · Screenshot

Every message the tool sent, in order (3)
  1. Progress

    I'll query the database to find the total amount of orders placed in August 2026.

  2. Generated SQL · not part of the answer
    SELECT COALESCE(SUM(total_amount), 0) AS combined_total
    FROM orders
    WHERE placed_at >= '2026-08-01' AND placed_at < '2026-09-01'
  3. Answer

    The final response, printed above.

Expected vs. Found
✓Expected: $13,441.94 combined across all August 2026 orders.Found: It gave $13,441.94, matching the reference total of $13,441.94 for the connected data.
✓Expected: It should cover August 2026 and include all orders.Found: Its SQL covers all August orders and the answer says all orders placed in August 2026.
Supporting proof
Proof 1Screenshot
Chat screenshot: August total query and $13,441.94 answer
Open original ↗
Why this result

It summed all August orders with no status filter and returned the expected total of $13,441.94.

Also observed in this test case

Period in one line. The answer gives the total and repeats the month, but the status breakdown is only visible in the SQL.

Tested by Ayush Ghosh · evidence dated 6 October 2026
Top product by units ordered in a monthPassEvidence
What we sent

“As of September 2, 2026, which product had the most units ordered in August 2026, and how many units?”

What the tool returned

As of September 2, 2026, the product with the most units ordered in August 2026 was the Soy candle trio, with 5 units ordered.

Transcribed from Proof 1 · Screenshot

Every message the tool sent, in order (3)
  1. Progress

    I'll find which product had the most units ordered in August 2026.

  2. Generated SQL · not part of the answer
    SELECT p.name, SUM(oi.quantity) AS total_units
    FROM order_items oi
    JOIN orders o ON o.id = oi.order_id
    JOIN products p ON p.id = oi.product_id
    WHERE o.placed_at >= '2026-08-01' AND o.placed_at < '2026-09-01'
    GROUP BY p.name
    ORDER BY total_units DESC
    LIMIT 1;
  3. Answer

    The final response, printed above.

Expected vs. Found
✓Expected: A top product for August 2026: Brass picture frame 8x10, Outdoor rug 6x9, or Soy candle trio, tied at 5 units.Found: It named Soy candle trio, one of three products tied at the reference maximum of 5 units.
✓Expected: 5 units, the highest product total for August 2026.Found: It gave 5 units, the maximum on the data it queried.
Supporting proof
Proof 1Screenshot
Chat screenshot: top-product query and 5-unit answer
Open original ↗
Why this result

It summed August units across products and returned Soy candle trio with 5 units, which matches one of the tied maxima.

Also observed in this test case

Single top product. The answer names one product only, so the tie is not visible from the message alone.

No tie breaker. The SQL stops after one row without a tie-breaker, so a tied winner can vary.

Tested by Ayush Ghosh · evidence dated 6 October 2026

Configuration and setup

How this tool was set up for the run and what the test needed in place. Each row is a fact from the run's records; a fact the records do not hold is left out, not guessed.

Software that produced the output
AskYourDatabase
Build or version
Not exposed by the hosted product
Surface
Web app
Tested
6 October 2026 · Ayush Ghosh

How this scenario is graded

How we decide Pass, Fail and Not gradable. The same rules apply to every tool tested on this scenario.

How results are decided

Each test case gets one result per tool: Pass, Fail or Not gradable. A test case we haven't run yet shows Untested. There are no partial results.

The rules
  • Pass: the tool did everything the test expected, and nothing it said contradicts the correct answer.
  • Fail: at least one expected behaviour clearly didn't happen; the test case's result says which one and quotes the tool.
  • Not gradable: our evidence couldn't settle the outcome (for example a record we needed is missing). It is never counted as a fail, and the test case's result says what's missing.

Where this sits in the benchmark

This page is one cell of a larger study: one tool, one scenario. Only this benchmark's frame appears here.

History of this result

What has happened to this result since it was first published. Runs and grades are never overwritten: a retest or a re-grade publishes a new result and keeps the earlier one readable.

from the publication record
13 November 2026First publishedAI Database Agents v1

Act on this result

Nothing filed here edits the run or the grade. A challenge opens a review, and a review can produce a new run or a re-grade — which becomes the current result and leaves this one in the history.

This matches what I see

You run the same kind of test against your own setup and get the same behaviour.

Agree →
This does not match

Yours behaves differently. Tell us what you got, with a screenshot if you have one.

Disagree →
Point out an issue

Something here is wrong — a reference value, a transcription, a grade.

Report an issue →
Request a retest

On a newer build, a larger dataset, or your own setup.

Request a retest →
We have fixed this

Tell us what changed and we schedule a rerun of the failing test case. The old result stays as history.

Vendor notice →
Filed against this evidenceNothing yet. Challenges, counter-evidence and fix notices appear here with their outcome, and stay on the page after they are resolved.
Cite this result
aidemos.com/benchmarks/ai-database-agents/results/askyourdatabase/the-answer-is-available-in-the-connected-data · 3 pass · 1 fail · coverage 4/4 · graded 2026-10-18

The same record is available as structured data through the AI Demos MCP server, with the counts, the coverage and every per-test-case reason carried as fields.

Verify the proof files

These files support this result. Open a file to inspect the original evidence.

File fingerprints (SHA-256)

A fingerprint identifies the exact file used for this result.

Order count in a defined period · Proof 1 · Screenshotca12702a1f57a2a1a1453aba4b494331ddff61952044a9d4979dc94d3de14004
Order-count difference between two months · Proof 1 · Screenshot73c94f1d5d689b38a370396461e7c448c5a3472b323b5dfdb74d6869d9a1bbd3
Total order value in a calendar month · Proof 1 · Screenshota2651ef3ae583392fa428d1d5878957bcc4c2a90229c97721631e18e2b72cd82
Top product by units ordered in a month · Proof 1 · Screenshotca132877befc212f87b56124a4a0b8265d3b8530b1670facb7a001783c086955