AI Demos Research — the structured-intelligence platform. Every verdict on these pages opens to the execution behind it.
Graded 16 November 2026

Can camelAI answer questions using connected database data?

camelAI answered three of the four database questions as the reference queries did: 10 more orders in August than July, an August total of $14,403.87 across 28 orders, and a tie at 6 units for the top product. Asked for orders in the last 30 days as of 2 September, it counted from 4 August instead of 3 August and gave 24, against the reference 26.

3 of 4 test cases passed · 1 failed

Every test case in this scenario has a result.

Pass rate75%3 of 4 gradable
Coverage4 of 4test cases with a result
3PassDid everything it was expected to do.
1FailMissed at least one thing it was expected to do; the reason and proof are with each test case below.
0Not gradableEvery result here could be graded.
0UntestedEvery test case in this scenario has a result.
The pass rate is a summary. The evidence is the 4 test cases below: what we sent, what we checked, what the tool returned, and the proof.

The 4 test cases

Each test case is judged on its own: Pass, Fail, Not gradable, or Untested. The scenario result above is based on these test cases.

Order count in a defined periodFailEvidence
What we sent

“As of September 2, 2026, how many orders were placed in the last 30 days?”

What the tool returned

24 orders were placed in the 30 calendar days from August 4 through September 2, 2026 (inclusive), based on public.orders.placed_at in the connected database. The latest matching order was placed on September 1 UTC.

Transcribed from Proof 1 · Screenshot

Every message the tool sent, in order (2)
  1. Query camelAI ran · not part of the answer
    const r = await tools.camel__connections_query({connection:"customer_support_db", query:"SELECT COUNT(*) AS orders_last_30_calendar_days, MIN(placed_at) AS earliest_in_window, MAX(placed_at) AS latest_in_window, COUNT(*) FILTER (WHERE placed_at IS NULL) AS null_placed_at FROM public.orders WHERE placed_at >= TIMESTAMPTZ '2026-08-04 00:00:00+00' AND placed_at < TIMESTAMPTZ '2026-09-03 00:00:00+00'"}); return r;
  2. Answer

    The final response, printed above.

Expected vs. Found
✗Expected: 26 orders from 3 August through 2 September 2026: the 30 days before the as-of date plus the as-of date itself.Found: camelAI answered 24 orders. The reference query returned 26 for 3 August through 2 September, and 24 for camelAI's own window of 4 August through 2 September.
✗Expected: The answer covers 3 August through 2 September 2026 inclusive (30 days before the as-of date, plus that date), stated or evident.Found: camelAI stated its window as 4 August through 2 September 2026, inclusive, and the query it ran used that window, which starts a day after 3 August.
Supporting proof
Proof 1Screenshot
Screenshot: the question and camelAI's answer of 24 orders from 4 August to 2 September
Open original ↗
Proof 2Screenshot
Screenshot: camelAI's work trace, with the count query it ran and its result
Open original ↗
Proof 3Screen recording
Screen recording: the question, camelAI's answer, its work trace and the reference query
Key moments
Open original ↗
Proof 4Screenshot
Reference query result: 26 orders from 3 August, 24 from 4 August
Open original ↗
Why this result

The test database was refreshed after the test was written, so each answer is compared with a reference query run at test time. The window is the 30 days before 2 September plus that day, so it starts 3 August; camelAI counted from 4 August and returned 24, not 26.

Also observed in this test case

Date window stated in answer. camelAI's answer named its date window and the latest matching order, and its query also checked for orders with no order date, so the one-day difference in the window is visible from the answer itself.

Tested by Anchal Rana · evidence dated 27 October 2026
Order-count difference between two monthsPassEvidence
What we sent

“As of September 2, 2026, how many more orders were placed in August 2026 than in July 2026?”

What the tool returned

10 more orders were placed in August 2026: 28 in August versus 18 in July, based on the orders.placed_at dates in the connected database.

Transcribed from Proof 1 · Screenshot

Every message the tool sent, in order (2)
  1. Query camelAI ran · not part of the answer
    return await tools.camel__connections_query({connection:"customer_support_db", query:"SELECT date_trunc('month', placed_at)::date AS month, COUNT(*) AS orders_placed, MIN(placed_at) AS first_order, MAX(placed_at) AS last_order FROM orders WHERE placed_at >= TIMESTAMPTZ '2026-07-01 00:00:00+00' AND placed_at < TIMESTAMPTZ '2026-09-01 00:00:00+00' GROUP BY 1 ORDER BY 1"});
  2. Answer

    The final response, printed above.

Expected vs. Found
✓Expected: August had 10 more orders than July (28 against 18), as a reference query on the database returned at test time.Found: camelAI answered 10 more orders in August. The reference query returned 28 August orders, 18 July orders and a difference of 10.
✓Expected: August had more orders than July (28 against 18 at test time).Found: camelAI named August as the larger month and gave both counts in its answer: 28 in August and 18 in July.
Supporting proof
Proof 1Screenshot
Screenshot: the question and camelAI's answer, 10 more orders in August than July
Open original ↗
Proof 2Screenshot
Screenshot: camelAI's work trace, counting 18 July orders and 28 August orders
Open original ↗
Proof 3Screen recording
Screen recording: the question, camelAI's answer, its work trace and the reference query
Key moments
Open original ↗
Proof 4Screenshot
Reference query result: 28 August orders, 18 July orders, a difference of 10
Open original ↗
Why this result

Both the difference and its direction had to agree with a reference query run at test time. camelAI's answer did, and the query it ran used the same calendar-month boundaries as the reference.

Tested by Anchal Rana · evidence dated 27 October 2026
Total order value in a calendar monthPassEvidence
What we sent

“As of September 2, 2026, what was the combined total amount of all orders placed in August 2026?”

What the tool returned

Orders placed in August 2026 totaled $14,403.87 USD across 28 orders. I summed orders.total_amount for orders placed from August 1 through August 31, 2026.

Transcribed from Proof 1 · Screenshot

Every message the tool sent, in order (2)
  1. Query camelAI ran · not part of the answer
    const r=await tools.camel__connections_query({connection:"customer_support_db",query:"SELECT currency, COUNT(*) AS order_count, SUM(total_amount) AS combined_total, MIN(placed_at) AS first_placed_at, MAX(placed_at) AS last_placed_at FROM public.orders WHERE placed_at >= TIMESTAMPTZ '2026-08-01 00:00:00+00' AND placed_at < TIMESTAMPTZ '2026-09-01 00:00:00+00' GROUP BY currency ORDER BY currency"}); return r;
  2. Answer

    The final response, printed above.

Expected vs. Found
✓Expected: The August 2026 total is $14,403.87 across 28 orders, the result of a reference query on the database at test time.Found: camelAI answered $14,403.87 across 28 orders; the reference query returned 28 orders totalling 14,403.87 US dollars.
✓Expected: The answer covers August 2026 and all orders, with none left out because of their status.Found: camelAI stated that it summed the order totals for 1 through 31 August 2026, and its query had no status filter, like the reference query.
Supporting proof
Proof 1Screenshot
Screenshot: the question and camelAI's answer, $14,403.87 across 28 August orders
Open original ↗
Proof 2Screenshot
Screenshot: camelAI's work trace, its August total grouped by currency
Open original ↗
Proof 3Screen recording
Screen recording: the question, camelAI's answer, its work trace and the reference query
Key moments
Open original ↗
Proof 4Screenshot
Reference query result: 28 August orders totalling 14,403.87 US dollars
Open original ↗
Why this result

The total had to agree with a reference query run at test time, over every August order. camelAI's $14,403.87 across 28 orders equals the reference, and its query used the same window with no status filter.

Also observed in this test case

Total grouped by currency. camelAI split its August total by currency instead of assuming one; its single US dollar row shows no mixed-currency sum is hidden in the $14,403.87.

Tested by Anchal Rana · evidence dated 27 October 2026
Top product by units ordered in a monthPassEvidence
What we sent

“As of September 2, 2026, which product had the most units ordered in August 2026, and how many units?”

What the tool returned

There's a tie for the most units ordered in August 2026: Brass picture frame, 8×10 and Round wall mirror, 24in, with 6 units each. I summed order-item quantities for orders placed from August 1 through August 31 (UTC).

Transcribed from Proof 1 · Screenshot

Every message the tool sent, in order (2)
  1. Query camelAI ran · not part of the answer
    const r = await tools.camel__connections_query({connection:"customer_support_db", query:"SELECT p.id AS product_id, p.name AS product, SUM(oi.quantity)::bigint AS units_ordered, COUNT(DISTINCT o.id)::bigint AS orders FROM orders o JOIN order_items oi ON oi.order_id=o.id JOIN products p ON p.id=oi.product_id WHERE o.placed_at >= TIMESTAMPTZ '2026-08-01 00:00:00+00' AND o.placed_at < TIMESTAMPTZ '2026-09-01 00:00:00+00' GROUP BY p.id, p.name ORDER BY units_ordered DESC, p.name ASC LIMIT 10"}); return r;
  2. Answer

    The final response, printed above.

Expected vs. Found
✓Expected: The top product is the tie between Brass picture frame, 8x10 and Round wall mirror, 24in, as a reference query returned at test time.Found: camelAI reported a tie between Brass picture frame, 8x10 and Round wall mirror, 24in. The reference query returned both at 6 units and the next products at 5.
✓Expected: 6 units for each tied product, as the reference query returned at test time.Found: camelAI answered 6 units each; its own query returned 6 units for both products and 5 for the next two.
Supporting proof
Proof 1Screenshot
Screenshot: the question and camelAI's answer naming two products tied at 6 units
Open original ↗
Proof 2Screenshot
Screenshot: camelAI's work trace, its units-by-product query and its top rows
Open original ↗
Proof 3Screen recording
Screen recording: the question, camelAI's answer, its work trace and the reference query
Key moments
Open original ↗
Proof 4Screenshot
Reference query result: the top five products by August units, two tied at 6
Open original ↗
Why this result

Any product tied for the most units counts as the top product. The reference query run at test time showed a tie at 6 units, and camelAI named both tied products with 6 units each, using the same join and date window.

Also observed in this test case

Tie kept visible in query. camelAI said plainly there was a tie instead of picking one product, and its query listed the top ten products by units and then by name, so both tied products show in its work trace.

Tested by Anchal Rana · evidence dated 27 October 2026

Configuration and setup

How this tool was set up for the run and what the test needed in place. Each row is a fact from the run's records; a fact the records do not hold is left out, not guessed.

Software that produced the output
camelAI
Build or version
Not exposed by the hosted product · model camelCode
Plan or tier
Free
Surface
Web app
Tested
27 October 2026 · Anchal Rana

How this scenario is graded

How we decide Pass, Fail and Not gradable. The same rules apply to every tool tested on this scenario.

How results are decided

Each test case gets one result per tool: Pass, Fail or Not gradable. A test case we haven't run yet shows Untested. There are no partial results.

The rules
  • Pass: the tool did everything the test expected, and nothing it said contradicts the correct answer.
  • Fail: at least one expected behaviour clearly didn't happen; the test case's result says which one and quotes the tool.
  • Not gradable: our evidence couldn't settle the outcome (for example a record we needed is missing). It is never counted as a fail, and the test case's result says what's missing.

Where this sits in the benchmark

This page is one cell of a larger study: one tool, one scenario. Only this benchmark's frame appears here.

History of this result

What has happened to this result since it was first published. Runs and grades are never overwritten: a retest or a re-grade publishes a new result and keeps the earlier one readable.

from the publication record
20 November 2026First publishedAI Database Agents v1

Act on this result

Nothing filed here edits the run or the grade. A challenge opens a review, and a review can produce a new run or a re-grade — which becomes the current result and leaves this one in the history.

This matches what I see

You run the same kind of test against your own setup and get the same behaviour.

Agree →
This does not match

Yours behaves differently. Tell us what you got, with a screenshot if you have one.

Disagree →
Point out an issue

Something here is wrong — a reference value, a transcription, a grade.

Report an issue →
Request a retest

On a newer build, a larger dataset, or your own setup.

Request a retest →
We have fixed this

Tell us what changed and we schedule a rerun of the failing test case. The old result stays as history.

Vendor notice →
Filed against this evidenceNothing yet. Challenges, counter-evidence and fix notices appear here with their outcome, and stay on the page after they are resolved.
Cite this result
aidemos.com/benchmarks/ai-database-agents/results/camelai/the-answer-is-available-in-the-connected-data · 3 pass · 1 fail · coverage 4/4 · graded 2026-11-16

The same record is available as structured data through the AI Demos MCP server, with the counts, the coverage and every per-test-case reason carried as fields.

Verify the proof files

These files support this result. Open a file to inspect the original evidence.

File fingerprints (SHA-256)

A fingerprint identifies the exact file used for this result.

Order count in a defined period · Proof 1 · Screenshot5f5dafae76fddcb9a543549d31a82070fabfc4d6fe92cd99b0a3629b268ed381
Order count in a defined period · Proof 2 · Screenshot99c3b69f72c522ccf98b8ebca06c23b9628e2f154321f7543d22720e7fedc69a
Order count in a defined period · Proof 3 · Screen recording3a4e7393152065bc5d005e1a30f35bdf855d7bf551e29aa6d4a19ec19a5509db
Order count in a defined period · Proof 4 · Screenshot103a062af4e8fabd1e8ff8affafa595da48766bbbc952e1c5586181f0b68b6b1
Order-count difference between two months · Proof 1 · Screenshot0a0369859b3398f22dd09526512656d1406c139852fc5e35b42b3478187a763f
Order-count difference between two months · Proof 2 · Screenshot4622d5de8140503834272212fc8c2490eb4e4f7ff5594ff9fd7282b12820e9c0
Order-count difference between two months · Proof 3 · Screen recording3e35879d4f3878b7b199c7199deff18f515f3056907a167179ef96404034427d
Order-count difference between two months · Proof 4 · Screenshot323812375615131965a36a4f3bae1748cde2caba9a2dc245b99ec16143e07129
Total order value in a calendar month · Proof 1 · Screenshotaa71d5afa7e67c6de37273346b164443211a9385c58b3b832a545c241260bf6a
Total order value in a calendar month · Proof 2 · Screenshot0a04c4f94488a35185c2a9ed7e088fc35ab6325eccb91742dadba6220bb1217d
Total order value in a calendar month · Proof 3 · Screen recordingdd5752e774d69257df54b86a60b1f5305bb72eeef47cb254d272a292b9e2b3b2
Total order value in a calendar month · Proof 4 · Screenshota61c865a0264f427795705bb2d670f52a0961b040f5ac80dc707594e8d5b3faf
Top product by units ordered in a month · Proof 1 · Screenshotefc692b0bd669c951805fbd5ba69d78806417471301dcd9ab84ac5a033f1c236
Top product by units ordered in a month · Proof 2 · Screenshot1f4808ced26c7353c99e5820a2df3d0fc47228563d05e66cd26cc222644ca6b5
Top product by units ordered in a month · Proof 3 · Screen recordingf178fdf9d24e47ab5d544839e44421538942ae48b3ca0d291734d9b8821045a6
Top product by units ordered in a month · Proof 4 · Screenshot6f999af06e15ca69cdfa826c486cbf54bc1eaae17712a99ca68439dc57df6435