It adds interpretation instead of only listing rows: Rahul is framed as the strongest all-around customer, and the tool warns that Deepak and Karan's 'usual' payment method is not yet a stable preference because each has only one order.
What was measured
Business Insight
Does it explain what the result means?
decisive for this rankingtransformation
Explaining what the result means is part of making the database answer actually useful to a business user. (2 of 3 judges)
What was given, what came back
Test input: Best customers with unpaid-order and payment-method follow-ups · text · group: live-database-plain-english-queries
Input — what we sent
The exact prompt
Who are my best customers — the ones who order the most and spend the most? Follow-up 1: For the top 3 from that list — do any of them have unpaid orders? Follow-up 2: What payment methods do these top 3 usually use?
A conversational multi-table customer analysis with follow-up questions. It asks for the best customers by both order volume and spend, then drills into unpaid orders for the top 3 and their usual payment methods. Designed to test ranking logic, join-heavy analysis, and follow-up context retention.
Why this input is hard
- · ambiguous business term interpretation
- · multi-table joins
- · aggregation and ranking
- · follow-up context retention
- · scoping to a selected subset
- · payment behavior analysis
Output — unretouched
Also checked on this input — same tool, 4 other criteria
Ambiguity Handling✓ WorkedFaced with 'order the most and spend the most,' it did not silently choose one dimension; it built a combined equal-weight rank and explicitly surfaced the trade-off between all-around value and single-big-order spend.Follow-Up Context✓ WorkedIt carried the top 3 customers through both follow-ups and, on the payment-method question, correctly did not inherit the main query's paid/non-cancelled filter.Result Readability◐ MixedThe follow-up answers are readable but terse: each is a one-line response, while the supporting table and SQL are pushed out of the main surface into Full logs.SQL Generation✓ WorkedIt generated CTE-based SQL that joins customers to paid, non-cancelled orders, aggregates order_count and total_spend, and ranks customers across both measures.
Provenance
- Observation
- 86d2a8b3-d9d0-4b8e-8e46-8bc17c6b5c6d
- Evidence run
- af2abc96-3311-484b-a19d-854a2fdd2bf3
- Study
- Query Live Databases Using Plain English with AI
- Research task
- 86b9y6c99
- Tested at
- not recorded
- Source
- first-party
- Evidence state
- verified
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "dot",
scenario: "live-database-plain-english-queries"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 7 other tools
measured on Business Insight
AskYourDatabase✓ WorkedIt adds interpretation such as Rahul Sharma being the all-rounder and Mohan Vishe being a payment-risk red flag, rather than just listing rows.Basedash✓ WorkedIt explains what the numbers mean by naming Rahul Sharma as the best overall customer, Mohan Vishe as the most frequent, and Deepak Kulkarni as the biggest spender.BlazeSQL✓ WorkedIt explains the follow-up data in business terms, including that Rahul Sharma has 1 unpaid order worth $2,199 in CONFIRMED status while Deepak Kulkarni and Karan Joshi are fully paid up.camelAI◐ MixedIt gives a one-line business takeaway by naming Rahul Sharma as the strongest frequency-and-value combination, but it stops there without any deeper interpretation or recommendation.Definite✓ WorkedIt adds useful interpretation by flagging Rahul Sharma's unpaid $2,199 order, noting it had been confirmed but unpaid since April 2025, and linking his payment methods to the issue as likely an oversight rather than a pattern.Draxlr⚠ StruggledIt produced query descriptions, but no after-the-table narrative that distilled the result into a takeaway like which top customer had unpaid orders.FutureSmart NL2SQL Agent◐ MixedThe payment-method follow-up gives a concise dominant-method insight for the top customers, but the result is incomplete as a business answer because it does not surface the full payment-method picture implied by the customer totals.
This evidence is published in
From the same study (page rebuilt from a later run)
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com