All three best-customer result tables keep customer_id as the lead column, so the output stays UUID-heavy and hard to scan.
What was measured
Result Readability
Is the answer easy for a non-technical user to understand?
decisive for this rankingtransformation
The point is to get usable answers from non-technical users, so the result has to be understandable. (3 of 3 judges)
What was given, what came back
Test input: Best customers with unpaid-order and payment-method follow-ups · text · group: live-database-plain-english-queries
Input — what we sent
The exact prompt
Who are my best customers — the ones who order the most and spend the most? Follow-up 1: For the top 3 from that list — do any of them have unpaid orders? Follow-up 2: What payment methods do these top 3 usually use?
A conversational multi-table customer analysis with follow-up questions. It asks for the best customers by both order volume and spend, then drills into unpaid orders for the top 3 and their usual payment methods. Designed to test ranking logic, join-heavy analysis, and follow-up context retention.
Why this input is hard
- · ambiguous business term interpretation
- · multi-table joins
- · aggregation and ranking
- · follow-up context retention
- · scoping to a selected subset
- · payment behavior analysis
Output — unretouched
Also checked on this input — same tool, 5 other criteria
Ambiguity Handling✓ WorkedIt resolved the vague phrase best customers as highest order count plus highest total spend instead of asking for a definition.Business Insight⚠ StruggledIt produced query descriptions, but no after-the-table narrative that distilled the result into a takeaway like which top customer had unpaid orders.Follow-Up Context✓ WorkedIt kept the top 3 from that list context intact across both follow-ups and continued analyzing the same three customers.Plain English Query Handling✓ WorkedIt accepted the informal best-customers request and both follow-up questions across the three-turn conversation.SQL Generation✓ WorkedAcross the three turns it generated production-grade SQL, including ranking logic, unpaid-order checks, and payment-method aggregation with CTEs, ROW_NUMBER, COALESCE, and LEFT JOIN.
Provenance
- Observation
- 492b3c4c-2b5a-42db-80b7-7b8f2464e506
- Evidence run
- af2abc96-3311-484b-a19d-854a2fdd2bf3
- Study
- Query Live Databases Using Plain English with AI
- Research task
- 86b9y6c99
- Tested at
- not recorded
- Source
- first-party
- Evidence state
- verified
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "draxlr",
scenario: "live-database-plain-english-queries"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 5 other tools
measured on Result Readability
AskYourDatabase✓ WorkedIt presents the answer as clearly labeled ranking tables and customer-level payment summaries, with visual risk cues for unpaid or shipped orders.Basedash✓ WorkedThe output stays readable for non-technical users, using compact tables and short summaries with explicit customer names and amounts.Definite✓ WorkedIt returns a ranked table with order counts, total spend, and average order value that is straightforward to read and scan.Dot◐ MixedThe follow-up answers are readable but terse: each is a one-line response, while the supporting table and SQL are pushed out of the main surface into Full logs.Querio◐ MixedThe answer is usable, but it is less readable than it could be because the output centers customer UUIDs instead of human names.
This evidence is published in
From the same study (page rebuilt from a later run)
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com