It added useful follow-up commentary by flagging Rahul Sharma's unpaid $2,199 order as worth chasing because he was #2 by spend, and by linking that unpaid order to his payment behavior as likely an oversight rather than a pattern.
What was measured
Business Insight
Does it explain what the result means?
transformation
What was given, what came back
Test input: Best customers with unpaid-order and payment-method follow-ups · text · group: ecommerce-nl2sql-benchmark
Input — what we sent
The exact prompt
Who are my best customers — the ones who order the most and spend the most? Follow-up 1: For the top 3 from that list — do any of them have unpaid orders? Follow-up 2: What payment methods do these top 3 usually use?
A conversational multi-table customer analysis that identifies best customers by both order volume and spend, then drills into unpaid orders and payment methods for the top 3.
Why this input is hard
- · Ambiguous business-term interpretation
- · Multi-table joins across customers orders and payments
- · Aggregation and ranking
- · Follow-up context retention
- · Scoped drill-down to the top 3 customers
- · Readable customer-level output
Output — unretouched
Also checked on this input — same tool, 3 other criteria
Ambiguity Handling✗ FailedIt did not clarify the ambiguous 'best customers' request and instead silently narrowed the task to total spend, ignoring the 'order the most' part of the question.Follow-Up Context✓ WorkedIt retained context across follow-ups, allowing the user to ask about unpaid orders and then payment methods for the top 3 from the earlier spend-ranked list.Result Readability✓ WorkedThe ranked customer table was readable and easy to scan, with rank, orders, total spend, and average order value clearly laid out for the top 20 customers.
Provenance
- Observation
- 6161c827-6469-4922-ac82-0511d300dc87
- Evidence run
- db2bb5d5-0e0e-4cb3-8d76-3555c45c23cd
- Study
- Query Live Databases Using Plain English with AI
- Research task
- 86b9y6c99
- Tested at
- not recorded
- Source
- first-party
- Evidence state
- verified
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "definite",
scenario: "ecommerce-nl2sql-benchmark"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 2 other tools
measured on Business Insight
AskYourDatabase✓ WorkedGenerated useful takeaways automatically, including Rahul Sharma as the standout all-rounder, Mohan Vishe as the most loyal frequent buyer, and Vikram Singh as the largest spender.Basedash✓ WorkedIt turns the ranking tables into a plain-English business takeaway by naming Rahul Sharma as the best overall customer, Mohan Vishe as the most frequent buyer, and Deepak Kulkarni as the biggest spender.
This evidence is published in
From the same study (page rebuilt from a later run)
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com