Explains the comparison rather than just reporting numbers, including that the month-to-date view is an early signal and should not be treated as a stable trend yet.
What was measured
Business Insight
Does it explain what the result means?
transformation
What was given, what came back
Test input: Order pipeline breakdown with paid-pending edge case and last-month comparison · text · group: ecommerce-nl2sql-benchmark
Input — what we sent
The exact prompt
How many orders do we have at each stage right now? Follow-up 1: What percentage of our orders were successfully delivered vs cancelled? Follow-up 2: Are there any orders that are pending but already paid? Follow-up 3: Compare that to last month — same breakdown, I want to see if things have improved or got worse.
A deeper operational analysis of current order stages, delivery-versus-cancellation rates, pending-but-paid edge cases, and a month-over-month comparison of the same breakdown.
Why this input is hard
- · Order pipeline analysis
- · Percentage calculation
- · Edge-case detection
- · Payment/order status joins
- · Multi-turn context retention
- · Month-over-month comparison
- · Ambiguity handling for 'same breakdown'
Output — unretouched
Also checked on this input — same tool, 5 other criteria
Chart / Visualization Support✓ WorkedAutomatically generates chart views for the order-stage breakdown and the delivered-vs-cancelled comparison, including the month-over-month share chart.Follow-Up Context⚠ StruggledKeeps the conversation going across multiple turns, but on the final comparison it reuses the delivered-versus-cancelled thread instead of the pending-paid edge-case context, so context selection breaks on one follow-up.Plain English Query Handling✓ WorkedAccepts a multi-step operational question in plain English across several follow-ups without requiring SQL from the user.Result Readability✓ WorkedPresents the operational breakdown in compact tables with counts and percentages, making the current-stage and month-over-month results easy to scan.SQL Visibility✓ WorkedExposes the generated SQL directly in the chat, including the explicit filter for pending-but-paid orders.
Provenance
- Observation
- 3f5103f1-b095-4b6f-b13e-20b0b7dc4af3
- Evidence run
- db2bb5d5-0e0e-4cb3-8d76-3555c45c23cd
- Study
- Query Live Databases Using Plain English with AI
- Research task
- 86b9y6c99
- Tested at
- not recorded
- Source
- first-party
- Evidence state
- verified
- Proof shown
- input + output shown
- Cost / latency
- not captured
- Repeat run
- not captured
- Tester
- not captured
The last three rows are honest blanks, not placeholders — our capture has no field for them yet.
Query this
get_evidence({
tool: "querio",
scenario: "ecommerce-nl2sql-benchmark"
})MCP · mcp.aidemos.com/api/mcp
Free with attribution.
Same input, same check — 3 other tools
measured on Business Insight
AskYourDatabase✓ WorkedThe follow-up summaries explained what the raw counts meant operationally, including delivery/cancellation rates and the backlog reduction story.Definite✓ WorkedIt explained what the numbers meant, including the 14% cancellation rate, the completed-orders caveat, the 2 pending-but-paid orders that need attention, and the warning that May is still early because 17 of 21 orders remain pending.Draxlr✓ WorkedCan turn numeric results into plain-English takeaways; the delivered-vs-cancelled follow-up automatically summarized the split and framed delivered orders as more common than cancelled orders.
This evidence is published in
From the same study (page rebuilt from a later run)
Real inputs and real outputs, no retouching · every cell queryable via API & MCP · aidemos.com