Can BlazeSQL preserve earlier constraints when a user narrows a request?
BlazeSQL preserved the earlier 90-day window when the user narrowed the request to Texas. The follow-up query kept the original date filter and added only the Texas filter, and the Texas export matched the Texas subset of the first export.
1 of 1 test case passed
Every test case in this scenario has a result.
The test case
Each test case is judged on its own: Pass, Fail, Not gradable, or Untested. The scenario result above is based on this test case.
Narrow the same question to one regionPassEvidence
Input 1: “Show me all orders from the last 90 days, as of September 2, 2026.”
Input 2: “Only the ones shipped to Texas”
The tool returned 2 files, shown under Proofs.
Response 1 replies to Input 1.
Response 2 replies to Input 2.
Response to input 1 91 orders were returned for the 90-day window (June 4 – September 2, 2026). A few quick observations: • 📦 Most orders are delivered, with a subset still in processing or packed stages (the most recent ones) • ❌ Several cancellations are scattered throughout the period • 💰 All orders are in USD, spanning a wide range of totals Would you like to dig deeper — e.g., break down by status, category, or customer? Response to input 2 7 orders shipped to Texas in that period. Would you like to do anything further with these results?
Transcribed from Proof 1 · Screenshot
Transcribed from Proof 2 · Screenshot
| order_number | customer_name | customer_email | placed_at | status | category | total_amount | currency | city | state | payment_method |
|---|---|---|---|---|---|---|---|---|---|---|
| ORD-2026-0091 | Ingrid Sorensen | ingrid.sorensen@example.com | 2026-09-01 23:32:58.158541 | processing | storage_and_organization | 144 | USD | Tucson | AZ | google_pay |
| ORD-2026-0090 | Wei Zhang | wei.zhang@example.com | 2026-09-01 23:05:58.158541 | processing | kitchen_and_dining | 207.99 | USD | Evanston | IL | paypal |
| ORD-2026-0089 | Priya Raghavan | priya.raghavan@example.com | 2026-09-01 22:43:58.158541 | processing | bedding_and_rugs | 156.99 | USD | Reading | PA | card |
| ORD-2026-0088 | Ingrid Sorensen | ingrid.sorensen@example.com | 2026-09-01 22:08:58.158541 | processing | home_decor | 59.98 | USD | Pittsburgh | PA | card |
| ORD-2026-0087 | Jae-won Park | jae-won.park@example.com | 2026-09-01 17:24:58.158541 | processing | bedding_and_rugs | 638.98 | USD | Seattle | WA | apple_pay |
| ORD-2026-0086 | Nathan Boone | nathan.boone@example.com | 2026-09-01 16:38:58.158541 | processing | outdoor_and_patio | 616.99 | USD | Nashville | TN | card |
| ORD-2026-0085 | Wei Zhang | wei.zhang@example.com | 2026-09-01 14:29:58.158541 | processing | furniture | 3244.89 | USD | Evanston | IL | affirm |
| ORD-2026-0084 | Priya Raghavan | priya.raghavan@example.com | 2026-09-01 13:08:58.158541 | processing | lighting | 525.5 | USD | Reading | PA | card |
| ORD-2026-0083 | Marcus Whitfield | marcus.whitfield@example.com | 2026-09-01 11:49:58.158541 | packed | home_decor | 650.51 | USD | Denver | CO | paypal |
| ORD-2026-0082 | Dev Raghavan | dev.raghavan@example.com | 2026-09-01 11:07:58.158541 | packed | lighting | 683.91 | USD | Boulder | CO | card |
| ORD-2026-0080 | Priya Raghavan | priya.raghavan@example.com | 2026-09-01 10:58:58.158541 | packed | storage_and_organization | 99 | USD | Reading | PA | card |
| ORD-2026-0081 | Wei Zhang | wei.zhang@example.com | 2026-09-01 10:58:58.158541 | packed | home_decor | 504.84 | USD | Evanston | IL | card |
| ORD-2026-0079 | Marcus Whitfield | marcus.whitfield@example.com | 2026-08-31 11:25:58.158541 | packed | storage_and_organization | 166.99 | USD | Denver | CO | card |
| ORD-2026-0078 | Nathan Boone | nathan.boone@example.com | 2026-08-31 11:15:58.158541 | packed | bedding_and_rugs | 296.5 | USD | Boulder | CO | card |
| ORD-2026-0077 | Ingrid Sorensen | ingrid.sorensen@example.com | 2026-08-31 11:07:58.158541 | packed | outdoor_and_patio | 579.99 | USD | Portland | OR | card |
| ORD-2026-0076 | Ana Lucia Ferreira | ana.luciaferreira@example.com | 2026-08-30 11:48:58.158541 | shipped | home_decor | 468.9 | USD | Providence | RI | card |
| ORD-2026-0075 | Cassandra Pope | cassandra.pope@example.com | 2026-08-30 11:27:58.158541 | shipped | kitchen_and_dining | 1448.06 | USD | Charlotte | NC | google_pay |
| ORD-2026-0074 | Yewande Adeyemi | yewande.adeyemi@example.com | 2026-08-30 11:05:58.158541 | shipped | bedding_and_rugs | 156.99 | USD | Atlanta | GA | card |
| ORD-2026-0073 | Rosa Delgado | rosa.delgado@example.com | 2026-08-29 11:36:58.158541 | shipped | home_decor | 52.99 | USD | Duluth | MN | apple_pay |
| ORD-2026-0072 | Deshawn Carter | deshawn.carter@example.com | 2026-08-29 11:09:58.158541 | shipped | storage_and_organization | 702.49 | USD | Savannah | GA | card |
| ORD-2026-0071 | Deshawn Carter | deshawn.carter@example.com | 2026-08-28 11:45:58.158541 | shipped | lighting | 1703.09 | USD | Columbus | OH | card |
| ORD-2026-0070 | Yewande Adeyemi | yewande.adeyemi@example.com | 2026-08-28 11:12:58.158541 | shipped | lighting | 779.85 | USD | Atlanta | GA | affirm |
| ORD-2026-0069 | Deshawn Carter | deshawn.carter@example.com | 2026-08-28 11:05:58.158541 | shipped | storage_and_organization | 432 | USD | Savannah | GA | card |
| ORD-2026-0068 | Ingrid Sorensen | ingrid.sorensen@example.com | 2026-08-28 11:00:58.158541 | shipped | home_decor | 158.97 | USD | Portland | OR | card |
| ORD-2026-0067 | Alice Prosser | alice.prosser@example.com | 2026-08-27 11:56:48.178874 | shipped | bedding_and_rugs | 164 | USD | Denver | CO | card |
| ORD-2026-0065 | Priya Raghavan | priya.raghavan@example.com | 2026-08-26 11:31:58.158541 | shipped | bedding_and_rugs | 141.29 | USD | Reading | PA | apple_pay |
| ORD-2026-0066 | Wei Zhang | wei.zhang@example.com | 2026-08-26 11:31:58.158541 | shipped | home_decor | 436.97 | USD | San Antonio | TX | card |
| ORD-2026-0064 | Deshawn Carter | deshawn.carter@example.com | 2026-08-26 11:23:58.158541 | shipped | storage_and_organization | 166.99 | USD | Savannah | GA | paypal |
| ORD-2026-0063 | Marcus Whitfield | marcus.whitfield@example.com | 2026-08-25 11:48:58.158541 | shipped | outdoor_and_patio | 1108.45 | USD | Denver | CO | paypal |
| ORD-2026-0062 | Hannah Kowalski | hannah.kowalski@example.com | 2026-08-25 11:07:58.158541 | shipped | home_decor | 488.69 | USD | San Antonio | TX | card |
| ORD-2026-0061 | Bruno Halvorsen | bruno.halvorsen@example.com | 2026-08-24 11:56:48.178874 | shipped | lighting | 245 | USD | Reno | NV | card |
| ORD-2026-0060 | Rosa Delgado | rosa.delgado@example.com | 2026-08-23 13:54:58.158541 | cancelled | storage_and_organization | 44.55 | USD | Phoenix | AZ | card |
| ORD-2026-0059 | Omar Haddad | omar.haddad@example.com | 2026-08-20 06:47:58.158541 | delivered | storage_and_organization | 623.44 | USD | Dearborn | MI | card |
| ORD-2026-0058 | Ana Lucia Ferreira | ana.luciaferreira@example.com | 2026-08-20 06:11:58.158541 | delivered | storage_and_organization | 307.71 | USD | Austin | TX | gift_card |
| ORD-2026-0057 | Priya Raghavan | priya.raghavan@example.com | 2026-08-20 05:51:58.158541 | delivered | lighting | 311.98 | USD | Eugene | OR | paypal |
| ORD-2026-0056 | Priya Raghavan | priya.raghavan@example.com | 2026-08-20 00:49:58.158541 | delivered | kitchen_and_dining | 207.99 | USD | Reading | PA | card |
| ORD-2026-0055 | Yewande Adeyemi | yewande.adeyemi@example.com | 2026-08-17 05:06:58.158541 | cancelled | lighting | 126.5 | USD | Atlanta | GA | paypal |
| ORD-2026-0054 | Tomas Lindqvist | tomas.lindqvist@example.com | 2026-08-16 19:42:58.158541 | delivered | furniture | 431.85 | USD | Minneapolis | MN | card |
| ORD-2026-0053 | Omar Haddad | omar.haddad@example.com | 2026-08-12 04:57:58.158541 | delivered | lighting | 283.49 | USD | Dearborn | MI | card |
| ORD-2026-0052 | Ingrid Sorensen | ingrid.sorensen@example.com | 2026-08-11 19:03:58.158541 | delivered | storage_and_organization | 772.39 | USD | Portland | OR | card |
| ORD-2026-0051 | Jae-won Park | jae-won.park@example.com | 2026-08-11 14:51:58.158541 | delivered | home_decor | 219.94 | USD | Cleveland | OH | card |
| ORD-2026-0050 | Deshawn Carter | deshawn.carter@example.com | 2026-08-10 05:05:58.158541 | delivered | outdoor_and_patio | 1757.7 | USD | Columbus | OH | card |
| ORD-2026-0049 | Cassandra Pope | cassandra.pope@example.com | 2026-08-09 17:00:58.158541 | delivered | bedding_and_rugs | 444.45 | USD | Charlotte | NC | card |
| ORD-2026-0048 | Alice Prosser | alice.prosser@example.com | 2026-08-09 11:56:48.178874 | delivered | furniture | 812.5 | USD | Denver | CO | card |
| ORD-2026-0047 | Jae-won Park | jae-won.park@example.com | 2026-08-06 16:08:58.158541 | cancelled | outdoor_and_patio | 1159.98 | USD | Seattle | WA | card |
| ORD-2026-0046 | Ingrid Sorensen | ingrid.sorensen@example.com | 2026-08-05 09:36:58.158541 | cancelled | furniture | 231.9 | USD | Pittsburgh | PA | affirm |
| ORD-2026-0045 | Wei Zhang | wei.zhang@example.com | 2026-08-04 06:55:58.158541 | cancelled | lighting | 155.99 | USD | San Jose | CA | paypal |
| ORD-2026-0044 | Tomas Lindqvist | tomas.lindqvist@example.com | 2026-08-03 00:24:58.158541 | delivered | storage_and_organization | 603.43 | USD | Ann Arbor | MI | card |
| ORD-2026-0043 | Cassandra Pope | cassandra.pope@example.com | 2026-08-02 17:58:58.158541 | cancelled | outdoor_and_patio | 1233.98 | USD | Charlotte | NC | google_pay |
| ORD-2026-0042 | Ana Lucia Ferreira | ana.luciaferreira@example.com | 2026-08-02 17:07:58.158541 | delivered | home_decor | 1269.94 | USD | San Antonio | TX | card |
| ORD-2026-0041 | Jae-won Park | jae-won.park@example.com | 2026-07-30 11:26:58.158541 | delivered | outdoor_and_patio | 651 | USD | Asheville | NC | card |
| ORD-2026-0040 | Tomas Lindqvist | tomas.lindqvist@example.com | 2026-07-29 16:53:58.158541 | delivered | kitchen_and_dining | 207.99 | USD | Minneapolis | MN | affirm |
| ORD-2026-0039 | Yewande Adeyemi | yewande.adeyemi@example.com | 2026-07-29 01:27:58.158541 | cancelled | outdoor_and_patio | 1850.97 | USD | Atlanta | GA | google_pay |
| ORD-2026-0038 | Alice Prosser | alice.prosser@example.com | 2026-07-27 11:56:48.178874 | delivered | bedding_and_rugs | 128 | USD | Denver | CO | apple_pay |
| ORD-2026-0037 | Ana Lucia Ferreira | ana.luciaferreira@example.com | 2026-07-27 11:51:58.158541 | delivered | lighting | 253 | USD | Providence | RI | paypal |
| ORD-2026-0036 | Cassandra Pope | cassandra.pope@example.com | 2026-07-26 06:52:58.158541 | cancelled | storage_and_organization | 300.58 | USD | Charlotte | NC | card |
| ORD-2026-0035 | Rosa Delgado | rosa.delgado@example.com | 2026-07-25 13:46:58.158541 | delivered | outdoor_and_patio | 824 | USD | Phoenix | AZ | apple_pay |
| ORD-2026-0034 | Ingrid Sorensen | ingrid.sorensen@example.com | 2026-07-20 21:24:58.158541 | delivered | bedding_and_rugs | 1853.53 | USD | Portland | OR | card |
| ORD-2026-0033 | Marcus Whitfield | marcus.whitfield@example.com | 2026-07-19 16:46:58.158541 | delivered | storage_and_organization | 792.05 | USD | Denver | CO | google_pay |
| ORD-2026-0032 | Wei Zhang | wei.zhang@example.com | 2026-07-16 04:18:58.158541 | delivered | lighting | 253 | USD | San Jose | CA | card |
| ORD-2026-0031 | Dev Raghavan | dev.raghavan@example.com | 2026-07-13 23:13:58.158541 | delivered | furniture | 693 | USD | Reading | PA | card |
| ORD-2026-0030 | Ana Lucia Ferreira | ana.luciaferreira@example.com | 2026-07-12 22:14:58.158541 | delivered | home_decor | 127.45 | USD | Austin | TX | card |
| ORD-2026-0029 | Omar Haddad | omar.haddad@example.com | 2026-07-12 09:03:58.158541 | delivered | home_decor | 58.05 | USD | Dearborn | MI | card |
| ORD-2026-0028 | Omar Haddad | omar.haddad@example.com | 2026-07-11 23:01:58.158541 | cancelled | furniture | 115.95 | USD | Dearborn | MI | apple_pay |
| ORD-2026-0027 | Omar Haddad | omar.haddad@example.com | 2026-07-11 04:03:58.158541 | delivered | bedding_and_rugs | 268 | USD | Dearborn | MI | card |
| ORD-2026-0026 | Wei Zhang | wei.zhang@example.com | 2026-07-11 00:32:58.158541 | delivered | outdoor_and_patio | 616.99 | USD | San Jose | CA | paypal |
| ORD-2026-0025 | Dev Raghavan | dev.raghavan@example.com | 2026-07-10 11:30:58.158541 | delivered | bedding_and_rugs | 2223.99 | USD | Reading | PA | google_pay |
| ORD-2026-0024 | Dev Raghavan | dev.raghavan@example.com | 2026-07-05 21:07:58.158541 | delivered | outdoor_and_patio | 616.99 | USD | Reading | PA | paypal |
| ORD-2026-0023 | Hannah Kowalski | hannah.kowalski@example.com | 2026-07-04 14:50:58.158541 | delivered | furniture | 3108.45 | USD | San Antonio | TX | affirm |
| ORD-2026-0022 | Wei Zhang | wei.zhang@example.com | 2026-07-04 12:24:58.158541 | delivered | storage_and_organization | 1583.35 | USD | San Jose | CA | affirm |
| ORD-2026-0021 | Priya Raghavan | priya.raghavan@example.com | 2026-07-04 10:22:58.158541 | delivered | kitchen_and_dining | 198.99 | USD | Tucson | AZ | card |
| ORD-2026-0020 | Ingrid Sorensen | ingrid.sorensen@example.com | 2026-06-29 19:01:58.158541 | delivered | bedding_and_rugs | 187.94 | USD | Tucson | AZ | card |
| ORD-2026-0019 | Omar Haddad | omar.haddad@example.com | 2026-06-29 16:56:58.158541 | delivered | home_decor | 105.98 | USD | Dearborn | MI | paypal |
| ORD-2026-0018 | Dev Raghavan | dev.raghavan@example.com | 2026-06-29 11:29:58.158541 | delivered | bedding_and_rugs | 134.97 | USD | Reading | PA | google_pay |
| ORD-2026-0017 | Wei Zhang | wei.zhang@example.com | 2026-06-28 16:04:58.158541 | delivered | kitchen_and_dining | 211 | USD | San Jose | CA | card |
| ORD-2026-0016 | Omar Haddad | omar.haddad@example.com | 2026-06-28 00:49:58.158541 | delivered | kitchen_and_dining | 415.98 | USD | Dearborn | MI | paypal |
| ORD-2026-0015 | Rosa Delgado | rosa.delgado@example.com | 2026-06-27 03:36:58.158541 | delivered | outdoor_and_patio | 580.5 | USD | Phoenix | AZ | apple_pay |
| ORD-2026-0014 | Rosa Delgado | rosa.delgado@example.com | 2026-06-24 08:29:58.158541 | delivered | furniture | 536.99 | USD | Duluth | MN | card |
| ORD-2026-0013 | Alice Prosser | alice.prosser@example.com | 2026-06-23 11:56:48.178874 | delivered | home_decor | 76 | USD | Denver | CO | card |
| ORD-2026-0012 | Hannah Kowalski | hannah.kowalski@example.com | 2026-06-19 20:14:58.158541 | delivered | home_decor | 204.49 | USD | San Antonio | TX | affirm |
| ORD-2026-0011 | Hannah Kowalski | hannah.kowalski@example.com | 2026-06-18 00:23:58.158541 | delivered | storage_and_organization | 306.99 | USD | Chicago | IL | card |
| ORD-2026-0010 | Deshawn Carter | deshawn.carter@example.com | 2026-06-17 02:45:58.158541 | delivered | bedding_and_rugs | 249.29 | USD | Columbus | OH | card |
| ORD-2026-0009 | Wei Zhang | wei.zhang@example.com | 2026-06-16 20:47:58.158541 | delivered | outdoor_and_patio | 651 | USD | Evanston | IL | paypal |
| ORD-2026-0008 | Marcus Whitfield | marcus.whitfield@example.com | 2026-06-13 04:05:58.158541 | delivered | outdoor_and_patio | 1244.58 | USD | Denver | CO | card |
| ORD-2026-0007 | Ingrid Sorensen | ingrid.sorensen@example.com | 2026-06-13 03:56:58.158541 | cancelled | furniture | 676.35 | USD | Sacramento | CA | gift_card |
| ORD-2026-0006 | Jae-won Park | jae-won.park@example.com | 2026-06-12 18:40:58.158541 | cancelled | kitchen_and_dining | 351.97 | USD | Seattle | WA | card |
| ORD-2026-0005 | Deshawn Carter | deshawn.carter@example.com | 2026-06-12 17:56:58.158541 | cancelled | lighting | 126.5 | USD | Savannah | GA | card |
| ORD-2026-0004 | Rosa Delgado | rosa.delgado@example.com | 2026-06-12 17:00:58.158541 | delivered | bedding_and_rugs | 370.63 | USD | Duluth | MN | apple_pay |
| ORD-2026-0003 | Rosa Delgado | rosa.delgado@example.com | 2026-06-11 14:27:58.158541 | delivered | storage_and_organization | 739.88 | USD | Phoenix | AZ | paypal |
| ORD-2026-0002 | Hannah Kowalski | hannah.kowalski@example.com | 2026-06-11 05:14:58.158541 | delivered | lighting | 1587.9 | USD | Chicago | IL | card |
| ORD-2026-0001 | Priya Raghavan | priya.raghavan@example.com | 2026-06-10 18:14:58.158541 | delivered | bedding_and_rugs | 1430.08 | USD | Reading | PA | paypal |
| order_number | customer_name | customer_email | placed_at | status | category | total_amount | currency | city | state | payment_method |
|---|---|---|---|---|---|---|---|---|---|---|
| ORD-2026-0066 | Wei Zhang | wei.zhang@example.com | 2026-08-26 11:31:58.158541 | shipped | home_decor | 436.97 | USD | San Antonio | TX | card |
| ORD-2026-0062 | Hannah Kowalski | hannah.kowalski@example.com | 2026-08-25 11:07:58.158541 | shipped | home_decor | 488.69 | USD | San Antonio | TX | card |
| ORD-2026-0058 | Ana Lucia Ferreira | ana.luciaferreira@example.com | 2026-08-20 06:11:58.158541 | delivered | storage_and_organization | 307.71 | USD | Austin | TX | gift_card |
| ORD-2026-0042 | Ana Lucia Ferreira | ana.luciaferreira@example.com | 2026-08-02 17:07:58.158541 | delivered | home_decor | 1269.94 | USD | San Antonio | TX | card |
| ORD-2026-0030 | Ana Lucia Ferreira | ana.luciaferreira@example.com | 2026-07-12 22:14:58.158541 | delivered | home_decor | 127.45 | USD | Austin | TX | card |
| ORD-2026-0023 | Hannah Kowalski | hannah.kowalski@example.com | 2026-07-04 14:50:58.158541 | delivered | furniture | 3108.45 | USD | San Antonio | TX | affirm |
| ORD-2026-0012 | Hannah Kowalski | hannah.kowalski@example.com | 2026-06-19 20:14:58.158541 | delivered | home_decor | 204.49 | USD | San Antonio | TX | affirm |
| ✓ | Expected: Returns the subset of t1's answer, with t1's other conditions still applied.Found: It returned only the Texas rows while keeping the original date window. |
It is a pass because the follow-up kept the first answer's date window and added only Texas, so the Texas rows are a subset of the first result.
Columns kept. Both exports kept all 11 columns.
The narrowing was checked against the 91 rows the product returned, not the full 100-row window.
Configuration and setup
How this tool was set up for the run and what the test needed in place. Each row is a fact from the run's records; a fact the records do not hold is left out, not guessed.
How this scenario is graded
How we decide Pass, Fail and Not gradable. The same rules apply to every tool tested on this scenario.
Each test case gets one result per tool: Pass, Fail or Not gradable. A test case we haven't run yet shows Untested. There are no partial results.
- Pass: the tool did everything the test expected, and nothing it said contradicts the correct answer.
- Fail: at least one expected behaviour clearly didn't happen; the test case's result says which one and quotes the tool.
- Not gradable: our evidence couldn't settle the outcome (for example a record we needed is missing). It is never counted as a fail, and the test case's result says what's missing.
Where this sits in the benchmark
This page is one cell of a larger study: one tool, one scenario. Only this benchmark's frame appears here.
| Level | Name | Scope |
|---|---|---|
| Benchmark | AI Database Agents → | 28 scenarios · 11 tools |
| Capability | Conversational Interaction → | |
| Scenario | The user narrows what they just asked → | |
| Tool | BlazeSQL → |
Global scenario definition → · Global capability definition → · BlazeSQL product page →
History of this result
What has happened to this result since it was first published. Runs and grades are never overwritten: a retest or a re-grade publishes a new result and keeps the earlier one readable.
Act on this result
Nothing filed here edits the run or the grade. A challenge opens a review, and a review can produce a new run or a re-grade — which becomes the current result and leaves this one in the history.
You run the same kind of test against your own setup and get the same behaviour.
Agree →Yours behaves differently. Tell us what you got, with a screenshot if you have one.
Disagree →Something here is wrong — a reference value, a transcription, a grade.
Report an issue →Tell us what changed and we schedule a rerun of the failing test case. The old result stays as history.
Vendor notice →The same record is available as structured data through the AI Demos MCP server, with the counts, the coverage and every per-test-case reason carried as fields.
Verify the proof files
These files support this result. Open a file to inspect the original evidence.
File fingerprints (SHA-256)
A fingerprint identifies the exact file used for this result.