AI Demos Research — the structured-intelligence platform. Every verdict on these pages opens to the execution behind it.
Graded 16 November 2026

Can Extend AI add one member and remove another in a repeated set?

Extend AI let a user add and remove invoice items through Human Review. After approval, the traffic cones were present, the site fencing was absent and the telehandler and generator values were unchanged.

1 of 1 test case passed

Every test case in this scenario has a result.

Pass rate100%1 of 1 gradable
Coverage1 of 1test case with a result
1PassDid everything it was expected to do.
0FailNo test case failed.
0Not gradableEvery result here could be graded.
0UntestedEvery test case in this scenario has a result.
The pass rate is a summary. The evidence is the test case below: what we sent, what we checked, what the tool returned, and the proof.

The test case

Each test case is judged on its own: Pass, Fail, Not gradable, or Untested. The scenario result above is based on this test case.

Add one member and remove onePassEvidence
What we sent

In Human Review, we removed the site fencing and added traffic cones, then saved the corrections and approved the result.

InputDocument (PDF)
Input PDF: the two-page hire invoice before corrections
Open the PDF ↗
What the tool returned
JSON
{
  "line_items": [
    {
      "amount": 850,
      "quantity": 1,
      "unit_price": 850,
      "description": "Telehandler, 4-tonne, weekly hire"
    },
    {
      "amount": 420,
      "quantity": 2,
      "unit_price": 210,
      "description": "Generator, 20kVA, weekly hire"
    },
    {
      "amount": 45,
      "quantity": 3,
      "unit_price": 15,
      "description": "Traffic cones, set of 20, weekly hire"
    }
  ]
}

Reviewed JSON: the saved set with traffic cones and without site fencing

Copied from Output file (JSON), lines 1–22

Open the JSON file ↗
Expected vs. Found
✓Expected: The added member must be saved, the removed member absent, and every other member unchanged.Found: The saved set contained the required 3 traffic-cone sets (£45), no site fencing, and unchanged entries for 1 telehandler (£850) and 2 generators (£420).
Supporting proof
Proof 1Captured file (JSON)
Comparison JSON: the original extraction before Human Review
Cited values3 values from this file
line_items 1
{ "amount": 850, "quantity": 1, "unit_price": 850, "description": "Telehandler, 4-tonne, weekly hire" }
line_items 2
{ "amount": 420, "quantity": 2, "unit_price": 210, "description": "Generator, 20kVA, weekly hire" }
line_items 3
{ "amount": 192, "quantity": 8, "unit_price": 24, "description": "Site fencing, per 10m panel, weekly" }
Show the whole file (1 KB)
JSON
{
  "line_items": [
    {
      "amount": 850,
      "quantity": 1,
      "unit_price": 850,
      "description": "Telehandler, 4-tonne, weekly hire"
    },
    {
      "amount": 420,
      "quantity": 2,
      "unit_price": 210,
      "description": "Generator, 20kVA, weekly hire"
    },
    {
      "amount": 192,
      "quantity": 8,
      "unit_price": 24,
      "description": "Site fencing, per 10m panel, weekly"
    }
  ]
}
Open the JSON file ↗
Proof 2Screenshot
Review screen: the invoice and extracted items before changes
Open original ↗
Proof 3Screenshot
Confirmation dialog: the traffic-cones row to be saved
Open original ↗
Proof 4Screenshot
Comparison screen: the original extraction beside the reviewed set
Open original ↗
Proof 5Screenshot
Result screen: the corrected set after approval
Open original ↗
Proof 6Screenshot
Playground screen: the separate extraction view without row-addition or row-deletion controls
Open original ↗
Cited values4 values from this file
line_items 1
{ "amount": 850, "quantity": 1, "unit_price": 850, "description": "Telehandler, 4-tonne, weekly hire" }
line_items 2
{ "amount": 420, "quantity": 2, "unit_price": 210, "description": "Generator, 20kVA, weekly hire" }
line_items 3
{ "amount": 45, "quantity": 3, "unit_price": 15, "description": "Traffic cones, set of 20, weekly hire" }
line_items
[ { "amount": 850, "quantity": 1, "unit_price": 850, "description": "Telehandler, 4-tonne, weekly hire" }, { "amount": 420, "quantity": 2, "unit_price": 210, "description": "Generator, 20kVA, weekly hire" }, { "amount": 45, "quantity": 3, "unit_price": 15, "description": "Traffic cones, set of 20, weekly hire" } ]
Why this result

Human Review, rather than the playground, is the product's correction screen. It saved both membership changes without altering the other items, satisfying both required operations; the result shown is the reviewed set within Extend AI, not the downstream webhook payload.

Also observed in this test case

Playground editing limits. The Extract playground's Form view offered no way to add or delete rows, including when a row was under the cursor. These edits were available in Human Review.

Tested by Rugved Nichite · evidence dated 25 September 2026

Configuration and setup

How this tool was set up for the run and what the test needed in place. Each row is a fact from the run's records; a fact the records do not hold is left out, not guessed.

Software that produced the output
Extend AI
Build or version
Not exposed by the hosted product
Surface
Web app
Set up before the run
The hire invoice's extracted line items and schema had to be available in Human Review. The task was to add traffic cones, remove site fencing, then read back the saved set.
Workflow
Trigger → Parse → Extract a list of line items with description, quantity, unit price and amount → Human Review → Webhook Response
Tested
25 September 2026 · Rugved Nichite

How this scenario is graded

How we decide Pass, Fail and Not gradable. The same rules apply to every tool tested on this scenario.

How results are decided

Each test case gets one result per tool: Pass, Fail or Not gradable. A test case we haven't run yet shows Untested. There are no partial results.

The rules
  • Pass: the tool did everything the test expected, and nothing it said contradicts the correct answer.
  • Fail: at least one expected behaviour clearly didn't happen; the test case's result says which one and quotes the tool.
  • Not gradable: our evidence couldn't settle the outcome (for example a record we needed is missing). It is never counted as a fail, and the test case's result says what's missing.

Where this sits in the benchmark

This page is one cell of a larger study: one tool, one scenario. Only this benchmark's frame appears here.

LevelNameScope
BenchmarkStructured Document Extraction →19 scenarios · 10 tools
CapabilityReview and Correction →
ScenarioRepair a record →
ToolExtend AI →

History of this result

What has happened to this result since it was first published. Runs and grades are never overwritten: a retest or a re-grade publishes a new result and keeps the earlier one readable.

from the publication record
17 November 2026First publishedStructured Document Extraction v1

Act on this result

Nothing filed here edits the run or the grade. A challenge opens a review, and a review can produce a new run or a re-grade — which becomes the current result and leaves this one in the history.

This matches what I see

You run the same kind of test against your own setup and get the same behaviour.

Agree →
This does not match

Yours behaves differently. Tell us what you got, with a screenshot if you have one.

Disagree →
Point out an issue

Something here is wrong — a reference value, a transcription, a grade.

Report an issue →
Request a retest

On a newer build, a larger dataset, or your own setup.

Request a retest →
We have fixed this

Tell us what changed and we schedule a rerun of the failing test case. The old result stays as history.

Vendor notice →
Filed against this evidenceNothing yet. Challenges, counter-evidence and fix notices appear here with their outcome, and stay on the page after they are resolved.
Cite this result
aidemos.com/benchmarks/structured-document-extraction/results/extend-ai/repair-a-record · 1 pass · 0 fail · coverage 1/1 · graded 2026-11-16

The same record is available as structured data through the AI Demos MCP server, with the counts, the coverage and every per-test-case reason carried as fields.

Verify the proof files

These files support this result. Open a file to inspect the original evidence.

Input · Document (PDF) ↗application/pdf · 78 KB
Output file (JSON) ↗application/json · 0 KB
Proof 1 · Captured file (JSON) ↗application/json · 0 KB
Proof 2 · Screenshot ↗image/png · 152 KB
Proof 3 · Screenshot ↗image/png · 187 KB
Proof 4 · Screenshot ↗image/png · 176 KB
Proof 5 · Screenshot ↗image/png · 152 KB
Proof 6 · Screenshot ↗image/png · 45 KB
File fingerprints (SHA-256)

A fingerprint identifies the exact file used for this result.

Input · Document (PDF)fdcf03ac5ee53b083da58166f17fd41e7c72de79a9f3a3f6b67daa1360ddbca9
Output file (JSON)88df9a44ca798482e1d04a6169680cf32a848592ce24917f342afd7186766df6
Proof 1 · Captured file (JSON)888a8efa20fe724527104d1074aed87738a8ba5c6256b85622a4bc99db6159c4
Proof 2 · Screenshot0c72b1251fb9a95e47eb5201bc1bf615753757cb965dd17c7d7ce73c8d1450a8
Proof 3 · Screenshot5c30ba73e25adff7b499d4a5070a18d133baa946fbe55d34a5c962dea2026aae
Proof 4 · Screenshot78ad5f8c87ddb9eca71fb3f3f87b371f61665281392c4f5c10c919bd3ed6cdfd
Proof 5 · Screenshot225b7b60dbe161ceb3eb5917fcef83e00c1c30cd1305299f3024999d016fb5d9
Proof 6 · Screenshot050dd00b6b4d2679732167d81f55b374cedd64dd84002f42f54417ce136ee757