AI Demos Research — the structured-intelligence platform. Every verdict on these pages opens to the execution behind it.
Graded 14 September 2026

Can FutureSmart Document Intelligence answer when the answer is available in the knowledge base?

FutureSmart Document Intelligence surfaced the supporting passage for the knowledge-base answer. The returned text matched the passage saying a Priority 1 fault reported outside business hours is acknowledged by a named duty engineer within 90 minutes, and the UI showed A01 first in Sources. Only four of eight source passages were visible in one capture, while the full notebook response was available in the attached text artifact.

1 of 1 test case passed

Every test case in this scenario has a result.

Pass rate100%1 of 1 gradable
Coverage1 of 1test case with a result
1PassDid everything it was expected to do.
0FailNo test case failed.
0Not gradableEvery result here could be graded.
0UntestedEvery test case in this scenario has a result.
The pass rate is a summary. The evidence is the test case below: what we sent, what we checked, what the tool returned, and the proof.

The test case

Each test case is judged on its own: Pass, Fail, Not gradable, or Untested. The scenario result above counts this row.

Naturally phrased question the corpus answersPassEvidence
What we sent

“How quickly does Fenwake acknowledge a Priority 1 fault reported outside business hours?”

What the tool returned

A Priority 1 fault reported outside business hours is acknowledged by a named duty engineer within 90 minutes of the ticket being raised.

Transcribed from Proof 1 · Screenshot

Output file (text)
Search output showing three returned sections, including the 90-minute acknowledgement passage.
Text196 lines
TC90 · Retrieval — corpus holds the answer
Query: How quickly does Fenwake acknowledge a Priority 1 fault reported outside business hours?

--- API call ---
POST /collections/{CID}/search
Body: {"query": "How quickly does Fenwake acknowledge a Priority 1 fault reported outside business hours?"}
(default params — no top_k, no filters)

--- Raw retrieved response ---
status: 200 | sections returned: 3

score=1.0000  document_id=88399604-1153-40ef-be66-f4da43a873a2
Where a site requires a permit, an induction or a vessel movement, the attendance clock is suspended
until access is confirmed in writing. The suspension is recorded on the ticket with the reason.

Fenwake internal - service operations
Page 3

---



Fenwake Instruments Ltd - Field Service Policy and Charges (United Kingdom)

# 6. **Out-of-hours cover**

Outside business hours the service desk is not staffed. A duty engineer rota covers every hour of
the year, and the out-of-hours number in section 8 pages the engineer on call.

## 6.1 **Acknowledgement out of hours**

A Priority 1 fault reported outside business hours is acknowledged by a named duty engineer within
90 minutes of the ticket being raised. The engineer confirms ownership by telephone and, where the
customer asks for it, in writing on the ticket. The target applies on every cover level; the cover
level determines what happens after acknowledgement, not how quickly the call is answered.

## 6.2 **What follows acknowledgement**

Having acknowledged, the duty engineer attempts a remote diagnosis using the telemetry record. Where
the fault cannot be cleared remotely, attendance is arranged under section 5 from the start of the
following working day, except on Critical cover, where the duty engineer may authorise attendance
immediately.

## 6.3 **Priority 2 and Priority 3 out of hours**

Priority 2 and Priority 3 tickets raised outside business hours are queued and acknowledged when the
service desk opens. The business-hours targets in section 4 then apply from opening.

Fenwake internal - service operations
Page 4

---



Fenwake Instruments Ltd - Field Service Policy and Charges (United Kingdom)

# 7. **Escalation**

A customer may escalate at any time, and the service desk escalates on its own initiative when a
target is missed. Each step below is a named person, not a queue.

score=0.5833  document_id=88399604-1153-40ef-be66-f4da43a873a2
File Name: A01_field_service_policy_and_charges_uk.pdf

Extracted Text:

Fenwake Instruments Ltd

# Field Service Policy and Charges

United Kingdom operations

## 1. Purpose

This document states what a customer holding a support contract with Fenwake Instruments Ltd may
expect when an instrument fails in service, what the service desk in Bristol is obliged to do in
response, and what is charged for work that falls outside cover. It applies to the Tolvan A3, the
Tolvan C5 and the Brannoc telemetry gateway, on every cover level.

## 2. Scope

Part A covers fault handling: acknowledgement, attendance, spares and escalation. Part B sets out
the charges that apply outside cover. Neither part describes the goods warranty carried by a new
instrument.

## 3. Fault priorities

Every ticket is given a priority when it is raised. The priority is set by the effect on the
customer's operation, not by which component failed.

- **Priority 1** - a deployed instrument is not returning data, or is returning data known to be
invalid, and no spare is rigged.
- **Priority 2** - an instrument is degraded but usable, or a spare has been rigged and the primary
unit is down.
- **Priority 3** - a question, a configuration change, or a fault with no operational effect.

A priority may be raised by the duty engineer at any point. It is lowered only with the customer's
agreement, recorded on the ticket.

Fenwake internal - service operations
Page 1

---



Fenwake Instruments Ltd - Field Service Policy and Charges (United Kingdom)

# 4. Acknowledgement during business hours

Business hours are 08:00 to 18:00, Monday to Friday, excluding statutory non-working days in England
and Wales. Within those hours the service desk is staffed continuously and the targets below are
measured from the moment the ticket is raised, by telephone or through the portal, whichever is
earlier.

score=0.5333  document_id=88399604-1153-40ef-be66-f4da43a873a2
<table>
  <tr>
    <th>Priority</th>
    <th>Acknowledgement target</th>
    <th>Measured from</th>
  </tr>
<tr>
    <td>Priority 1</td>
<td>30 minutes</td>
<td>Ticket raised</td>
  </tr>
<tr>
    <td>Priority 2</td>
<td>4 working hours</td>
<td>Ticket raised</td>
  </tr>
<tr>
    <td>Priority 3</td>
<td>The next working day</td>
<td>Ticket raised</td>
  </tr>
</table>

An acknowledgement is a named engineer taking ownership of the ticket and telling the customer what
happens next. An automatic reply from the ticketing system is not an acknowledgement and does not
stop the clock.

## 4.1 What the target does not cover

The acknowledgement target is not a repair target. Attendance and parts are dealt with in section 5,
and the two clocks run independently: a ticket may be acknowledged inside target and still take
several days to close.

Fenwake internal - service operations
Page 2

---



Fenwake Instruments Ltd - Field Service Policy and Charges (United Kingdom)

# 5. Attendance and parts

Where a fault cannot be cleared remotely, the service desk arranges attendance. Attendance targets
are working-day targets and are quoted from the point at which the customer confirms site access.

<table>
  <tr>
    <th>Cover level</th>
    <th>Attendance target</th>
    <th>Spares held</th>
  </tr>
<tr>
    <td>Standard</td>
<td>Five working days</td>
<td>Central store only</td>
  </tr>
<tr>
    <td>Extended</td>
<td>Three working days</td>
<td>Central store only</td>
  </tr>
<tr>
    <td>Critical</td>
<td>Next working day</td>
<td>Regional depot</td>
  </tr>
</table>

## 5.1 Consumables

Anti-fouling guards, sacrificial anodes and cable glands are consumables. They are replaced during
any attendance and charged at the rates in Part B.

## 5.2 Access

Where a site requires a permit, an induction or a vessel movement, the attendance clock is suspended
until access is confirmed in writing. The suspension is recorded on the ticket with the reason.
Open the text file ↗
Expected vs. Found
What this scenario evaluates

These are scenario-level criteria. Each test case's Expected and Found are listed separately.

  • Whether the system surfaces the answer when the relevant source already contains it.
  • Whether the output is grounded in the source content rather than invented.
  • Whether the system handles a present answer as a successful source-based result.
How this scenario is judged →
✓Found: The supporting passage was returned and ranked first.
Supporting proof
Proof 1Screenshot
Conversations view showing the 90-minute answer and A01 first in Sources.
Open original ↗
Proof 2Screenshot
Notebook cell showing the search request with default parameters and truncated output.
Open original ↗
Why this result

The returned sections included the supporting passage, and the UI showed the same answer with A01 first in Sources. That is the supporting-passage behavior the test looks for.

Tested by Vivek Tomar · evidence dated 7 September 2026

Configuration and setup

How this tool was set up for the run and what the test needed in place. Each row is a fact from the run's records; a fact the records do not hold is left out, not guessed.

Software that produced the output
FutureSmart Document Intelligence
Surface
Conversations view
Set up before the run
The knowledge base held one document that answered the naturally phrased question, and no other document covered it, so retrieval was unambiguous.
search params
default params - no top_k, no filters
Tested
By 7 September 2026 · Vivek Tomar

How this scenario is graded

How we decide Pass, Fail and Not gradable. The same rules apply to every tool tested on this scenario.

How results are decided

Each test case gets one result per tool: Pass, Fail or Not gradable. A test case we haven't run yet shows Untested. There are no partial results.

The rules
  • Pass: the tool did everything the test expected, and nothing it said contradicts the correct answer.
  • Fail: at least one expected behaviour clearly didn't happen; the row says which and quotes the tool.
  • Not gradable: our evidence couldn't settle the outcome (for example a record we needed is missing). It is never counted as a fail, and the row says what's missing.

Where this sits in the benchmark

This page is one cell of a larger study: one tool, one scenario. Only this benchmark's frame appears here.

History of this result

What has happened to this result since it was first published. Runs and grades are never overwritten: a retest or a re-grade publishes a new result and keeps the earlier one readable.

from the publication record
23 October 2026First publishedManaged RAG v1

Act on this result

Nothing filed here edits the run or the grade. A challenge opens a review, and a review can produce a new run or a re-grade — which becomes the current result and leaves this one in the history.

This matches what I see

You run the same kind of test against your own setup and get the same behaviour.

Agree →
This does not match

Yours behaves differently. Tell us what you got, with a screenshot if you have one.

Disagree →
Point out an issue

Something here is wrong — a reference value, a transcription, a grade.

Report an issue →
Request a retest

On a newer build, a larger dataset, or your own setup.

Request a retest →
We have fixed this

Tell us what changed and we schedule a rerun of the failing test case. The old result stays as history.

Vendor notice →
Filed against this evidenceNothing yet. Challenges, counter-evidence and fix notices appear here with their outcome, and stay on the page after they are resolved.
Cite this result
aidemos.com/benchmarks/managed-rag/results/futuresmart-document-intelligence/answer-is-available-in-the-knowledge-base · 1 pass · 0 fail · coverage 1/1 · graded 2026-09-14

The same record is available as structured data through the AI Demos MCP server, with the counts, the coverage and every per-test-case reason carried as fields.

Verify the proof files

These files support this result. Open a file to inspect the original evidence.

Output file (text) ↗text/plain · 6 KB
Proof 1 · Screenshot ↗image/png · 87 KB
Proof 2 · Screenshot ↗image/png · 366 KB
File fingerprints (SHA-256)

A fingerprint identifies the exact file used for this result.

Output file (text)5f164bccff698b76204ba1ef204062758fc1dc2fa8ea7b231060e45ccb48e10e
Proof 1 · Screenshot60acac13958079e6293bf98d0001107162f409eff05731a197443d5c0146b357
Proof 2 · Screenshot8cea418464d5c81754ae57f8c4a9469216812b035a3b1fc0714b6b4977c5a4f0