AI Demos Research — the structured-intelligence platform. Every verdict on these pages opens to the execution behind it.
Graded 30 October 2026

Can Exa search only the allowed websites?

Exa met the site-only request. It returned docs.github.com results, including the Copilot page at rank 2, and no GitHub blog result; a rank-5 preview-policy page mentions other dates but not the Copilot launch year.

1 of 1 test case passed

Every test case in this scenario has a result.

Pass rate100%1 of 1 gradable
Coverage1 of 1test case with a result
1PassDid everything it was expected to do.
0FailNo test case failed.
0Not gradableEvery result here could be graded.
0UntestedEvery test case in this scenario has a result.
The pass rate is a summary. The evidence is the test case below: what we sent, what we checked, what the tool returned, and the proof.

The test case

Each test case is judged on its own: Pass, Fail, Not gradable, or Untested. The scenario result above is based on this test case.

Search only these websitesPassEvidence
What we sent

Input · Search limited to docs.github.com (includeDomains filter): “In what year did GitHub first launch a technical preview of GitHub Copilot?”

What the tool returned
JSON
{
  "requestId": "abbbd42db87964c784779f2fb3ca271f",
  "resolvedSearchType": "",
  "results": [
    {
      "id": "https://docs.github.com/en/get-started/using-github/exploring-early-access-releases-with-feature-preview",
      "title": "",
      "url": "https://docs.github.com/en/get-started/using-github/exploring-early-access-releases-with-feature-preview",
      "text": "# Exploring early access releases with feature preview\n\nYou can use feature preview to see products or features that are available in public preview and to enable or disable each feature for your personal account.\n\n## GitHub's release cycle … [1,827 characters in the file]"
    },
    {
      "id": "https://docs.github.com/en/copilot/get-started/what-is-github-copilot",
      "title": "",
      "url": "https://docs.github.com/en/copilot/get-started/what-is-github-copilot",
      "text": "# What is GitHub Copilot?\n\nLearn what Copilot is and what you can do with it.\n\nGitHub Copilot is an AI coding assistant that helps you write code faster and with less effort. Then, you can focus more energy on problem solving and collaborat … [3,388 characters in the file]"
    },
    {
      "id": "https://docs.github.com/en/copilot/concepts/learning-about-new-features-and-models",
      "title": "",
      "url": "https://docs.github.com/en/copilot/concepts/learning-about-new-features-and-models",
      "text": "# Learning about new features and models\n\nStay informed about GitHub Copilot features and models, and make confident decisions about enabling them for your enterprise.\n\nGitHub Copilot is evolving rapidly, with new features and models releas … [5,603 characters in the file]"
    },
    {
      "id": "https://docs.github.com/en/copilot/concepts/preparing-for-new-features-and-models",
      "title": "",
      "url": "https://docs.github.com/en/copilot/concepts/preparing-for-new-features-and-models",
      "text": "# Learning about new features and models\n\nStay informed about GitHub Copilot features and models, and make confident decisions about enabling them for your enterprise.\n\nGitHub Copilot is evolving rapidly, with new features and models releas … [6,256 characters in the file]"
    },
    {
      "id": "https://docs.github.com/en/site-policy/github-terms/github-dpa-previews",
      "title": "GitHub DPA-Covered Previews - GitHub Docs",
      "url": "https://docs.github.com/en/site-policy/github-terms/github-dpa-previews",
      "publishedDate": "2025-10-27T00:00:00.000Z",
      "text": "# GitHub DPA-Covered Previews\n\n## GitHub DPA-Covered Previews\n\nThe pre-release products and features listed below use data handling that is the same as when the software becomes generally available. Under GitHub’s Pre-Release License Terms, … [780 characters in the file]"
    },
    {
      "id": "https://docs.github.com/en/copilot",
      "title": "GitHub Copilot documentation - GitHub Docs",
      "url": "https://docs.github.com/en/copilot",
      "text": "# GitHub Copilot documentation\n\nYou can use GitHub Copilot to enhance your productivity and assist as you work on code.\n\n## Links\n\n### Getting started\n\n- What is GitHub Copilot? Learn what Copilot is and what you can do with it.\n- Quickstar … [65,855 characters in the file]"
    },
    {
      "id": "https://docs.github.com/en/copilot/get-started",
      "title": "Get started with GitHub Copilot",
      "url": "https://docs.github.com/en/copilot/get-started",
      "text": "# Get started with GitHub Copilot\n\nLearn how to sign up for and use GitHub Copilot.\n\n## Links\n\n* Quickstart for GitHub Copilot\n\n Quickly learn how to use GitHub Copilot.\n\n* What is GitHub Copilot?\n\n Learn what Copilot is and what you can do … [649 characters in the file]"
    },
    {
      "id": "https://docs.github.com/en/copilot/get-started/features",
      "title": "",
      "url": "https://docs.github.com/en/copilot/get-started/features",
      "text": "# GitHub Copilot features\n\nGitHub Copilot offers a suite of features for users and administrators.\n\n## Assistive features\n\nThese tools are used synchronously, providing advice or suggestions as people work on a task.\n\n### Copilot Chat\n\nA ch … [6,468 characters in the file]"
    },
    {
      "id": "https://docs.github.com/en/copilot/concepts",
      "title": "Concepts for GitHub Copilot",
      "url": "https://docs.github.com/en/copilot/concepts",
      "text": "# Concepts for GitHub Copilot\n\nLearn the core concepts that you'll need to understand GitHub Copilot.\n\n## Links\n\n* Completions for GitHub Copilot\n\n Learn how Copilot can provide inline suggestions.\n\n* About GitHub Copilot Chat\n\n Learn how y … [2,393 characters in the file]"
    },
    {
      "id": "https://docs.github.com/en/copilot/concepts/agents/copilot-cli/about-copilot-cli",
      "title": "About GitHub Copilot CLI",
      "url": "https://docs.github.com/en/copilot/concepts/agents/copilot-cli/about-copilot-cli",
      "text": "# About GitHub Copilot CLI\n\nFind out about using Copilot from the command line.\n\n## Introduction\n\nThe command-line interface (CLI) for GitHub Copilot allows you to use Copilot directly from your terminal. You can use it to answer questions, … [22,158 characters in the file]",
      "image": "https://docs.github.com/assets/images/help/copilot/copilot-cli-welcome.png"
    }
  ],
  "searchTime": 4581.9,
  "costDollars": {
    "total": 0.007,
    "search": {
      "neural": 0.007
    }
  }
}

Search response: permitted documentation pages, including the Copilot introduction at rank 2

Copied from Output file (JSON), raw_response · an excerpt

Open the whole JSON file ↗
Expected vs. Found
✓Expected: Exclude GitHub's blog announcement and return the relevant Copilot introduction page from the permitted GitHub documentation site.Found: All ten results came from GitHub Docs, including the required Copilot introduction at rank 2. GitHub's blog announcement did not appear.
Supporting proof
Cited values2 values from this file
answer_text
null
status
ok
Why this result

Exa retained the required relevant page while excluding the answer-bearing blog, satisfying both parts of the restriction. The result covers search with Exa's native domain filter, not a generated answer.

Tested by Aman · evidence dated 23 September 2026

Configuration and setup

How this tool was set up for the run and what the test needed in place. Each row is a fact from the run's records; a fact the records do not hold is left out, not guessed.

Software that produced the output
Exa
Build or version
Not exposed by the hosted product
Surface
API
Set up before the run
Allowed site: docs.github.com, which has Copilot pages but no launch year. The answer, 2021, is on GitHub's blog announcement. Both pages were checked on 7 September 2026 and re-checked the morning of the run.
Search
docs.github.com only · native includeDomains parameter
Tested
23 September 2026 · Aman

How this scenario is graded

How we decide Pass, Fail and Not gradable. The same rules apply to every tool tested on this scenario.

How results are decided

Each test case gets one result per tool: Pass, Fail or Not gradable. A test case we haven't run yet shows Untested. There are no partial results.

The rules
  • Pass: the tool did everything the test expected, and nothing it said contradicts the correct answer.
  • Fail: at least one expected behaviour clearly didn't happen; the test case's result says which one and quotes the tool.
  • Not gradable: our evidence couldn't settle the outcome (for example a record we needed is missing). It is never counted as a fail, and the test case's result says what's missing.

Where this sits in the benchmark

This page is one cell of a larger study: one tool, one scenario. Only this benchmark's frame appears here.

LevelNameScope
BenchmarkWeb Search for AI →10 scenarios · 12 tools
CapabilityWeb Retrieval →
ScenarioSearch only these websites →
ToolExa →

History of this result

What has happened to this result since it was first published. Runs and grades are never overwritten: a retest or a re-grade publishes a new result and keeps the earlier one readable.

from the publication record
16 November 2026First publishedWeb Search for AI v1

Act on this result

Nothing filed here edits the run or the grade. A challenge opens a review, and a review can produce a new run or a re-grade — which becomes the current result and leaves this one in the history.

This matches what I see

You run the same kind of test against your own setup and get the same behaviour.

Agree →
This does not match

Yours behaves differently. Tell us what you got, with a screenshot if you have one.

Disagree →
Point out an issue

Something here is wrong — a reference value, a transcription, a grade.

Report an issue →
Request a retest

On a newer build, a larger dataset, or your own setup.

Request a retest →
We have fixed this

Tell us what changed and we schedule a rerun of the failing test case. The old result stays as history.

Vendor notice →
Filed against this evidenceNothing yet. Challenges, counter-evidence and fix notices appear here with their outcome, and stay on the page after they are resolved.
Cite this result
aidemos.com/benchmarks/web-search-for-ai/results/exa/allowlisted-web-search · 1 pass · 0 fail · coverage 1/1 · graded 2026-10-30

The same record is available as structured data through the AI Demos MCP server, with the counts, the coverage and every per-test-case reason carried as fields.

Verify the proof files

These files support this result. Open a file to inspect the original evidence.

Output file (JSON) ↗application/json · 235 KB
File fingerprints (SHA-256)

A fingerprint identifies the exact file used for this result.

Output file (JSON)ff079cff8eab9eba8fa1bdcde282397c4152f922fa1046b49d64fe34ddfd3ae7