The AI Citation Index

Why does this dataset not already exist?

Pick any statistic circulating about AI search and try to trace it. Most of the time you land on a marketing blog citing another marketing blog, and the trail goes cold before it reaches a methodology. The field has an enormous amount of published conclusion and very little published evidence.

That is not because the research is bad — Ahrefs in particular has done the most rigorous public work in this space. The problem is structural: the organisations holding the largest citation datasets are visibility software companies, and a company whose product is proprietary citation data cannot open-source it without dismantling its own business. The incentive to publish a null result is close to zero.

The peer-reviewed exception, the Princeton-led GEO paper presented at KDD in 2024, ran a controlled 10,000-query benchmark and remains the only causal study most of the field can point to. Ahrefs' public re-measurement of its own AI Overview overlap figure — roughly 76% in mid-2025 down to roughly 38% by early 2026 — is the field's best example of a source correcting itself. What never appeared is an independent party whose entire purpose is measurement: a dataset anyone can download, re-run, and disagree with using their own numbers.

What do we already know, and how well?

The current evidence base is thin and unevenly distributed. Source-mix figures exist for ChatGPT and Perplexity, are much weaker for Gemini, and are close to absent for Claude. Every figure this site republishes is graded in the statistics hub.

Three findings shape this design. Each is tagged with how much weight it can bear, and none has been independently replicated.

If ranking and citation were the same problem, all three bars would sit near 100%. They do not: whatever gets a page cited by ChatGPT is substantially not what gets it to rank, and the Google surfaces behave differently from the non-Google ones.

Each engine has a centre of gravity. If that holds, "AI visibility" is not one problem with one answer but at least three problems wearing the same name.

That is the economic backdrop to publisher hostility toward AI crawlers: a search engine taking roughly five pages per visitor it sends is a trade, one taking over two thousand is something else. Blocking a crawler is a visibility decision, which is why the robots.txt blocking census runs alongside this, and why what those crawlers can actually render is a prerequisite question.

What does the Index measure?

One fixed query set, run against seven surfaces, with every citation recorded. Not a score, not a composite index of visibility — a count of what appeared, where, and when.

Each citation record captures:

• URL, domain and domain category (publisher, brand, forum, encyclopedia, documentation, video)
• Position within the answer, and which sub-answer it supported
• The query, the engine, the run number and the timestamp
• The snippet or claim the citation was attached to
• Joined third-party metrics — domain rating, estimated traffic, domain age — recorded at collection time

What does a citation record look like?

A single illustrative record in the shape the Index will publish — invented values, showing structure, not a collected row:

{
  "query_id": "q-0417",
  "query_text": "best budget mirrorless camera 2027",
  "engine": "chatgpt_search",
  "run": 3,
  "timestamp": "2027-01-14T09:32:11Z",
  "citation": {
    "url": "example.com/best-mirrorless-cameras",
    "domain": "example.com",
    "domain_category": "publisher",
    "position": 2,
    "supports_subanswer": "value_pick",
    "snippet": "…the X-T50 remains the strongest sub-$1000 option…"
  },
  "domain_metrics": { "dr": 61, "est_traffic": 84000, "domain_age_years": 7 }
}

Every field is joinable — DR against citation rate, domain age against citation position. That is the point of publishing raw records rather than a summary table: a summary answers the questions the publisher thought to ask, a raw file answers the ones nobody asked yet.

What is in the query set?

A study is only as good as what it asks, so the set is published in full — it is the part most likely to be criticised, which is exactly why it goes out in the open.

It is weighted toward informational and commercial intent, where AI answers displace clicks most visibly; navigational queries are a small slice because engines resolve them trivially. The ten verticals — consumer electronics, personal finance, health and wellness, home and garden, software and SaaS, travel, food and recipes, B2B services, education, local services — carry roughly 100 queries each at v1, enough for a directional per-vertical read only once v2 expands to 5,000.

The set does not change within a release cycle. Queries are added only at a version boundary, with the additions logged, so quarter-on-quarter figures stay comparable. Silently swapping queries between releases would make every trend line meaningless.

How is the data collected?

The whole point is that you can check this: the query set, the raw records and the collection code ship with every release.

Why 72 hours: a single instant risks catching a blip — a model rollback, a brief outage — and reporting it as the quarter's baseline, while a full month reintroduces the drift the window exists to control.

Why five runs, not one?

Variance is why single-run studies mislead. The same question can return a different source list minutes later, which is also the mechanism behind citation half-life and part of why query fan-out is hard to observe from outside.

Language models are not deterministic: ask the same question twice and you may get a different answer built from different sources. Almost every AI-visibility figure in circulation — including the ones charted above — comes from a single run per query. If run-to-run source disagreement is meaningful, some proportion of every published citation statistic is measurement error rather than signal, and nobody currently knows what that proportion is. Quantifying it is hypothesis H2.

Five runs does not eliminate the problem. It makes the problem visible. Every figure the Index publishes carries the spread across runs alongside the central estimate, so a reader can see whether a difference between two engines is real or within noise.

Which metrics are reported?

Each of these exists because there is a real measurement problem with no current answer.

One metric is deliberately not on that list: a composite "AI Visibility Score." Every visibility vendor has one, they are unfalsifiable, and building one here would undercut the only thing this project has going for it.

How is each metric calculated?

Each formula, then a worked example on invented numbers for a hypothetical domain, "acmegear.com."

Same hypothetical: 340 citations out of 9,800 recorded that quarter gives a Share of Voice of roughly 3.5%, which accounts for how crowded the citation set was. If logs show an AI bot crawled 12,000 of its pages, Citation Efficiency is (340 ÷ 12,000) × 1,000 ≈ 28.3 citations per 1,000 pages crawled — a figure that only becomes interesting next to a competitor's.

Pre-registered hypotheses

These are posted now, before collection, with the direction predicted and the analysis specified. If the data contradicts them, that gets published as the result.

H5 and H6 are the interesting ones, because the prediction is that they fail. Registering a hypothesis you expect to reject is the cheapest available protection against reading a pattern into noise after the fact — and if H6 does hold, that is a genuinely surprising result worth more than a confirmation.

Limitations: what it will not prove

It will not establish causation, and it will not produce a single visibility score. Correlational work on mentions versus backlinks and the registered schema experiment exist precisely because observation alone cannot answer those questions.

The Index is an observational instrument, and observational data has hard limits that are easy to forget once a chart looks convincing.

• It cannot establish causation. If highly-cited domains share a trait, that does not mean the trait produced the citations. Causal claims need randomised intervention, which is a separate programme.
• It measures the engines, not your site. Citation rates on a fixed public query set describe the ecosystem. They do not tell an individual site owner what will happen to them.
• It is a snapshot of moving targets. These products change without notice. A finding is true of the surface as it behaved during a stated 72-hour window, and every figure is published with that window attached.
• Geography and language are fixed. Collection runs English-language queries from a single stated region. Multilingual and multi-region collection is planned but not funded, and no figure should be generalised beyond the language and region tested.
• Engine coverage stops at seven answer surfaces. Agentic browsers, MCP-mediated retrieval and in-app assistants are excluded because they do not emit a comparable citation list, so the Index describes AI answer citation, not every route an AI system takes to a page.
• Reproduction is only as good as the products allow. A third party re-running the published code in a later week is measuring a different surface state. Agreement on direction and magnitude is the realistic bar, not exact replication of a number.
• Sampling bias is real. A 1,000-query set is a choice, and a different set would produce different numbers. That is why the set is published rather than described.
• Residual personalisation may survive the controls. If an engine varies results by IP geography or device fingerprint even in a logged-out state, the recorded citation set may not match what every real user sees.

How can you check the work?

The collection code is intended for open release, described in open-source SEO agent tooling, and every planned file is listed in the dataset strategy. Results that contradict a prediction go to the null results registry.

An open study nobody audits is a closed study with extra steps. Three things ship with every release so the findings can be attacked. Any error found gets a dated changelog entry rather than a silent edit, credited to whoever found it.

When does each release land?

Quarterly full releases, monthly pulse updates in between. Each release is versioned and permanent: older versions stay online rather than being overwritten. If a release slips, the delay and the reason get posted here; if the project stops, that gets posted too.

Get the first release, or contribute to it

There is no data yet, so the honest thing to offer is notice when there is: the newsletter carries each release and its raw files the day they publish. Two contributions would materially improve the work. Queries from your vertical that belong in the set, which is public and credits contributors. And anonymised server logs from a site you run, which unlock crawl-side measurements query data alone cannot reach. The about page explains how to get in touch.

Quick glossary

For every other term used across this site's research — GEO, AEO, query fan-out, crawl-to-referral ratio and more — see the full AI search glossary.