The Citation That Proves Nothing: Why AI Sources Don't Mean AI Accuracy

2026-05-21

When a public AI system answers a question about your organisation and includes a neat row of citations underneath, it feels authoritative. A linked source signals that the answer has been checked, verified, grounded in something real.

It hasn't.

The Confidence Gap

A recent analysis of Google's AI Overviews — the AI-generated summaries that now sit above traditional search results — found something that should unsettle every executive with a stake in how their organisation is perceived.

After Google upgraded its AI Overview engine to Gemini 3, 56 percent of answers deemed "correct" cited sources that did not actually support the information presented. The sources were real. The links worked. But the connection between what the AI claimed and what the source actually said was broken.

This isn't a fringe finding. At Google's scale — over five trillion searches a year — even a 9 percent error rate translates to tens of millions of wrong answers every hour. And more than half the time, even the "right" answers point to sources that don't back them up.

Why This Matters More Than Raw Hallucination

There is a meaningful difference between an AI system that makes something up and one that makes something up while pointing to a credible-looking source.

The first is a known risk. Most sophisticated stakeholders — investors running due diligence, journalists preparing a story, regulators building a case file — understand that public AI systems sometimes fabricate. They discount accordingly.

The second is far harder to catch. When an AI answer arrives with citations attached, the natural instinct is to trust it. The presence of a source short-circuits the verification step. A portfolio manager scanning AI-generated research on a potential acquisition target is unlikely to click through every linked page and cross-reference each claim. A regulatory analyst preparing background on a company before an examination will note the sources, assume they check out, and move on.

The citation becomes armour for the error.

The Compounding Problem

This matters because AI-generated summaries are increasingly the first — and sometimes only — touchpoint between your organisation and the people making decisions about it.

Ninety-four percent of B2B buyers now use generative AI during their purchasing process. Forty-four percent of consumers start their research with an AI-powered search rather than a traditional one. The audience is enormous, and it is growing.

When those audiences see a cited AI response that misrepresents your organisation — wrong revenue figures, outdated leadership information, a fabricated regulatory action, a competitor's product attributed to you — the citation doesn't just make the error more believable. It makes it harder to challenge. Try telling a prospect that the AI-generated summary they read was wrong when it came with three blue hyperlinks underneath. The burden of proof shifts to you.

What Organisations Miss

Most companies that think about AI accuracy at all focus on outright hallucination: the fabricated lawsuit, the invented executive, the completely false claim. These are serious, but they're also relatively easy to spot.

The citation problem is subtler. The AI says something partially true, or true in a different context, or true about a different organisation — and attaches a source that looks relevant but doesn't actually confirm the specific claim. Audits across 50 companies found that 72 percent had at least one factual error in AI-generated responses about them. The most common errors were incorrect pricing and outdated product information — the kind of mistakes that look plausible, come with sources attached, and quietly erode trust.

The Takeaway

The era of "just check the sources" as a defence against AI misinformation is already over. Citations in public AI systems are not verification — they are decoration. They create an illusion of rigour that makes errors about your organisation more durable, more persuasive, and harder to correct.

The question for leadership teams is no longer whether public AI systems are saying accurate things about their organisation. It's whether anyone is checking what those systems say and whether the sources they cite actually support it.

Because right now, for most organisations, nobody is.