Why AI Hallucinates - and How to Reduce It

What an AI Hallucination Actually Is

A hallucination is an output that sounds fluent, plausible and confident but is factually wrong or entirely fabricated. The model is not 'lying' in the human sense - it has no intent and no awareness that something is false. The term is a metaphor: the system generates content with no counterpart in its training data or in reality.

• Fabricated facts: names, dates, figures or events that never existed.
• False sources: non-existent studies, rulings or URLs with real-looking titles.
• Conflation: correct fragments combined into a wrong overall statement.

The Mechanism: Prediction, Not Knowledge

A language model stores no database of facts. It learns from vast amounts of text which word tends to follow which, and for each request it computes the most likely continuation - token by token. Truth is not a criterion: the model optimises for plausibility, not correctness. As long as right and wrong answers sound equally fluent, it can produce both with the same conviction.

On top of that, generating is harder than recognising. A model can often judge whether a given statement is true, yet fail to produce the same information correctly from scratch. For rare facts with many possible values - an exact date, a precise number - the probability of hitting the single correct value is correspondingly low.

Why Training and Evaluation Reward Guessing

Hallucinations are not purely a technical accident; they are partly taught. A widely noted analysis by OpenAI researchers (Kalai, Nachum, Vempala, Zhang, September 2025) argues that common benchmarks treat models like exam candidates: a guessed answer may score points, whereas an honest 'I don't know' scores a guaranteed zero. Under such incentives guessing pays off - and that is exactly what we train models to do.

The practical upshot: as long as leaderboards reward confidence and penalise restraint, confident errors remain a systematic by-product. Part of the fix is therefore not a new model but better evaluation that once again values calibrated uncertainty and abstention.

When Hallucinations Are Especially Likely

• Rare niche knowledge: facts that barely appeared in the training data (long-tail entities, local details).
• Precise details: exact numbers, quotations, legal provisions, DOIs or citations.
• Recency: events after the model's knowledge cut-off with no connection to live sources.
• Ambiguous or leading prompts that take a false premise for granted.
• Long chains of reasoning in which early errors propagate.
• High 'temperature': more creative, more random outputs raise the risk on factual tasks.

Reducing Hallucinations: Grounding, RAG, Verification

No model never hallucinates - but a layered approach cuts the risk sharply. Each measure addresses a different class of error; combined, they work best. Start with grounding and abstention, which remove the largest share of errors for the least effort, and add verification where a wrong answer is expensive.

• Grounding / RAG: pull relevant documents into the context and instruct the model to answer only from them and cite sources.
• Allow abstention: explicitly permit 'I don't know' instead of forcing an answer.
• Verification: check critical outputs against sources - via a second model, chain-of-verification or human review.
• Constrained output: structured formats, pick-lists or schemas that make free invention harder.
• Lower temperature and precise prompting for factual tasks; give a clear role, context and format.
• Data quality: curate, update and organise the knowledge base by topic - poor sources produce poor answers.

The Swiss Angle: AI in Regulated Contexts

In Switzerland and the wider DACH region, AI outputs are increasingly used in sensitive domains - law, health, finance, public administration. Here a hallucination is not a cosmetic flaw but a liability and compliance risk. Anyone processing personal data remains accountable under the revised Data Protection Act (revDSG); the FDPIC expects traceable, accurate processes. An invented legal provision or diagnosis is, in case of doubt, not the AI's fault but that of the organisation that adopts it unchecked.

Practical upshot: in regulated processes, always pair AI outputs with human sign-off (human-in-the-loop), documented sources and clear responsibilities. Treat the model as a fast but unreliable assistant - not as a source of truth.