Measuring and reducing hallucinations

In short: A hallucination is a fluent statement by a language model that is false or not backed by any source. You can’t get it to zero, but you can measure it and cut it a lot.

Why models hallucinate

A language model predicts the most likely next word. Likely means “sounds plausible”, not “is proven”. Where knowledge is missing, the model fills the gap with something fitting anyway. More in Limits of AI: what models can’t do.

Two kinds

• Factual errors: the statement contradicts the world (wrong year, invented law).
• Faithfulness errors: the answer contradicts the given text or context, for example in a summary.

Measuring

1. Build a test set with questions and verified answers.
2. Split answers into individual claims and check each against the source.
3. Count the share of unsupported claims. That is the hallucination rate.
4. Also check whether the model says “I don’t know” when it lacks the knowledge.

Checkers can be people, fixed rules or a second model (see AI evaluation: tests, benchmarks and LLM-as-judge). A model as judge is cheap but makes mistakes itself and needs spot checks.

Reducing

• Provide sources and let it answer only from them (RAG pipeline).
• Require quotes and check that they really appear in the source.
• Allow abstaining: say explicitly “Answer only if the text supports it, otherwise say so”.
• Low randomness (temperature) for factual tasks.
• Check step: a second request verifies the first.
• Small tasks: short, well-defined questions hallucinate less than open ones.

What remains

Even with all of this a residual risk stays. For medicine, law and money a person who checks the sources must always be involved.