Hallucination is when AI plausibly produces content that's factually wrong or doesn't match the material it was given. OpenAI's research points to the cause as a setup in which models are trained and evaluated in a way that leads them to guess rather than say they don't know when they're unsure. In this post, I've summarized that cause along with the ways to reduce hallucination recommended in Anthropic's and Google's official guides.
You've probably heard stories at least once about handing research to AI and finding nonexistent statistics or sources mixed in. Because the sentences are smooth and sound confident, the tricky part of hallucination is that the wrong parts are even harder to notice. As I studied this while using AI at work, I found that hallucination isn't just bad luck; it has causes, and so the ways to reduce it are fairly well documented too. In this post, I'll go through this step by step.
1. What is hallucination?
Anthropic's guide to reducing hallucinations describes hallucination as a phenomenon in which even the most advanced models sometimes generate text that is factually incorrect or inconsistent with the given context. OpenAI's paper "Why Language Models Hallucinate," published in September 2025, calls it "plausible but false statements" and notes that it still persists even in the latest models.
An example in the paper is striking. When an open-source model was asked for the birthday of one of the paper's authors, with the condition "answer only with the date if you know it," it gave three different wrong dates across three attempts. It guessed even though it had been told not to answer if it didn't know.
2. Why does hallucination happen?
The OpenAI paper explains that hallucinations originate in the pretraining stage and persist through later training because of the way models are evaluated.
- Pretraining stage: Models learn language patterns from vast amounts of text, and the paper explains that facts with no pattern that appear only about once in the training data, like a particular person's birthday, are hard to tell right from wrong. By contrast, frequently mentioned facts like Einstein's birthday are almost never wrong.
- Evaluation methods: The paper points out that most benchmarks give 1 point for a correct answer and 0 points for a wrong answer or "I don't know," so models that guess when unsure end up scoring higher.
The paper compares this to a student taking a test. If there's no penalty for wrong answers on a multiple-choice test, it's better for your score to pick any option rather than leave a question blank, even if you don't know the answer. The explanation is that because models are optimized for these kinds of tests, they're always in "test-taking mode" and give plausible answers instead of saying they don't know. So the paper proposes changing evaluation methods to penalize wrong answers and not penalize "I don't know."
Flipping this explanation around from the user's side, it means it's important to give conditions where it's okay to say "I don't know" and material that can serve as the basis for the answer. In fact, the methods recommended by the official guides also converge on these two directions.
3. How do the official guides say to reduce it?
The Anthropic guide introduces three basic methods.
| Method | Official explanation |
|---|---|
| Allow it to say "I don't know" | Explicitly give permission to admit uncertainty when it isn't sure. The guide says this is simple but can drastically reduce false information |
| Have it quote the source first | When working with long documents over 20,000 tokens, have it extract the relevant parts word for word before doing the task |
| Have it verify with sources | Have it attach the quote and source supporting each claim, and remove any claim for which it can't find support |
For example, when asking it to summarize competitor analysis material, you could write something like this, based on the guide's examples.
Using only the content in the <documents> below, summarize the pricing policies of the three competitors.
Don't guess at anything you can't confirm from the documents; write "Not in documents" instead.
After you finish the summary, quote word for word the sentence from the documents that supports each sentence,
and remove from the summary anything for which you can't find a sentence to quote.
<documents>
(Paste the material here)
</documents>The same guide also introduces more advanced methods: having it explain its reasoning step by step before answering, asking the same question several times and comparing whether the answers contradict each other, having it re-verify a previous answer, and instructing it not to use general knowledge beyond the material provided.
Google's Gemini API prompting guide points in a similar direction. It recommends turning on grounding, which backs answers with Google Search, when you need facts that are less well known or recent, and turning on the code execution tool when you need calculations or counting. OpenAI's prompt engineering guide also introduces the approach of including material the model wasn't trained on in the prompt, to narrow the scope of the answer to that material.
4. Why do people still need to check?
The Anthropic guide states clearly that these methods greatly reduce hallucinations but don't eliminate them completely. So it recommends always verifying important information separately, especially information used for high-stakes decisions.
Applying this to marketing work, I think this covers numbers and dates in reports, product specifications, and anything about laws or policies. If you have the AI attach sources, you know right away where to look when verifying, so requiring sources is both a way to reduce hallucination and a way to cut the time people spend checking. For the same reason, drafts on this blog attach an official source to each claim, and I read them myself before publishing.
Summary
Hallucination comes from a setup in which models are trained and evaluated to guess even when they don't know, so the most basic countermeasure is to give room to say "I don't know" and material to base the answer on. Adding source citations and verification on top of that makes it much easier to find what's wrong. Please refer to the prompt example and checkpoints above when you hand research or summaries off to AI.
FAQ
- Can hallucination be eliminated completely?
- According to the official guides, that's difficult. The Anthropic guide says the methods it introduces greatly reduce hallucinations but don't eliminate them completely, and recommends always verifying important information separately.
- What should I write in a prompt to reduce hallucination?
- The Anthropic guide introduces these basic methods: allowing it to answer "I don't know" when unsure, having it quote the relevant parts of long documents first, and having it attach a supporting quote to each claim and remove claims without support. You can also instruct it not to use knowledge outside the material provided.
- How should I ask about recent information?
- The model may not know information from after its training cutoff. Google's Gemini API guide recommends turning on Google Search grounding when you need facts that are less well known or recent, and if you use other tools, you can paste in reliable material yourself and have it answer only from that material.
Related posts
- Tokens and context windows explained: basics to know before giving AI long material - What changes when you put in long material
- Chatbots vs. AI agents vs. automation - How to pick the right AI approach for the job
- AI comes to Google Search: how SEO changes in the AI Overviews era - How AI answers affect search traffic