Why AI assistants make confident mistakes, a step-by-step method for checking AI answers, warning signs of invented facts and sources, and when not to rely on AI at all.
AI assistants write fluently and confidently, which makes their mistakes easy to miss. They can invent facts, quotes, statistics, legal cases and even references that look real. That doesn't make them useless, but it does mean treating their output as a first draft to check, not a final answer.
Why AI gets things wrong?
- Prediction, not knowledge: language models generate likely-sounding text. When they lack reliable information, they can produce plausible inventions, often called "hallucinations".
- Out-of-date information: models are trained on data up to a point in time. Unless the tool searches the web, it may not know about recent changes to laws, prices or events.
- Mixing sources: answers can blend UK and US rules, or combine details from different products.
- Agreeing with you: a leading question can produce an answer that tells you what you seem to want.
A five-step checking method
- Identify the claims that matter: names, numbers, dates, prices, legal rules, medical advice and quotes.
- Find a primary source for each: the official website (GOV.UK, the NHS, a regulator), the original study, or the company's own page. Don't rely on another AI summary.
- Open every reference. If the AI cites a source, check it exists, says what's claimed and is current. Invented references are common.
- Check the date and location. Is the rule current, and does it apply in the UK, or in England rather than Scotland?
- Cross-check important points with a second independent, reputable source.
Warning signs
- very precise figures without a source;
- references with plausible titles but links that don't work or lead elsewhere;
- quotes attributed to real people that you can't find anywhere else;
- answers that change significantly when you ask the same question again;
- US terms or rules in an answer about the UK (for example "401(k)" or "Social Security").
Ask better questions
- Ask the tool to say when it is unsure, and to separate facts from assumptions.
- Specify the country and date: "under UK rules in 2026".
- Ask for sources, then check them yourself.
- Use tools with web search for current information, but still open the links.
When not to rely on AI?
For health, legal, financial or safety decisions, use AI only to understand the topic or prepare questions. Then check with official sources or a qualified professional: the NHS website or 111 for health, Citizens Advice for consumer and legal problems, and MoneyHelper for money guidance. There have been cases in the UK and elsewhere where people cited non-existent legal cases generated by AI in court.
Images, audio and video
AI can also generate realistic fake images, voices and videos. Be sceptical of shocking content, check whether reputable news organisations are reporting it, and look for the original source. Fraudsters use cloned voices in scams, so verify any urgent request for money through a separate channel.
A worked example
Suppose an assistant tells you the State Pension age is rising and gives a date. Search GOV.UK for "State Pension age", open the official page, and compare the date and the birth-date ranges it applies to. If the AI's figure differs, trust the official source, and treat the rest of the AI answer with more caution.
Using AI well
AI is good at explaining concepts, summarising text you provide, drafting, brainstorming and suggesting what to look up. It's weakest when you need precise, current, verifiable facts. Use it for the first, and check the second.
Frequently asked questions
Are AI tools with web search reliable?
They are better for current information but can still misread or misquote sources. Open the links.
Can I use AI-generated text in my work?
Check your employer's or institution's rules, check facts carefully, and take responsibility for the final result.
Sources and further reading
Facts checked on 20 October 2026. Spotted an error? Tell us and we will correct it.