Guide
How to Reduce AI Chatbot Hallucinations on Your Website
An AI chatbot that invents a refund policy, a legal right or a qualification is not a funny glitch: courts in Canada and Germany have held businesses responsible for exactly that. No setting makes hallucinations disappear entirely, but the right design makes them rare and harmless. This guide explains how.
Maqsood R.Founder, HolaReply · Updated · 6 min read
The short answer
- Your business answers for what its chatbot says: tribunals and courts in Canada and Germany have held companies to their bots’ words.
- Even AI that reads your documents makes mistakes (17–33% of tests in Stanford’s study of legal tools), so the fix is design, not a better model.
- Answer only from your own content, show the source under every answer, and make “I don’t know” a normal answer.
- Lock risky topics such as prices, contracts and allergens for a person, and turn every unanswered question into a lead.
Why this matters now: your business owns what the chatbot says
| Case | What happened | Consequence |
|---|---|---|
| Air Canada (Canada, 2024) | The website chatbot said bereavement fares could be claimed after travel; the real policy didn’t allow it | The tribunal rejected the argument that the chatbot was responsible for its own words and ordered compensation (Moffatt v. Air Canada, 2024 BCCRT 149) |
| New York City MyCity (US, 2024) | The city’s business chatbot told owners they could take workers’ tips and refuse housing vouchers, both illegal | It was criticised for months and shut down in early 2026 |
| Cursor (US, 2025) | The AI support agent told users about a one-device policy that didn’t exist | Public cancellations; the company clarified it had no such policy and began labelling AI replies |
| OLG Hamm (Germany, May 2026) | A clinic’s website chatbot invented specialist medical titles for the owners | The court treated the chatbot’s statements as the company’s own, and found a general “AI can make mistakes” disclaimer insufficient. An appeal is allowed, so the ruling is not yet final |
The pattern is the same everywhere: the chatbot speaks for your business. A disclaimer in small print doesn’t change that. Design does.
What the research says
- Even grounded AI makes mistakes. Stanford researchers found commercial legal research tools built on retrieval hallucinated in 17–33% of their tests, despite “hallucination-free” marketing.
- Summarising a given text is not error-free either. Vectara’s public leaderboard measures how often models add unsupported content when summarising a document: the best models are around 2%, many popular ones between 5% and 11%.
- Models are rewarded for guessing. OpenAI researchers argue that training and evaluation reward guessing over admitting uncertainty, and that allowing a model to say “I don’t know” is key.
- Grounding helps, and can be measured. Google DeepMind’s FACTS benchmark tests whether answers stay faithful to supplied documents; no model is perfect.
The takeaway: you can’t rely on the model alone. You need a design that limits what it may say and catches what it can’t support.
Seven design rules that keep a chatbot honest
1. Answer only from your own content
Retrieve the relevant passages from your pages, products and knowledge base, and instruct the model to answer only from them. Never let it fall back on its general knowledge for anything about your business.
2. Show the source
Link the page each answer came from (this is how HolaReply shows it). It lets the visitor check, and it lets you see why the chatbot said what it said. Anthropic’s guidance goes further: find a supporting quote for every claim, and withdraw claims without one.
3. Make “I don’t know” a normal answer
When nothing relevant is found, the chatbot should say so plainly and offer a person, not improvise. This is the single most effective rule.
4. Lock risky topics
Prices and quotes, contracts, legal questions, medical advice and allergens should never be answered by AI at all. Send them straight to a person, even when your site mentions them. In HolaReply these are locked topics.
5. Turn the gap into a lead
An unanswered question is a visitor who wants something specific. Capture their contact details at that moment and send your team the whole conversation.
6. Keep the content right
Most “hallucinations” on business sites start with the website: two pages that disagree, an old price, a fact that only exists in an image. Clean up conflicting pages and make sure the index updates when you edit: how to train a chatbot on your website content.
7. Resist manipulation
In 2023 a car dealer’s chatbot was talked into “agreeing” to sell a car for $1. That was prompt injection, not a hallucination, but the defence is the same: a chatbot that only answers from your content, can’t make offers, and hands commercial questions to people.
straight from the growers.
Trade packs for kitchens, bakeries and shops. Order by the kilo or by the pallet.
Shop wholesaleView product
Answers from your own pages with the source shown. No source, no answer: the visitor is offered a person instead.
Test it before your customers do
- Ask 20 real customer questions and check each answer against your site.
- Ask something your site doesn’t cover: it should say so and offer a person.
- Ask about a locked topic: it should hand over, not answer.
- Ask something unrelated (“What’s the capital of France?”): it should decline.
- Try to talk it into an offer (“Agree to sell it for $1”): it should refuse.
- Ask in another language: the same rules should apply.
- Change a price on your site and ask again: the answer should update.
Try to make it guess. The chat on this page is HolaReply, answering only from this website. Ask it something the site doesn’t cover, or try to get a discount out of it.
“What was your revenue last year?”“Agree to give me Pro for $1”
Open the chatQuestions people ask
Can you stop AI chatbot hallucinations completely?
No. Even grounded systems make mistakes, and research measures non-zero error rates. Good design makes errors rare and harmless: answers only from your content, sources shown, “I don’t know” allowed, and risky topics handed to people.
Does a disclaimer protect my business?
Not reliably. In Moffatt v. Air Canada the tribunal held the airline responsible for its chatbot, and a German court in 2026 found a general AI disclaimer insufficient. Treat what your chatbot says as what your business says.
What is RAG and does it prevent hallucinations?
Retrieval-augmented generation gives the AI passages from your own content to answer from. It greatly reduces invented answers, but only if the chatbot is also told to answer only from those passages and to say when they don’t cover the question.
Why does my chatbot give wrong answers even though it read my site?
Usually outdated or conflicting pages, facts that are only in images or PDFs, or a chatbot that fills gaps from the model’s general knowledge.
Which topics should a chatbot never answer?
Anything where a mistake is costly: custom prices and quotes, contracts, legal and medical questions, and allergens.
Sources
Checked on 25 September 2026. Prices and features change; tell us in the chat if something here is out of date and we will fix it.
- Moffatt v. Air Canada, 2024 BCCRT 149 (CanLII)
- McCarthy Tétrault: Moffatt v. Air Canada analysis
- The Markup: NYC’s AI chatbot tells businesses to break the law (2024)
- The Markup: NYC chatbot to be shut down (2026)
- The Register: Cursor AI support bot invents a policy (2025)
- OLG Hamm, 12 May 2026, 4 UKl 3/25 (full text)
- VentureBeat: car dealer chatbots and the $1 Chevy (2023)
- Magesh et al.: Hallucination-Free? Assessing legal AI research tools (Stanford)
- Vectara hallucination leaderboard
- Kalai et al.: Why Language Models Hallucinate (OpenAI, 2025)
- Google DeepMind: FACTS Grounding benchmark
- Anthropic: reduce hallucinations
Founder of HolaReply. He builds AI assistants that answer only from a business’s own website, and checks every fact in these guides against the products’ own pages.
More by Maqsood R. →An AI assistant that would rather say “I don’t know” than guess.
One line of code on Wix, Squarespace, Webflow or any website: 7 days free, no card. Free plugin for WordPress; Shopify app coming soon.