We’re all very aware of the reported downfalls of AI and probably a bit scared to put our own and our brand’s name on the line. But with the right setup and guardrails in place, you can feel confident in embracing the future.
AI hallucinations are not glitches. When a language model invents a fact, it is doing what it was trained to do: produce the most plausible answer. That makes hallucination a structural problem, but a fixable one, because the fix sits outside the model: verified, well-structured source material, strict grounding, and serving critical answers verbatim rather than asking an AI to regenerate them. The same AI tool can hallucinate on half its answers or none of them; the difference is how the system around it is built.
What is an AI hallucination?
The word makes it sound like the machine is just a bit unwell or has temporarily gone off the rails… just a bit of glitchiness the next update will cure.
When an AI system invents a fact, cites a report that does not exist, or confidently describes a policy nobody ever wrote… it is not malfunctioning. It is doing exactly what it was built to do: predict the next word based on the information it has, and fill any gaps with whatever sounds right.
Plausible and believable is the job. Correct was never the bar.
That sounds like bad news, but it’s actually not. A problem with a known, well-understood mechanism is a problem you can engineer around, and the evidence for how to do that is now solid, peer-reviewed, and surprisingly clear.
This article walks through what hallucination really is, what it has already cost businesses that ignored it, and what the research says actually fixes it. Spoiler: the fix has far more to do with your information than with anyone’s model.
And it is worth understanding properly, because the future it prepares you for… is arriving whether you like it or not.
Something has quietly shifted in how people expect to deal with a business. The preference for typing a question in plain language and expecting a direct, useful answer is no longer an early-adopter behaviour; it is becoming the default.
The One NZ AI Trust Report 2026, a census-weighted survey of just over 1,000 New Zealanders, found that 76% had used an AI-powered tool or service in the past year, with chat-based assistants leading the way. Before long, a customer landing on your website and finding no way to simply ask their question will feel like we’re back in time, not ruined, but noticeably behind.
So for most businesses the real question is not whether an AI assistant is coming. It is how to get one you can stand behind.
And that is where a healthy wariness kicks in, because you might have heard the horror stories. If you are hesitant to let someone bolt an AI onto your brand, that instinct is sound. The same One NZ research found 62% of New Zealanders would stop using a company over concerns about its AI use, and 70% of AI users had struck problems in the past year, with incorrect information high on the list. Your customers will hold you, not your technology vendor, responsible for what your AI says.
Here is the encouraging part. The failures are not random, and they are not inevitable. The difference between a chatbot that damages a brand and one that quietly strengthens it is not luck, and it is not the model. It is whether the people building it understood the problem before they started. New Zealanders themselves seem to grasp this instinctively: the One NZ report concluded that people are not asking for less AI, they are asking for stronger guardrails around how it is used.
“real question is not whether an AI assistant is coming. It is how to get one you can stand behind.”
“The difference between a chatbot that damages a brand and one that quietly strengthens it is not luck”
Let’s start with the most instructive examples on record, and these did not happen to a bunch of careless amateurs.
Within nine months, three of the world’s Big Four professional services firms published AI-fabricated material under their own brands. In October 2025, Deloitte Australia repaid the final instalment of a A$440,000 contract, A$97,587.11, after its assurance review for the Department of Employment and Workplace Relations was found to contain fabricated academic citations and a made-up Federal Court quote. In May 2026, EY Canada retracted a 44-page cybersecurity report the same day researchers published evidence that 16 of its 27 citations were fabricated, misattributed, or broken, including a McKinsey report that has never existed. In June 2026, KPMG withdrew a global report on agentic AI after UBS, NHS Greater Manchester, Swiss Federal Railways, and Transport for London all publicly disputed the case studies describing their AI use. UBS called the claims about it “factually incorrect.”
These are firms whose product is verification. They audit the internal controls of the largest organisations on earth. They sell AI governance frameworks. What their stumbles prove is not that AI cannot be trusted. It is that AI output cannot be trusted by default, and that even world-class organisations get burned when they skip the unglamorous verification work. The failure was never the technology alone. It was the absence of a process around it.
That distinction matters, because the wider market is full of providers offering the technology without the process.
“AI output cannot be trusted by default, and that even world-class organisations get burned when they skip the unglamorous verification work.”
The AI solutions market right now is loud, overconfident, certain it knows better, and growing faster than its own ability to coordinate. Call it adolescence.
The numbers behind that impression are sobering once you strip out the hype in both directions. IBM’s 2025 study of 2,000 CEOs across 33 countries found only 25% said their AI initiatives had delivered the expected return, and only 16% had scaled anything enterprise-wide. S&P Global Market Intelligence found 42% of companies abandoned most of their AI initiatives in 2025, up from 17% the year before. And in one fast-growing corner of the market, Gartner estimated in mid 2025 that of the thousands of vendors claiming to sell “agentic AI,” only around 130 were genuinely doing so. The rest had relabelled.
You may have seen a scarier statistic doing the rounds: that 95% of AI projects fail. Treat it carefully. It comes from a preliminary MIT-affiliated report that measured something narrower, whether pilots showed a measurable profit-and-loss impact within a short window, and its authors describe their own figures as directionally accurate. The way that number has been laundered into a hard fact is itself a lesson in how this industry handles evidence.
Here is the honest version: adoption is broad, production scale is rare, returns are thin, and the demos all look the same. The differences only surface after deployment, when the buyer is living with the system, and knowing how to tell “works in a scripted demo” apart from “never misleads your customers” is the most valuable skill a buyer can bring to this market.
Telling them apart starts with the mechanism behind the failures.
“CEOs across 33 countries found only 25% said their AI initiatives had delivered the expected return”
“adoption is broad, production scale is rare, returns are thin, and the demos all look the same.”
A language model is trained to predict the next plausible word from the context it has. The most direct statement of this comes from OpenAI itself, in a research paper published in September 2025, which argues that models hallucinate because training and evaluation procedures “reward guessing over acknowledging uncertainty.”
The mechanism is worth a minute of your time. Most of the industry’s benchmark tests score models the way a multiple-choice exam scores students: a right answer earns a point, and both a wrong answer and “I don’t know” earn zero. Under those rules, guessing always beats abstaining. The models learned to bluff because bluffing was rewarded. The same research shows a floor under the problem: facts that appear only once in a model’s training data follow no learnable pattern, so when asked about them, the model can only produce a plausible guess.
Why is any of this reassuring? If hallucination were random noise, no one could promise you anything. Because it is the predictable result of asking a probability engine to answer from its own memory, the solution is equally predictable: stop asking it to answer from its own memory. Everything that follows in this article is a version of that move.
It also tells you what will not work: waiting.
Academic work published in 2024 proved formally that hallucination cannot be fully eliminated from the models themselves, and the CEO of Vectara, a company that measures hallucination for a living, put it plainly: “Despite our best efforts, they will always hallucinate.” The businesses winning with AI are not the ones waiting for a perfect model. They are the ones building systems that do not depend on one.
“models hallucinate because training and evaluation procedures reward guessing over acknowledging uncertainty.”
If you go looking for a single “AI hallucination rate,” you will not find one, and anyone who quotes you one is selling something. What you find instead is a range, and the range is the most hopeful fact in this entire subject.
At one end, under the most favourable conditions measured, the numbers look excellent. Give a top model verified source material and a constrained task, and hallucination rates sit around 0.7 to 2%. At the other end, the same class of models, asked open-ended factual questions with no source material provided, hallucinate on roughly 40 to 52% of them by OpenAI’s own published measurements. General-purpose models asked specific legal questions got them wrong 58 to 88% of the time in a landmark Stanford-led study. A 2025 European study of AI assistants answering questions about the news found significant errors in 45% of responses.
The middle of the range is where most business systems live today. Retrieval-augmented systems, the architecture behind most serious business AI, measured 6.65 to 16.05% hallucination across frontier models in a 2025 academic benchmark. Purpose-built legal research tools, engineered by major vendors specifically to be grounded and reliable, still produced wrong or fabricated answers 17 to 33% of the time.
Now look at the spread, from under 1% to 88%, and notice what separates the ends. It is not the model. Often it is literally the same model. What changes is whether the system is constrained and grounded in verified source material, or left to answer from its own statistical memory. That distinction is the entire game. It is why the same technology that fabricated citations in a Big Four report can, properly harnessed, answer your customers’ questions more accurately than your website’s search bar ever did. And it is precisely the distinction a polished demo is designed to blur: a demo is a constrained, rehearsed, favourable-conditions test. Your customers are not.
The buyer’s job, then, is not to find magic technology. It is to find a builder who knows how to keep the system at the good end of the range. Which is knowable, checkable, and the subject of the rest of this article.
“models, asked open-ended factual questions with no source material provided, hallucinate on roughly 40 to 52% of them”
It is worth being clear-eyed about the stakes, because they are also the reason to do this properly rather than not at all.
In February 2024, a Canadian tribunal ordered Air Canada to compensate a passenger after its website chatbot invented a bereavement fare policy that did not exist. The airline argued, remarkably, that the chatbot was a separate legal entity responsible for its own actions. The tribunal rejected that outright, ruling the airline responsible for all information on its website, chatbot included. The dollar amount was small, C$812.02. The precedent was not: your AI’s words are your words.
The legal system’s own experience shows the scale of careless use. A public database maintained by legal researcher Damien Charlotin was tracking, as of July 2026, 1,745 court and tribunal decisions worldwide in which judges found or suspected reliance on AI-hallucinated material, including 96 in Australia and 7 in New Zealand. In February 2026 the issue reached New Zealand’s highest court, which found that submissions before it cited authorities that “appear to have been hallucinated” by an AI application, and warned that in serious cases such filings could amount to obstruction of justice or contempt of court. Two months later, the Federal Court of Australia issued a formal practice note governing generative AI in all filings.
And when an AI experience goes wrong, customers do not blame the AI vendor. A global YouGov survey found 71% of consumers hold the deploying company responsible for a chatbot’s errors; in Australia the figure was 80%. The Air Canada ruling made the same allocation legally. The brand wearing the chatbot wears the consequences.
None of this is a reason to sit out; the expectation shift is happening regardless. It is a reason to treat the build the way you would treat anything else that speaks publicly for your brand: as something worth doing properly, once.
“Vanta’s support lead reports fielding 45% fewer inbound inquiries over email even as its user base doubled.”
Given everything above, the tempting fixes are more technology: a smarter model, a cleverer prompt, another AI agent checking the first one. The evidence points somewhere less glamorous, and far more within your control.
Start with what does not work on its own. The research is blunt about prompting: a widely cited 2023 study found language models struggle to correct their own reasoning without external feedback, and sometimes get worse when asked to try. The model checking its own homework was never going to be the answer, because the homework and the checking come from the same statistical process.
What does work is grounding: connecting the model to verified source material and constraining it to answer from that material. But the most important study on the subject shows that grounding alone is not the fix either. The fix is what you ground it in.
In 2025, researchers published a peer-reviewed study in JMIR Cancer that comes remarkably close to a controlled experiment. They built AI chatbots to answer cancer information questions, holding the model constant and varying only the knowledge source. With no grounding at all, the chatbot hallucinated on roughly 40% of answers. Grounded in uncurated web search results, the rate fell to somewhere between 6 and 19%. Grounded in a curated, authoritative, purpose-built information corpus, the rate on in-scope questions fell to zero. Same model. Same architecture. The only variable that took hallucination to zero was the quality and structure of the source material. Just as telling: when questions fell outside the curated corpus, the well-grounded systems increasingly declined to answer rather than guess, while the ungrounded chatbot always produced something, including fabrications.
Read that result again, because it is the reassurance wary buyers are looking for, in peer-reviewed form. For the questions a system is built and scoped to answer, hallucination is not a gamble you are forced to accept. It is an engineering outcome that follows from doing the information work first. Gartner’s research points at the same truth from the failure side: through 2026 it expects organisations to abandon 60% of AI projects that are not supported by AI-ready data. The failures are data-shaped, not model-shaped. The unglamorous work of sorting your source material, resolving its contradictions, and filling its gaps before anything touches technology is not a preliminary to the real project. It is the project.
One honest limit, which any credible provider should volunteer before you ask: even well-built retrieval systems retain a residual error rate on open-ended generation. The Stanford legal study found purpose-built professional tools still hallucinating 17 to 33% of the time, and its authors concluded that retrieval is “no panacea.” Which leads to the final design move, the one that sounds almost heretical in an AI industry: sometimes the right answer is to take the AI out of the answer.
Where the words matter word for word, in compliance statements, pricing, policy terms, safety information, the safest architecture does not ask a model to regenerate the answer at all. It retrieves the sanctioned answer and serves it verbatim, with plain deterministic code, reserving the language model for the questions where genuine synthesis earns its place. A system that serves a stored, verified answer has no generation step in which to hallucinate. This is not exotic; it is the same logic your bank applies when it displays your balance from a database instead of asking a model to estimate it. If your expectation is to control outputs word for word, an AI model alone was never the right tool. A well-designed hybrid system, deterministic where exactness matters and generative where it adds value, can be.
“Vanta’s support lead reports fielding 45% fewer inbound inquiries over email even as its user base doubled.”
Pull the threads together and the picture is genuinely encouraging. Hallucination, the thing that rightly makes buyers wary, is structural but understood: the machine was built to be plausible, and plausible was rewarded. What moves a system from the dangerous end of the accuracy range to the dependable end is not the badge on the model but the constraint, structure, and quality of the information underneath it. All of which means the risk is not in having an AI assistant. The risk is in who builds it, and how.
So the question to ask any AI provider, ours included, is not whether they can build it. Almost anyone can get something running, and the demo will look wonderful. The question is: what is your plan for auditing this system and stopping it from being confidently wrong? What is served verbatim and what is generated? What happens when the answer is not in the source material? Who reviews the logs, and how does what they find flow back into the knowledge base?
A provider with real answers will start talking about your information before they talk about their technology, because that is where accuracy is actually made. A provider without real answers will change the subject back to the demo.
Get that question answered properly, and an AI assistant stops being a risk you are bracing for. It becomes what it should have been all along: the most direct, most accurate way your customers have ever had to ask you anything.
What is an AI hallucination?
When an AI system states something false with confidence: inventing facts, citing sources that do not exist, or describing policies nobody wrote. It is not a malfunction. Language models are trained to produce the most plausible answer, not the most correct one, so when the facts are missing they fill the gap with whatever sounds right.
How often do AI models hallucinate?
There is no single rate, and the range is the point. Grounded in verified source material on constrained tasks, top models sit around 0.7 to 2%. Asked open-ended factual questions with no source material, the same class of models err on roughly 40 to 52%. The variable is the system around the model, not the model itself.
Can AI hallucinations be fixed?
Not inside the model; research shows they cannot be fully eliminated from the models themselves. But they can be engineered around: ground the AI in a curated, verified knowledge base, constrain it to answer only from that material, decline out-of-scope questions, and serve critical answers verbatim rather than regenerating them. A peer-reviewed 2025 study measured zero hallucinations on in-scope questions with exactly this design.
Who is responsible when a company’s chatbot gets it wrong?
The company. A Canadian tribunal ordered Air Canada to compensate a passenger over a bereavement policy its chatbot invented, ruling the airline responsible for all information on its website. Consumers agree: 71% globally, and 80% in Australia, hold the deploying company, not the AI vendor, responsible for a chatbot’s errors.
Linkki builds BrandRAG systems: branded, retrieval-first AI assistants built on structured, verified source material, with strict guardrails and continuous auditing.
This article is part of an ongoing series on how AI is changing the way customers find and trust brands.
Or give us a call on +64 21 280 2773
Or give us a call on +64 21 280 2773