AI agents hallucinate in production because real conditions don’t exist in a demo. Messy data, multi-step tool chains, and live systems that shift mid-task never show up in a sandbox test.
A chatbot that hallucinates gives you a wrong sentence. An agent that hallucinates can write that sentence into your database or send it to a customer.
In February 2024, a tribunal held Air Canada liable for exactly this. In July 2025, an AI coding agent deleted a production database during a declared code freeze.
Quick answer: AI agent hallucinations come from four failure points: what the agent was told, what it could retrieve, what it could do, and what left without a check. Fix the specific layer where the failure happened, not “the model.”
Key Takeaways
- Agent hallucinations cost more than chatbot ones, because agents write to databases and send messages instead of just talking.
- Five real 2024 to 2025 incidents, at Air Canada, New York City, Cursor, Deloitte, and Replit, prove this isn’t rare.
- The strongest models still hallucinate at roughly 3% on grounded tasks, per Vectara’s 2026 leaderboard.
- Only one in five companies has mature AI governance, per Deloitte’s 2026 research.
- The fix is four layers working together, not a smarter model or a longer prompt.
Five Agents That Already Hallucinated in Production
| Company | What happened | What it cost |
|---|---|---|
| Air Canada (Feb 2024) | Chatbot invented a refund policy that didn’t exist | CAD $812.02 damages, plus legal liability for chatbot claims |
| NYC “MyCity” (2024) | Told businesses illegal advice was legal | Roughly $500,000 a year to run a tool later shut down |
| Cursor (Apr 2025) | Support bot invented a login policy | Real subscription cancellations |
| Deloitte Australia (2025) | Report contained fabricated citations | AUD $290K refund on a $440K contract |
| Replit (Jul 2025) | Agent deleted a production database during a code freeze | Total data loss; the agent rated itself 95/100 on its own failure |
Four of these five agents said something false that a person believed. Replit alone acted on its own false belief with no one in the loop.
That split is the real line between an agent and simple no-code automation: one of them can act on its own.
Why This Costs More Than a Chatbot Mistake
A hallucination is a plausible-sounding output with nothing true behind it. Models predict the next likely word. They don’t check it against a source first.
An agent hallucination doesn’t stop at the screen the way a chatbot’s does. It becomes the next tool call or the next database write, and errors compound across steps.
- At 95% accuracy per step, roughly 40% of ten-step tasks still fail.
- At 99% accuracy per step, close to 9% still fail over ten steps.
A single accuracy score on a dashboard tells you almost nothing about a real multi-step workflow.
The Real Root Causes
“The model made a mistake” is rarely a useful diagnosis. Three patterns explain most real incidents.
- Data gaps. Missing context, duplicate records, or stale policy docs the agent has to reconcile on its own.
- Retrieval failures. The wrong document gets fetched, or a huge context window buries the one fact that mattered.
- Execution errors. Reasoning drift across steps, wrong tool calls, or a failed task reported as done anyway.
The Diagnostic Most Teams Skip
Not every hallucination shares a cause. Split each incident into one of three buckets before touching anything.
- Never stored. The fact was never captured anywhere the agent could reach.
- Stored but not retrieved. It exists in your systems, but retrieval missed it.
- Retrieved but ignored. The model had the fact and still didn’t use it.
Bucket three is a prompt or context-length problem, not a retrieval problem. Confusing the two wastes weeks rebuilding a pipeline that was never actually broken.
The Four-Layer Hallucination Stack
This is the framework we use when scoping core AI solutions for a business: four layers, four separate owners.
| Layer | What breaks | The fix |
|---|---|---|
| 1. Input | Model guesses at ambiguous instructions | Sanitize input, allow “I don’t know” |
| 2. Grounding | Agent invents a missing fact | Require citations, fix retrieval |
| 3. Action | Agent takes an unauthorized action | Tool allowlists, propose-then-execute |
| 4. Output | Wrong result reaches a user unchecked | Structured output, a policy check |
Air Canada’s gap sat in Output, where nothing caught the fabrication. Replit’s gap sat in Action, where a declared code freeze got overridden anyway.
The Blast Radius Test
Score any agent on three questions before deciding how much rigor it needs.
- Reach. Can it act without a human checking first?
- Reversibility. Undo in minutes, or a cleanup project? The same factor in our readiness checklist.
- Detection speed. Who catches a confident wrong answer, and how fast?
Replit scored high risk on all three: total reach, near-zero reversibility, and a founder finding out only after the damage was done. Our guide to customer data access walks through the full tiering.
The Receipt-Before-Action Pattern
Most guardrails stop right before an agent acts. That’s too early to catch a stale permission.
A connector can show “Connected,” the agent proceeds, and the write fails minutes later on an expired token. Nothing looked wrong to the agent at the time.
The fix is a receipt check immediately before every real action, confirming three things:
- Is the credential valid right now, not five minutes ago?
- Does the scope cover this specific action?
- Are the parameters still intact?
This is the same watcher process behind our readiness checklist: a separate check that an action actually landed, not just that the agent believes it did.
How to Actually Stop It
No single technique closes every gap. Grounding fixes wrong facts, not bad tool calls, so all four layers need their own fix.
1. Fix Grounding
- Hybrid retrieval with reranking.
- A “no source, no answer” rule.
- Live systems of record, not stale exports.
2. Fix Action
- Tool allowlists per task or role.
- Propose-then-execute on anything destructive.
- A receipt check right before execution.
3. Fix Output
- Structured output: answer, source, confidence.
- A validation check before anything ships.
- Logging every decision, not just failures.
A second “critic” agent only helps if it checks against something deterministic. Two models with no ground truth often agree on the same mistake, so confirm a second agent is actually needed first.
The 30-Day Hardening Plan
- Week 1. Map every action the agent can take, score each with the Blast Radius Test.
- Week 1 to 2. Fix grounding, test against 20 or more real queries.
- Week 2. Add allowlists and propose-then-execute.
- Week 3. Add output validation, logging, and a spend cap.
- Week 3 to 4. Test with real messy input, not clean prompts.
- Week 4. Name one accountable owner. Not a team.
Mistakes That Make It Worse
A few habits quietly undo all of the above.
- Treating a stricter prompt as a substitute for grounding.
- Testing only on clean sandbox data.
- Skipping propose-then-execute because it “probably won’t happen.”
- Trusting a critic agent with no ground truth to check against.
- Dropping human review the moment the agent looks reliable.
- Running with no circuit breaker, so a broken loop just keeps going.
What This Costs and How Long It Takes
A low-reach agent needs modest engineering time, roughly the 30-day plan above. A high-reach agent with database or payment access needs dedicated grounding work, a real testing budget, and ongoing monitoring.
Over 40% of agentic AI projects are projected to be canceled by 2027, per Gartner. Weak models are rarely why. Skipped groundwork usually is.
One Last Check Before You Ship
None of this matters if the workflow wasn’t ready for an agent to begin with. Our readiness checklist scores that first, before you spend on anything else.
Stop Guessing Whether Your Agent Is Safe to Ship
Air Canada, Deloitte, and Replit didn’t find out from a dashboard. They found out from a customer, an auditor, or a founder watching data disappear in real time.
GVM Technologies AI reviews an agent’s four layers before recommending anything, including telling a client to wait.
Talk to our team about where your agent’s risk actually sits. No pitch, just a specific read on the four layers.
Frequently Asked Questions
1. Can AI agent hallucinations be eliminated completely?
No. It’s a property of how models generate text. The goal is reducing the rate and catching what slips through.
2. Does AI still hallucinate in 2026?
Yes. Vectara’s 2026 data puts the best models at roughly 3%. A newer model helps; it doesn’t replace the four layers.
3. Is RAG enough on its own?
No. It only helps if retrieval is accurate and the agent must cite what it retrieved.
4. How do I know if my agent is hallucinating?
Look for output with no source, actions a watcher process can’t confirm, and quiet quality drift over time.
5. What’s the difference between a hallucination and a tool-use error?
A hallucination is false information. A tool-use error is calling the wrong tool. A timeout often triggers the first when it’s really the second.
6. Does a second “critic” agent fix hallucinations?
Only if it checks against something deterministic, not just a second opinion.
7. How much does making an agent production-safe cost?
It scales with blast radius, not company size.
8. Should one hallucination pause a rollout?
Not automatically. Run the Blast Radius Test first. A high-reach failure is the real signal to pause.
The Bottom Line
Air Canada, NYC, Cursor, Deloitte, and Replit weren’t running weak models. They shipped agents whose input, grounding, action, and output layers were never checked against a real blast radius.
Fix the layer where the failure actually happened, split missing versus ignored before choosing a fix, and test against messy real data instead of clean prompts. That’s what turns a headline into a caught, logged, corrected non-event.
