Legal Liability for AI Hallucinations in Business Contexts

Every technology wave eventually meets the legal system. We are now deep inside that collision with artificial intelligence. Business owners, executives, and professionals across every sector are integrating AI tools into their workflows at a pace that outstrips the development of legal guardrails. That gap between capability and accountability is where liability is born.

This post examines one of the most consequential and least understood risks of the AI era: what happens when an AI system confidently produces false information — a hallucination — and someone acts on it. Who is responsible? Who pays? And what can your business do to reduce exposure right now?

What AI Hallucinations Are, and Why They Happen

The term “hallucination” entered the mainstream conversation about AI to describe a specific and persistent failure mode: a large language model (LLM) generating output that is factually wrong, entirely fabricated, or internally inconsistent, yet presented with the same confident tone as accurate information.

To understand why this happens, a brief technical note is useful for non-engineers. Models like ChatGPT, Google Gemini, and Microsoft Copilot are trained on enormous datasets of text. They do not store facts the way a database stores records. Instead, they learn patterns — statistical relationships between words and concepts. When asked a question, the model generates a response by predicting which sequence of words is most likely to follow from the input. That process is probabilistic. The model is not looking up a true answer; it is generating a plausible-sounding one.

When the model’s training data is incomplete, outdated, or internally conflicting on a particular topic, it can generate responses that sound authoritative but are wrong. It may cite cases that do not exist, quote studies that were never conducted, or describe regulatory requirements that were never enacted. The model has no internal alarm that fires when it does not know the answer. It generates text either way.

A 2024 Stanford study found that legal AI tools hallucinate in roughly one out of six benchmark queries. Research on medical AI has shown hallucination rates in certain clinical contexts running as high as 50 percent for transcription models and 12.5 percent for cancer treatment recommendations. These are not edge cases. They are foreseeable failure modes baked into the technology.

For business owners, the practical risk is not abstract. It is the contract clause your AI drafting tool fabricated, the compliance requirement your AI research tool invented, the case citation your outside counsel submitted without verifying.

The Case That Changed Everything: Mata v. Avianca

The most widely known AI hallucination case in the United States is Mata v. Avianca, Inc., 678 F.Supp.3d 443 (S.D.N.Y. 2023). It is worth understanding in detail because it exposes exactly how the liability chain works when AI-generated content reaches a decision-maker without verification.

Roberto Mata filed a personal injury lawsuit against the airline after a serving cart struck his knee on a flight. His attorneys — Steven Schwartz and Peter LoDuca of the Levidow, Levidow & Oberman firm — filed a brief opposing Avianca’s motion to dismiss. That brief cited six cases as supporting authority. Not one of them was real.

The cases had been generated by ChatGPT. They carried plausible-sounding names, fake docket numbers, fabricated quotations, and citations to internal sub-decisions that were themselves fictional. When Avianca’s counsel reported that the cited cases could not be located, the court ordered the attorneys to produce copies of the decisions. The attorneys — relying on ChatGPT to confirm the cases were real — produced documents from the chatbot. ChatGPT, when directly asked whether Varghese v. China Southern Airlines was a real case, responded that it was and that it could be found in reputable legal databases such as LexisNexis and Westlaw. That was also false.

Judge P. Kevin Castel’s June 22, 2023 opinion is a meticulous dissection of the fabrications. The “Varghese” decision, for example, bore a docket number associated with an entirely different case, quoted language that did not appear in any real decision, and cited sub-cases that themselves did not exist. The court found that attorney Schwartz had acted with “subjective bad faith” — not merely negligence — in part because by the time he submitted the cases, he had already begun to suspect they might not be real and had asked ChatGPT to confirm them rather than checking an actual legal database.

The sanctions order imposed $5,000 each on Schwartz, on LoDuca (who signed the brief without reviewing the research), and on their firm jointly and severally. The court noted that while the dollar amounts were relatively modest — calibrated to deter rather than destroy — the reputational harm and the principle established were far more significant.

The broader significance of Mata is that it established the rule clearly: Rule 11 of the Federal Rules of Civil Procedure imposes an affirmative duty on each attorney to conduct a reasonable inquiry into the viability of a pleading before signing it. “ChatGPT told me it was real” is not a defense.

A Growing Docket of AI Hallucination Consequences

Mata v. Avianca was not an isolated incident. As of mid-2026, researchers tracking AI-related court proceedings have documented over 1,300 matters involving AI-fabricated content submitted to courts. The sanctions have escalated dramatically since 2023.

In Couvrette v. Wisnovsky, a federal court in Oregon confronted a case involving an intrafamily dispute over a winery. Over five months and across three separate summary judgment briefs, plaintiffs’ counsel submitted 15 AI-generated fake case citations and eight fabricated quotations. The court struck the briefs, dismissed plaintiffs’ claims with prejudice, and fined lead counsel $15,500, in addition to awarding attorney fees for both the underlying summary judgment and the sanctions briefing. The court’s opinion pointed specifically to the attorneys’ failure to run the citations through a citator — a basic verification step that would have immediately exposed the fabrications.

Courts outside the United States have addressed the same problem. In Moffatt v. Air Canada, 2024 BCCRT 149, a British Columbia Civil Resolution Tribunal ruled that Air Canada was liable for negligent misrepresentation because its AI chatbot gave a consumer inaccurate information about bereavement fare refund eligibility. The chatbot told Jake Moffatt he could apply for a reduced bereavement rate retroactively, within 90 days of ticket issuance. Air Canada’s written policy said exactly the opposite. The tribunal awarded $812.02 in damages and fees. The tribunal’s reasoning was direct: Air Canada is responsible for all information on its website, whether that information comes from a static page or from a chatbot. The company cannot disclaim responsibility by pointing at the AI.

The medical context adds additional stakes. Studies published in peer-reviewed literature have documented instances where AI models generated entirely fabricated patient summaries, suggested non-existent medication dosages, and hallucinated drug interaction warnings that caused physicians to avoid effective treatments. No landmark malpractice verdict involving AI reliance has been reported as of mid-2026, but malpractice filings involving AI diagnostic tools increased by 14 percent between 2022 and 2024.

Negligence Theory: When Does Unreasonable Reliance Create Liability?

The core negligence question in AI hallucination cases is whether a business or professional acted reasonably when they relied on AI output without independent verification. Courts analyze negligence through four familiar elements: duty, breach, causation, and damages. AI hallucination cases fit within this framework, though with some novel wrinkles.

Duty. A duty of care exists where a defendant’s conduct creates a foreseeable risk of harm to a specific class of persons. In commercial and professional contexts, the existence of a contractual or service relationship typically establishes duty. The physician owes a duty to the patient. The attorney owes a duty to the client. The financial adviser owes a duty to the investor. When a business provides information to customers — including through an AI-powered interface — it owes a duty of reasonable care that the information is accurate, or that users are clearly warned of its limitations.

Breach. Whether reliance on AI output constitutes a breach depends on the circumstances. Courts will ask whether the plaintiff acted as a reasonably prudent person in the same situation. The foreseeability of AI hallucinations is now so well-documented and so widely publicized that a professional who relies on AI-generated legal research, medical recommendations, or financial analysis without independent verification is increasingly difficult to defend. Ignorance of a tool’s known limitations is not a shield — as the Australian court found in Handa & Mallick [2024] FedCFamC2F 957, where a solicitor who submitted hallucinated authorities was prohibited from handling trust money or practicing unsupervised for two years.

Causation. The plaintiff must show that the AI-generated false output, acted upon by the defendant, was the actual and proximate cause of the harm suffered. This element is often the most complex. Where a business used AI to generate a contract that missed a key provision, and that omission caused loss, causation is reasonably traceable. Where an AI tool gave wrong legal advice that a company followed, causing a regulatory fine, the chain is clear enough.

Damages. Economic loss — lost profits, transaction costs, regulatory penalties, remediation expenses — is recoverable. The difficulty is that courts in some jurisdictions require physical or property damage as a predicate for pure economic loss claims in negligence. This is why contract theories often run alongside negligence claims in AI hallucination disputes.

The “Reasonable Reliance” Question

Closely tied to negligence analysis is the doctrine of reasonable reliance, which appears across multiple legal theories — negligent misrepresentation, fraud, promissory estoppel, and consumer protection law. A plaintiff must generally show that their reliance on a false statement was objectively reasonable under the circumstances.

Courts applying this doctrine to AI will almost certainly ask: what did the user know, or what should they have known, about the limitations of the AI tool? AI providers display prominent warnings about hallucinations directly in their interfaces. Every major provider’s terms of service explicitly disclaim accuracy and tell users not to rely on AI output as a sole source of truth. A business owner who ignores those warnings and acts on AI output without verification will have a harder time arguing that their reliance was reasonable.

However, the sophistication of the AI’s presentation is legally relevant. A chatbot that presents information in formal, authoritative language — as Air Canada’s did — creates a more sympathetic case for the consumer who believed it. A medical AI that presents a fabricated treatment recommendation in clinical language indistinguishable from expert guidance creates a stronger argument for patient reliance. Courts will weigh the totality of circumstances: the nature of the relationship, the format and tone of the AI output, any warnings provided, and the plaintiff’s background and expertise.

Vendor Contracts: What the Fine Print Says

Every major AI provider — OpenAI, Google, Microsoft, and Anthropic — disclaims liability for the accuracy of AI outputs in their terms of service. Understanding what those disclaimers say, and where they have limits, is essential for any business deploying these tools.

OpenAI’s business terms state that services are provided “as is” with no warranties, express or implied. The company does not warrant that services will be accurate or error-free. Users accept that “any use of outputs from our service is at your sole risk” and that they “will not rely on output as a sole source of truth or factual information, or as a substitute for professional advice.” For consumer users, OpenAI’s aggregate liability is capped at the greater of amounts paid in the prior 12 months or $100. For enterprise customers, the cap is the total amount paid during the prior 12 months.

Microsoft and Google publish materially similar disclaimers for their AI products. The pattern is consistent: accuracy is not warranted, consequential damages are excluded, and damage caps are low relative to potential business losses.

Are these disclaimers enforceable? Generally, yes — courts have long enforced limitation of liability clauses in commercial contracts, particularly between businesses. But there are important limits. Unconscionability doctrine can void clauses that are both procedurally and substantively unreasonable. Public policy exceptions exist, particularly where the disclaimer would insulate a party from liability for conduct that rises to the level of gross negligence or fraud. Some state consumer protection statutes limit disclaimer enforceability in contracts with individual consumers.

For business owners, the practical upshot is this: you likely cannot sue OpenAI or Microsoft for $2 million in business losses caused by an AI hallucination. The contract terms will be enforced. Your claim — if you have one — runs against your employees who failed to verify the output, your outside counsel who submitted fabricated research, your accountant who relied on AI-generated tax analysis, or your vendor who built a client-facing chatbot on top of a raw AI API without adequate safeguards.

Professional Liability: The Stakes for Service Businesses

For lawyers, physicians, accountants, financial advisers, and other licensed professionals, AI hallucination exposure does not run through vendor contracts. It runs through professional liability and malpractice standards that predate AI by decades and adapt readily to it.

The American Bar Association addressed this directly in Formal Opinion 512, issued July 29, 2024. This was the ABA’s first comprehensive ethics guidance on lawyers’ use of generative AI tools. The opinion addresses six primary duties — competence, supervision, confidentiality, communication, fees, and candor — and makes clear that a lawyer’s uncritical reliance on AI output without “an appropriate degree of independent verification or review” can constitute a violation of the duty of competence under Model Rule 1.1. The opinion does not prohibit AI use; it requires that lawyers understand the tool’s limitations and take responsibility for the output.

The consequence of submitting AI-generated material to a court without verification is no longer just sanctions. Bar discipline proceedings have followed in multiple jurisdictions. Insurance underwriters writing lawyers’ professional liability policies are now issuing AI-specific questionnaires during renewals, asking whether firms have written AI use policies, whether verification protocols exist, and what tools are covered under existing technology errors and omissions coverage.

For physicians, the situation is similar. The American Medical Association has stated that clinicians should independently verify AI-generated clinical content, particularly diagnostic recommendations and treatment plans. State liability law generally holds physicians to the standard of care of a reasonably competent physician in the same specialty. If AI-generated clinical guidance falls below that standard, the physician who acted on it without checking bears liability — not the AI vendor.

Financial advisers operating under SEC oversight face additional regulatory layers. The SEC’s 2025 AI Task Force and Marketing Rule enforcement actions have specifically targeted advisers who claimed AI-driven processes their systems did not actually use. Advisers who rely on AI-generated research in making investment recommendations without verification face potential violations of their fiduciary duties as well as securities regulations.

Which Business Contexts Carry the Highest Risk

Not all AI use carries equal legal risk. The following contexts present the sharpest exposure for small and mid-sized businesses.

Legal Drafting and Research. Contracts drafted with AI assistance that contain fabricated citations, incorrect legal standards, or invented regulatory requirements create direct legal exposure for the business and malpractice exposure for any attorney who reviewed the work. The verification burden here is non-trivial and cannot be delegated back to the AI.

Medical and Clinical Recommendations. Any AI system touching diagnosis, treatment recommendations, drug interactions, or clinical documentation operates in a context where errors can cause physical harm. The combination of patient harm, regulatory oversight, and the existing malpractice framework makes this the highest-stakes category.

Financial Advice and Investment Analysis. AI-generated financial projections, tax advice, or investment research that is passed to clients without adequate review creates both malpractice and securities regulatory exposure. The SEC’s scrutiny of AI-related disclosures makes this a compliance risk as well as a liability risk.

Compliance and Regulatory Guidance. Businesses that use AI to research regulatory requirements — OSHA, FDA, FTC, state licensing — and act on hallucinated compliance standards face regulatory fines and potential enforcement actions. An AI tool that confidently describes a compliance safe harbor that does not exist puts the business in the position of having relied on advice that cannot be attributed to any responsible party.

Contract Analysis. AI tools used to review contracts for missing provisions, red flags, or industry-standard terms have been documented to miss material clauses or, worse, to report the presence of protections that do not exist. Acting on that analysis in negotiations creates real transaction risk.

What to Negotiate in AI Vendor Contracts

If your business is deploying AI tools through enterprise agreements — rather than consumer-grade subscriptions — you have more negotiating room than you might expect. Here is what to focus on.

The liability cap is the first priority. Standard terms cap vendor liability at fees paid in the preceding 12 months. For a business paying $10,000 per year for an AI tool and suffering $500,000 in losses from a hallucination-induced error, that cap leaves you nearly fully exposed. Negotiate for a meaningful cap — ideally keyed to the potential harm your use case could cause rather than the subscription price.

Accuracy and quality commitments matter. Some enterprise agreements can be negotiated to include output quality standards or commitments to maintain hallucination rates below specified thresholds for defined use cases. These commitments give you contractual recourse that you would otherwise lack.

Indemnification for third-party claims is critical if you are deploying AI tools to serve your own customers. You want the vendor to indemnify you for claims brought by your customers based on AI-generated output in defined circumstances — at minimum, where the hallucination arises from a known defect in the model.

Finally, audit rights and transparency. Enterprise contracts should give you access to information about the model version you are using, known defect disclosures, and notice of material changes that could affect output quality.

Practical Steps to Reduce Your Exposure

The law in this area is still developing, but the principles governing reasonable professional behavior are already clear. Courts and regulators are not asking businesses to stop using AI. They are asking businesses to supervise it.

Adopt a written AI use policy. Every business deploying AI tools — particularly in client-facing or decision-making workflows — should have a written policy governing acceptable use, verification requirements, and documentation obligations. This policy serves two purposes: it reduces risk by standardizing behavior, and it serves as evidence of reasonable care if litigation arises.

Require verification for consequential outputs. Not every AI-generated output needs the same level of scrutiny. But any output that will be presented to a client, submitted to a court, relied upon for a regulatory compliance decision, or incorporated into a contract should be independently verified by a qualified human before use. The standard is not zero tolerance for AI — it is meaningful human oversight of consequential decisions.

Document the verification process. When your team uses AI to draft or research and then verifies the output, document that process. Record what was checked, by whom, and when. This documentation creates a contemporaneous record of reasonable care that is valuable in litigation or regulatory proceedings.

Train your team. AI literacy is no longer optional for employees who use these tools. They need to understand what hallucinations are, why they occur, and what verification looks like in your specific business context. The Australian court’s observation applies equally in the United States: ignorance of a tool’s known limitations is not a defense.

Review your professional liability coverage. Insurance carriers have begun asking about AI use in professional liability renewals. Review your policy to understand whether AI-related claims are covered, whether consumer-grade tools create coverage gaps, and what documentation your carrier expects. If you have gaps, address them before a claim arises.

Conclusion

AI hallucinations are not a technical curiosity. They are a documented, quantifiable, foreseeable failure mode of current AI systems — and the legal system has already begun holding businesses and professionals accountable for the harm they cause.

The lessons from Mata v. Avianca, from the Air Canada chatbot case, and from the growing docket of AI sanctions proceedings are consistent: the person or business who acts on AI output without verification owns the consequences. Vendors disclaim accuracy. Courts will not excuse reliance on fabricated authority. Malpractice insurers are watching. Regulators are developing enforcement frameworks.

The businesses that will navigate this period successfully are the ones that treat AI as a powerful but fallible tool — one that requires human oversight, written governance, and professional accountability. The businesses that treat AI as infallible and outsource their judgment to it entirely are the ones that will be writing checks to opposing counsel.

Use the tools. Supervise them carefully. Verify before you act.

This post is for general educational purposes and does not constitute legal advice. If you have questions about AI liability exposure specific to your business, contact a qualified attorney.