Lawyers are getting sanctioned for using AI

A lawyer types a question into a chatbot, gets back a confident answer with named cases and citations, and files it. The citations look perfect. They are also completely made up. Weeks later, the lawyer is standing in front of a judge explaining why a brief cited decisions that do not exist.

This is not a hypothetical. It has already happened, and it keeps happening. For legal professionals, public AI tools carry two distinct risks. One is loud and embarrassing. The other is quiet and arguably worse.

The case that put this on the map

As widely reported, in Mata v. Avianca (United States District Court, Southern District of New York, 2023), two attorneys submitted a legal brief containing case citations that ChatGPT had fabricated. When opposing counsel and the court could not find the cases, the lawyers learned the hard way that the tool had invented them.

The judge, P. Kevin Castel, sanctioned the lawyers and ordered a 5,000 US dollar penalty. The dollar figure was small. The reputational damage, reported across the legal press, was not.

Since then, a growing list of similar cases has surfaced in courts across multiple countries. Judges have flagged briefs built on AI-fabricated citations, and some now require lawyers to disclose whether AI was used in preparing a filing. What started as one cautionary tale is now a pattern that courts watch for.

Why the model invents case law

It helps to understand what these tools actually do. A public chatbot is a text predictor. It produces the words that are statistically likely to follow your prompt. It is very good at sounding like a lawyer, because it has read a lot of writing by lawyers.

What it does not do is check whether the cases it names are real. When you ask for authority on a point, the model generates citations that look like the citations it has seen: a plausible case name, a plausible court, a plausible year, a plausible reporter number. The format is right. The content can be fiction.

This behavior is often called hallucination. The word makes it sound like a glitch, but it is closer to the core design. The model is not retrieving facts from a verified source. It is predicting plausible text, and a fake citation can be just as plausible as a real one.

A citation that looks real but does not exist is not a bug you can spot by reading carefully. It is built to read correctly.

That is exactly why these errors slip through. A busy lawyer skims a draft, sees properly formatted authority, and trusts it. The formatting is the trap.

The quieter risk almost nobody talks about

The fabricated citation problem gets headlines because it ends in a sanction. The second risk is harder to see, which is what makes it dangerous.

When a lawyer pastes the facts of a matter into a public AI tool to get a quick summary or a first draft, those facts leave the firm. Names, financial details, the substance of a dispute, the things a client shared in confidence: all of it can go into a system the firm does not control.

This puts two duties under pressure at once:

• Confidentiality, the obligation to protect what a client tells you
• Privilege, the protection that can be weakened once information is shared with an outside party

A fabricated citation can be caught before filing if someone checks. Privileged facts sent to a public model cannot be pulled back. There is no motion to undo it. For many lawyers, this is the risk that should keep them up at night, precisely because it produces no immediate, visible consequence.

How to use AI without these risks

Neither risk means lawyers should avoid AI. It means the tool has to be built for the duties lawyers already carry. The fix has two parts, one for each problem.

For fabricated law, the answer is grounding. Instead of letting a model predict what a citation should look like, you point it at the real body of law and require it to answer from that source. When the model cites something, it cites a document that actually exists, and you can click through to read it.

This is what Corpus (coming soon) is built to do. It grounds answers in the real law, so the model cites the source instead of guessing. The difference is structural, not cosmetic. A grounded system can show you where an answer came from. A predictor cannot, because there is no source behind a guess.

For confidentiality and privilege, the answer is masking. Before anything reaches a model, client identities and identifying details are stripped out and replaced. The model can still do useful work on the shape of the problem, but the people and specifics that make the matter confidential never leave in a readable form.

This is what Velum does. It masks client identities before a prompt reaches a model, so a lawyer can get the benefit of AI without handing over the things they are sworn to protect.

Used together, the two pieces close both gaps. One keeps the answer honest. The other keeps the client protected. The lawyer stays in control of both.

The takeaway for legal professionals

The Mata v. Avianca sanction was a warning, and the cases that followed it confirmed the warning was real. The lesson is not that AI is unusable in legal work. It is that a general-purpose chatbot was never built for a profession with duties this specific.

A tool that guesses at case law will eventually guess wrong in front of a judge. A tool that swallows whatever you paste into it will eventually hold something it should never have seen. The way forward is AI that is grounded in real law and that protects the client by default.

Start with the client-protection side. See how Velum masks client identities before anything reaches a model, and see the use cases for how legal teams put it to work.