When Legal AI Gets It Wrong (Again!) and Why Trust Matters More Than Ever

By Serena Wellen, Vice President of Product Management at LexisNexis Legal and Professional 

May 22, 2025 

A new article from Business Insider has brought yet another legal AI misstep into the spotlight. This time, counsel for a defendant in a high-profile federal case used a combination of generative AI tools, including ChatGPT and Westlaw's AI features, to assist in drafting a legal brief  — but the result was riddled with serious errors. 

U.S. District Judge Kathryn Kimball Mizelle called out the submission for containing “significant misrepresentations and misquotations,” citing non-existent quotes and miscited legal propositions from appellate rulings and even a congressional report. The judge demanded an explanation — and what followed was another troubling acknowledgment that generative AI tools, including Westlaw’s AI, were part of the drafting process. 

Once again, we’re reminded: generative AI is only as reliable as the systems it’s built on. 

In law, there’s no margin for fabricated citations, invented quotes, or sloppy sourcing. And yet, the tools meant to increase efficiency are now putting real legal professionals — and their clients — at risk. 

The Tools Aren’t the Problem. The Foundation Is. 

Generative AI has incredible potential to reshape the legal profession but only when built on trusted foundations.  

When legal professionals turn to general-purpose AI or platforms that blend open-web data with unclear sourcing methods, the consequences are predictable: 

• Fake legal citations that don’t exist in any court record 

• Misapplied quotes from unrelated rulings 

• Briefs that undermine credibility and expose firms to court sanctions 

The issue isn’t the idea of AI in law. The issue is how that AI is trained, sourced, and applied. 

Built to Protect Attorneys — Not Expose Them 

What happened in the Burke case, and other recent courtroom embarrassments, illustrates what can go wrong when legal professionals rely on consumer-grade AI or tools with opaque sourcing and limited legal grounding. 

At LexisNexis, we didn’t adapt a consumer chatbot and call it “legal AI.” We built Lexis+ AI from the ground up to meet the demands of real legal practice — where trust, reliability, and transparency are non-negotiable. 

Protégé in Lexis+ AI: Built for the Realities of Legal Work 

Here’s what that means in practice: 

1. We Don’t Generate from the Open Web 

Protégé in Lexis+ AI is grounded in the most authoritative legal content repository in the world, including Shepard’s®-reviewed case law, statutes, secondary materials, and Practical Guidance content. This gives users not only the legal authority they need to cite, but also step-by-step insights on how to apply that law in real-world scenarios, from drafting provisions to navigating procedural requirements. 

2. Citations That Stand Up in Court 

Where citations include links to source documents, users can instantly verify the underlying legal authority so know where the information comes from, and so will the court. 

3. Responses That Reflect Your Firm’s Voice 

Through DMS connectivity and Protégé Vault, Protégé can integrate a firm’s own trusted internal documents, from past briefs to contracts, to generate context-aware responses grounded in both authoritative legal content and private precedent. 

The Bottom Line: Trust Isn’t Optional 

As we’ve seen, misused AI can lead to real-world legal consequences, from damaged credibility to court sanctions. Lexis+ AI is engineered to prevent those risks by delivering what legal professionals actually need: reliable, transparent, and trustworthy responses grounded in authoritative content. 

In the world of legal AI, not all tools are created equal. Protégé isn’t just another generative tool — it’s a partner in the practice of law. 

Learn More 

Protégé is available now within Lexis® Create+ for full-document legal drafting in Microsoft Word, Lexis+ AI® legal workflow solution, and the Lex Machina® Legal Analytics® platform. Protégé will be seamlessly integrated across the LexisNexis ecosystem and Microsoft products, expertly assisting legal and business professionals wherever they work. 

To learn more about Protégé, please visit www.lexisnexis.com/protege.