Attention Buyer: Not All Legal AI Models Are Created Equal

By: Simon Weierman 

A 2024 LexisNexis survey of managing partners and C-suite leaders at major law firms and Fortune 1000 companies found that nearly all legal executives (90%) expect their investment in Generative Artificial Intelligence (Gen AI) technologies to increase over the next five years. The survey also found that roughly one-half (53%) of Law360® Leaderboard Pulse firms have already purchased Gen AI tools including both general purpose and legal practice specific AI solutions

In addition to the variety of general-purpose AI tools available to businesses and consumers, the past year has also seen the emergence of commercially available Legal AI tools — Gen AI tools tailored specifically for the legal profession. Many legal professionals are trying to figure out how they can take advantage of the opportunities afforded by this new technology while minimizing the risks that are well-documented in the industry.

We recently published “The Definitive Guide to Choosing a Gen AI Legal Research Solution,” a free buyer’s guide that details what law firms and in-house legal departments should look for so they can select the most appropriate solution for their organization. Over the course of the next several weeks, we will be publishing a series of blog posts that are drawn from the five pillars addressed in the buyer’s guide.

We kick off the series today by unpacking the importance of the Gen AI model itself. Before proceeding on the journey to acquire a Legal AI solution, it’s important to understand that not all Gen AI models are created equal.

Strengths and Weaknesses of LLMs

There is no need to become an expert in software engineering in order to be a savvy buyer of a Legal AI solution, but it does help to learn a bit about Large Language Models (LLMs) — a type of AI that are trained on vast amounts of data to mimic human intelligence — and how they operate.

 The fact is that different LLMs have their own strengths and weaknesses. Here are a few key differences between some of the LLMs used to power various Legal AI solutions you will find on the market:

• Architecture — Different underlying “neural network architectures” will impact capabilities, which is why some LLMs are better at tasks such as translation or summarization.
• Size — LLMs can range from millions to trillions of parameters, so larger models are generally more capable while smaller models can be more efficient.
• Training Data — Models trained on legal data will have different strengths than those trained on general purpose text, an important consideration for law firms and in-house legal teams.
• Fine-Tuning — LLMs can be further refined on niche datasets to improve their capabilities in specific domains.
• Public vs. Proprietary — Open source LLMs allow for greater transparency, while proprietary models offer a deeper understanding of the user’s intent and can therefore deliver higher quality responses.

The Multi-Model Approach

There is another strategy available to Gen AI development teams that allows users to benefit from the unique capabilities of each LLM while balancing out each one’s weaknesses: A multi-model approach that draws from more than one LLM in the creation of a new tool, enabling it to produce results that surpass that of any one model.

Questions to Ask About the Gen AI Model

When evaluating a Legal AI solution, here are some questions to pose to the provider about their Gen AI model:

1. Do you use a single model or a multi-model approach for creating your product?
2. What is the average time the AI takes to return an answer?
3. Are there any limitations on the number of prompts you can pose each day?
4. What is the underlying architecture of the AI model and how does that design impact its capabilities to perform legal-specific tasks?
5. Was the Gen AI model trained on legal-specific data or open-source data?
6. Does your AI solution incorporate a retrieval-augmented generation framework to find and link relevant source documents?

Lexis+ AI is our breakthrough Gen AI platform that we believe will transform legal work by providing a suite of legal research, drafting, and summarization tools that delivers on the potential of Gen AI technology.

Lexis+ AI pairs our unsurpassed legal content with breakthrough Gen AI technology in a way that could redefine the way that legal research is conducted and legal work product is created. Its answers are grounded in the world’s largest repository of accurate and exclusive legal content from LexisNexis with industry-leading data security and attention to privacy. Click here to request a free trial.

To download a copy of The Definitive Guide to Choosing a Gen AI Legal Research Solution, please click here.