Should You Do Your Own Jury Research? Deliberations Continue.

By Kristin Casler, featuring Dr. Christina Marinakis of Litigation Insights and Dr. David Ball of Malekpour & Ball

Some might argue that it is legal malpractice not to conduct jury research before heading to trial. These days, many clients expect it. But, what if you have no budget allowing for a consultant? Let’s face it, consultants who are any good cost good money—and they are often worth every penny. It is only natural, then, that companies and law firms involved in litigation may try to save bucks by conducting their own jury research without the aid of a litigation consulting firm. Unfortunately, DIY jury research can be fraught with risk—risk of bias, obtaining an unrepresentative jury pool, and worst of all, improperly analyzing the data. That’s why many experts argue that it is better to conduct no jury research than to do it poorly.

“Attorneys who conduct their own research risk making key strategic decisions on results that may not be methodologically sound,” said Dr. Christina Marinakis, director of jury research at Litigation Insights. “That is, there are reasons why attorneys should focus on legal advocacy and leave the research to the social science researchers. Attorney-conducted research can produce faulty or false-positive results, despite the attorneys’ best efforts and their experiences with jurors.”

Some Skills Can Be Taught

Dr. David Ball of Malekpour & Ball is a trial consultant who long ago realized that attorneys will do jury research themselves. Ball and his partner, Artemis Malekpour, along with colleague and Atlanta attorney Don Keenan, teach attorneys how to prepare, conduct and analyze focus groups properly from start to finish. Their teachings are found in books, demonstration videos, consulting sessions, live seminars and workshops.

“Well-meaning attorneys do focus groups when they can’t afford a consultant,” Ball said. “So the only thing to discuss is not whether they should do them, but how to do them right. That has not made me the most popular human being among trial consultants who worry they’ll lose business to the do-it-yourselfers.”

The truth, Ball says, is that too many consultants don’t do focus groups well either. “I see a ton of stuff they have done—it’s utter junk and misleading. We teach lawyers to do their own focus groups better than many—maybe even most—trial consultants.”

When attorneys do research on their own, they learn more and do better in trial, he said. As they do better in trial, they start getting bigger cases, and that’s when they’ll hire the experienced consultants, because they’ve learned how valuable they are.  But the risk of hiring a trial consultant who doesn’t know what s/he’s doing is actually greater than the risk of doing focus groups on your own—when you’ve been taught how. Just saying, ‘Never do them on your own,’ means that only the well-heeled clients, such as insurance companies, can benefit from them. It’s been our job to eliminate that unnecessary and unfair imbalance.”

The Risk: Introducing Bias—Drinking Your Own Whiskey

“One of the biggest problems with attorney-conducted research is bias,” Marinakis said. “Lawyers, by design, are zealous advocates for their clients. Investing so much time and energy into a case inevitably breeds bias when presenting the case, eliciting juror feedback and interpreting the results,” she said.

“When preparing for research projects, we continually have to play devil’s advocate and remind our defense clients to incorporate the pro-plaintiff facts and arguments, or to make sure to include plaintiff-friendly jury instructions or verdict questions,” Marinakis said.

Litigation consultants tame this bias. Their neutrality is particularly important when an attorney conducts a focus group or moderated discussion. “As humans, we often don’t realize the verbal and nonverbal cues that we produce when speaking, and an attorney will undoubtedly pose questions, produce inflections, and make facial and body reactions to juror responses that will reveal which side he or she favors,” Marinakis said. Jurors take their cues from this, molding their responses to what they think the moderator wants to hear. She said this can compromise not only the confidentiality of the research, but also the results. 

“Bias also emerges when interpreting the results,” she said. It’s easy for an attorney to highlight the positive findings and minimize or try to explain away the negative. A neutral consultant is better situated to evaluate the case based on experience and juror feedback. There is no substitute for an experienced jury consultant who has spoken to thousands of jurors and participated in dozens of diverse trials each year. “They are better suited to spot unusual findings, extrapolate to actual trial results, and compare your case and research results to other cases that were successes or failures at trial,” Marinakis said. 

Ball says he and his colleagues teach attorneys how to easily remove the biases and how to focus on the bad news. Ball agrees that do-it-yourself is a bad option—unless you are taught how to do it right.

The Risk: Juror Recruitment

Years of research and experience have shown that several people can hear the same exact facts and arguments, yet come to very different opinions and conclusions. Often, these differences in perception are due to the jurors’ diversity of backgrounds, experiences and beliefs.

“Attorneys who conduct their own research usually don’t have access to the diversity of people needed to get accurate feedback about their case,” Marinakis said. “They often use their friends, relatives and co-workers—or referrals from their friends, relatives and co-workers—as mock jurors in the research.”

There are two fundamental problems with this, she said. “First, these individuals, no matter how helpful they are trying to be, often filter responses in attempt to avoid upsetting the lawyers who recruited them, or subconsciously tell them what they think they want to hear. Second, these people are likely to be more educated, more affluent, more conservative and more knowledgeable about the legal process than the typical jury pool. This is particularly problematic given the correlation between all of those characteristics and a defense leaning. Using employees of a law firm—or even their friends and relatives—is likely to provide more optimistic feedback about your case than is warranted.”

Off-the-street juror recruitment often provides a panel no more representative of the jury pool, Marinakis said. “Usually, these recruits are unemployed or employed in hourly positions that are usually excused from jury service. And basing recruit quotas on census data does not accurately describe the people who show up for jury duty.” For instance, census data includes undocumented workers, people without drivers’ licenses or voter registrations, and people who just don’t show up for jury duty or rarely, if ever, are selected to serve.

“Jury consulting firms have the benefit of seeing dozens of jury pools in many jurisdictions, and in some instances, have records of the jurors who showed up for jury duty and were selected to serve,” she said. “Jury consultants also have the experience to collaborate with local counsel in determining what jury pools in your trial venue actually look like, and we are familiar with the scientifically sound methodologies for contacting these people, screening them and obtaining their consent to participate in the research.” 

Ball points out that he and his colleagues carefully teach lawyers how to recruit properly.

The Risk: Facility, Equipment and Logistics

Attorneys usually lack the means to conduct the research optimally. Litigation consultants have access to research facilities with two-way mirrors that allow attorneys and clients to unobtrusively observe the research presentations, focus groups or mock deliberations. “We often can obtain them at a reduced rate because of our repeat work, and we have preferred audio-visual teams that have appropriate equipment for filming the research and allowing observers to view and listen to simultaneous focus groups or deliberations in real time,” Marinakis said. Attorneys who rent this kind of equipment are often disappointed in the cost and quality of the work. Attempting to conduct research at the law firm creates a bias in how jurors will respond, not to mention confidentiality concerns (jurors know who is sponsoring the research, how to contact the attorneys involved and—with a quick Google™ search—can determine who is actually representing your opponent and how to contact them).

Further, attorneys who conduct their own research are often unprepared for the logistical problems that sometimes arise, and poor management of these problems can easily disrupt the research. 

Ball said his video demonstrates all the logistics. He especially counsels against one-way mirrors because they make jurors far more self-conscious and aware they’re being observed than do cameras right in the corner of the room, and that distorts their discussions and gives misleading results.

The Risk: Incorrectly Analyzing the Results

“Even if the DIY research goes smoothly and yields a wealth of information, attorneys often don’t have the means or know-how to cull that information into meaningful results,” Marinakis said.

Ball agrees that analysis is the most critical part. He often gets involved in a case at this stage and examines the focus group videos and data. For lawyers who cannot afford this kind of consulting service, Ball and his colleagues teach them to do their own analysis. “Obviously,” Ball says, “one of the good consultants out there might do a better analysis job. But it’s better for us to teach how to analyze so that all attorneys can benefit from focus groups, even when they can’t afford a good consultant.

“Almost every attorney can learn how to prepare, conduct and analyze a useful session,” he said. “And they can learn analysis. You’ll learn what jurors think about the case, what they are going to find most important, what points they’ll argue about, can change their minds about, are unlikely to change their minds about, what kinds of people will see the case in different ways, and even give words and analogies you can use in trial.”

Marinakis noted that statistical software to analyze the data can run in the thousands of dollars for a single subscription, and few attorneys have been trained to use it. “It can be very dangerous to base juror distinguishers or profiles—one of the most useful outcomes of jury research—on results that are not statistically valid or reliable,” she said. “Trained consultants also know how to filter for response bias; confirmation bias, order and commitment effects, and anchoring that impede reliability.

“If an attorney cannot explain these concepts, chances are the research will be fraught with design defects that render the results practically meaningless—or worse—misleading.”

What Is an Attorney or Company To Do?

“While some people may argue that any feedback from mock jurors is better than no feedback, we respectfully disagree, as the danger of misleading results is too great,” Marinakis said. That doesn’t mean, though, that attorneys on a tight budget have no options.

Marinakis said there are other ways to save costs that won’t compromise the integrity of your research. “For example, consider asking for an oral strategy session following the research, as opposed to a written analysis, or limit the scope of the research to the most important issues. A smaller research project can elicit helpful feedback, so long as a consultant can explain the caveats and limitations of such research,” she said.

For Ball, the main option is to learn how to do them properly on your own. Otherwise you are walking into trial blind. He has seen from experience how many attorneys manage to do useful sessions on their own, and almost always with case-altering results.

And there is a happy medium: You can learn to prepare and conduct the session, and then enlist a consultant to do the analysis. You can hire someone to properly recruit. You can use a facilitator who will do an excellent job for much less money, such as a school teacher. You can get free space at libraries, and even rent the video equipment at low cost. Ball insists that this kind of research is too important to say, “’Well, we can’t afford the high-priced spread so we’ll have to go hungry.’ Learn how to avoid the dangers, do the best you can, and then when the right case arises go ahead and hire a consultant.

“Plaintiff’s attorneys can also work with a non-recourse lender who will advance the money for a consultant. You don’t pay back the loan unless you do well in trial. This allows lawyers to have trial consultants even when they don’t have the money up front.”

Bottom line—if you conduct your own jury research, make sure you are well informed. Check out alternative funding sources for your research, and involve an experienced and reputable consultant for as much of the work as your budget allows.