Not a Lexis+ subscriber? Try it out for free.


Norton Rose Fulbright: Rise Of Predictive Coding For Pharmaceutical Companies

By Jessica Sykora |

Pharmaceutical and medical device companies in the midst of costly e-discovery battles and high-dollar litigation have more than likely heard the buzzwords "predictive coding" and "computer-assisted review" at some point during strategy sessions over how to sort through the overwhelming mountains of e-data maintained in relation to a particular drug product.

Given the ever-rising cost of e-discovery in today's age, pharmaceutical companies are looking for ways to reduce the time and money spent on reviewing internal documents for production. Predictive coding, one of the many tools in the technology-assisted review toolbox, has been promoted as an answer — a mechanism to achieve substantial cost savings for parties dealing with large volumes of litigation data.

But despite increased judicial acceptance of this technology, the question still remains: Is predictive coding the appropriate e-discovery tool for pharmaceutical and medical device clients faced with potentially complex and costly e-discovery battles in the future? And, if so, which predictive coding approach will best serve the client's e-discovery needs?

How is predictive coding different?

In the beginning, the "traditional" method of document review was manual and involved first year litigation associates sitting in a large conference room sorting through hundreds of thousands of pages of paper for relevance, privilege and "key" and "hot" documents. Over the years, the papers were scanned into a computer and the manual review continued electronically. Lawyers then turned to keyword searches in an attempt to narrow down the set of documents into a manageable number. But, key word searches had their own problems. Thanks to the low cost of computer memory, the number of documents maintained by the pharmaceutical and medical device companies continued to soar, and the costs continued to rise. While limiting the number of custodians might reduce the scope of review to a certain extent, a keyword search could still result in a huge volume of data that would need to be manually reviewed for relevance and privilege. And, unless both sides reached an agreement, choosing the keywords necessary to cull through the data could be equivalent to playing "Go Fish."

Not surprisingly, as the cost of discovery continues to rise, so does the desire for pharmaceutical and medical device companies to rein in these discovery costs. Predictive coding is now a cost-effective alternative to sorting through an ever-increasing number of corporate documents. It can cost far less than traditional key word searching or linear manual review and represents a way for pharmaceutical and medical device companies to reduce the mounting legal fees typically associated with a manual review of e-discovery.

The predictive coding process combines human review with computer-assisted key word searching in order to "train" document review software to gather potentially responsive documents within a quantified document universe. Instead of manually reviewing all documents or all documents that contain key words, in most cases, the producing party only reviews a small percentage of the documents.

Once the large universe of documents has been gathered from identified custodians, the producing party will pull a small random sample set to be reviewed and appropriately coded by a senior lawyer with a deep understanding of the case. The documents coded as relevant are referred to as the "seed" set. An experienced reviewer is critical to ensuring the accuracy of the seed set. The computer will then analyze the human reviewer's coding and find more documents similar to those coded as relevant, sorting the relevant documents from the irrelevant documents.

By updating the seed set as more relevant documents are found and continuing to train the system, the computer will become increasingly accurate in predicting how a human reviewer would have coded the much larger document universe. The technology will also rank the documents by responsiveness, such that "key" or "hot" documents (whether good or bad) are pulled and prioritized quickly for lawyer review. This prioritization allows lawyers to engage in case assessments at an earlier stage of the litigation.

Important considerations

After the predictive coding technology has prioritized and sorted the documents into a relevant and irrelevant set, the pharmaceutical and medical device companies can adopt one of three approaches:

  • ·         Approach 1: Review all computer-suggested relevant and nonrelevant documents, but in a prioritized fashion;
  • ·         Approach 2: Review all computer-suggested relevant documents in a prioritized fashion and merely sample and perform quality control checks on the computer-suggested nonrelevant documents; or
  • ·         Approach 3: Perform sampling and quality control checks on both the computer-suggested relevant and nonrelevant documents.

Each approach comes with its own independent considerations.

Approach 1 clearly provides the pharmaceutical and medical device companies with the most comprehensive analysis, as all documents — those designated relevant and nonrelevant — are individually reviewed. However, this manual follow-up review may defeat most, if not all, of the cost savings or efficiencies that the predictive coding technology provides. But, the benefit of Approach 1 is that, while the costs may still be extremely high, the predictive coding technology allows for the review to be in a prioritized fashion. This speeds up the review process and provides early information on "hot" or "key" documents to the case team.

Pharmaceutical and medical device companies with highly sensitive product information or case-damaging privileged communications may opt for Approach 1, as they may be willing to stomach the enormous — but slightly reduced — cost of a document-by-document review in hopes that these harmful documents will never surface. The fear of producing a marketing piece or new drug application submission from an irrelevant product may be minute in comparison to the risk of producing an email detailing the chemical composition of a prescription medication or a privileged communication spelling out a valuable litigation tactic. While "snap back" provisions can always be adopted, some clients may believe that "un-ringing the bell" is too great a risk.

Approach 3, on the other hand, provides the most cost savings. While the allure of a significantly reduced e-discovery price tag may draw many pharmaceutical companies toward adoption of Approach 3, there are additional factors that must be considered before fully committing to this process. Approach 3 allows a computer to predict which documents would likely be coded responsive by a human reviewer. Unlike with the other two approaches, only very small samplings of the documents are reviewed by human eyes as a quality control process. As such, while Approach 3 provides the most significant cost savings, the tradeoff is that this approach comes with the most risk, as the computer makes the majority of the decisions relating to relevance versus irrelevance, potentially leaving a much wider margin of error.

Also, if the predictive coding process is poorly designed or implemented, resulting in improper coding of the documents, plaintiffs may demand that the production be redone using more traditional methodologies. This could end up being more expensive and ineffective than having no technology at all. Having to reproduce an already finalized set of documents would defeat any cost savings that the pharmaceutical company would hope to achieve by using a predictive coding approach. Therefore, unless agreed to by opposing counsel, Approach 3 should only be reserved for very low risk litigation where the costs of discovery are extremely high.

Accordingly, many pharmaceutical and medical device companies find themselves somewhere in the middle, deciding that Approach 2 provides the best solution for their e-discovery needs. With Approach 2, all computer-suggested relevant documents are reviewed by human eyes before production. However, the significant cost savings are generated because only a small sampling of the nonrelevant documents are reviewed, providing a quality control check on the process. Approach 2 thus balances the need for cost savings with the company's desire to know what is contained in the relevant documents and have some control on what is produced to the other side.

Judicial acceptance

Initially, the fear with use of predictive coding was that courts would not approve the strategy as a way for companies to review and produce e-discovery. However, on Feb. 24, 2012, Magistrate Judge Andrew J. Peck of the U.S. District Court for the Southern District of New York assuaged these fears and issued the first opinion providing judicial acceptance of predictive coding in the litigation context. Moore v. Publicis Groupe & MSL Group, 11 Civ. 1279 (ALC) (AJP) (S.D.N.Y. Feb. 24, 2012) [enhanced opinion available to subscribers].

Judge Peck concluded that "computer-assisted review is an available tool and should be seriously considered for use in large-data-volume cases where it may save the producing party, or both parties, significant amounts of legal fees in document review." With this stamp of judicial approval, the adoption of predictive coding will more than likely spill over into the pharmaceutical and medical device context in the near future, opening the door for broader acceptance of predictive coding in this industry.

Courts that have approved a predictive-coding approach stress transparency and cooperation among counsel in order to reduce fears about the so-called "black box" of technology. In Moore, Judge Peck stated that "[e]lectronic discovery requires cooperation between opposing counsel and transparency in all aspects of preservation and production" and that counsel on both sides must work together to ensure that the "proposed methodology [is] quality control tested to assure accuracy in retrieval and elimination of 'false positives.'" Furthermore, Magistrate Judge Nolan in Kleen Prods LLC v. Packaging Corp of Am., No. 10 C 5711 (N.D. Ill. Sept. 28, 2012) [enhanced opinion] recognized that there must be a "paradigm shift" in how parties approach the discovery process in furtherance of collaboration and cooperation.

Thus, it appears that transparency and involving opposing counsel in strategy decisions as to how the predictive coding process will be structured is an important consideration in the minds of the courts in approving this process as an acceptable method of review. An early meet and confer with opposing counsel in an attempt to achieve an agreement on production cutoff dates, custodians and specified categories of relevant versus irrelevant document categories may make it more likely that the predictive coding strategy will be judicially accepted.

The transparency and required negotiations up front will also narrow the scope of documents to be collected and reviewed, as well as reduce the potential for discovery fights down the road after the document collection and review is completed. Furthermore, a follow-up manual review as contemplated in Approaches 1 and 2 above will also make it more likely that the courts will approve this type of computer-assisted technology.


Predictive coding is fast becoming integral to the litigation practice. No document review mechanism guarantees perfection; however, the adoption of predictive coding has the potential to drastically alter the way in which documents are reviewed and produced in complex pharmaceutical and medical device litigation.

Proponents of this e-discovery tool promise that it will allow for a more efficient, cost-effective and speedy review of large volumes of documents. By implementing defensible predictive coding technology in the appropriate case, pharmaceutical clients may be able to undertake a production of e-discovery in a cost-effective manner without the headache often associated with this necessary evil.

For more information about LexisNexis products and solutions, connect with us through our corporate site.