Not a Lexis+ subscriber? Try it out for free.


E-Discovery: We Still Haven’t Found What We’re Looking For


By Steven C. Bennett[1] |

 E-discovery, like the computer and telecommunications economy from which it derives, has come a long way.  Today, more than ever, in large case and small, “e-discovery IS discovery.”  Yet, despite more than 20 years of development of the commercial internet (and its attendant technologies, including email, instant messaging, text messaging, social media and more), the “golden era” of e-discovery, in which practitioners, clients and judges all agree on best, most efficient practices, and effective technology solutions are universally available, still eludes us.  Why?  This Article looks briefly at the history of computing, and compares the recent history of e-discovery.  The Article suggests that the fault lies not in lack of creative (indeed, epic) efforts to solve e-discovery problems, but rather in the fundamental fact that there is no “one size fits all” form of e-discovery project, or e-discovery management.  The Article concludes with some suggestions for how to promote further improvements in the efficiency of e-discovery systems. 

The “Big Picture” History Of Computing 

From the abacus (ancient) through Charles Babbage and his “difference” engine (1820s), through tabulating machines and IBM (turn of the last century), early “computers” were mechanical devices.[2]  The electronic period represents the great leap forward, from vacuum tubes in the Turing system (1940s), through transistors and integrated circuits (1950s and 1960s) and eventually to “chips” and the microprocessor revolution (1970s and beyond), together with ancillary technologies, which gave birth to the modern pervasive computing and telecommunications environment.  Consistently, throughout the modern era (now, more than half a century), the industry has sought standardization of essential elements, even as specific hardware, software and applications have varied greatly.  Standard computer programming languages, such as FORTRAN, BASIC, COBOL and C++, and standard systems for information exchange, such as ASCII and TCP/IP, make it possible for computer professionals to share information and coordinate operations, even as many companies compete (vigorously) in the market for computer and telecommunications services.  Today, moreover, many of the essential elements of computer and telecommunications systems are free (such as the LINUX operating system) or shared (such as the world wide web).

History Of E-Discovery

E-discovery, in rudimentary form, existed before commercialization of the internet.  The Advisory Committee on the Federal Rules included a note, in 1970, indicating that the term “documents” (in Rule 34) included electronic “data compilations.”  But large electronic database compilations are hardly the same as the pervasive sources of modern communications (originally, email, but now email and much more).  The development of the commercial internet (in the 1990s) marks the real start of modern e-discovery.  Courts and practitioners struggled, as volumes of information grew at unprecedented rates.  A brief period, of “don’t ask; don’t tell” ensued, wherein counsel often chose not to ask their opponents for electronic information, so long as the opponent did not do so.  But counsel (especially plaintiffs’ counsel in securities, consumer and other mass action cases) soon learned that the electronic form (especially email) was where “the good stuff” could be found.  Counsel also learned that, with just a few pieces of paper, substantial requests for information could be served, with potentially significant consequences for an action.  The burden of responding to such requests; the risk of a “smoking gun” document in the mass of information produced; the uncertainty of whether the producing party had done enough to preserve, search and produce responsive information—these kinds of concerns could affect the settlement calculus in a case.  Especially where “unbalanced” e-discovery obligations appeared (one side having very little information, and the other side holding vast troves), there might be little down-side to demanding a great deal of electronic information.

Meanwhile, an entire e-discovery industry grew up, seemingly overnight, to provide support to law firms and their clients in the burgeoning field of e-discovery.  The industry drew on an array of sources: basic services (like photocopying); investigation and analytic services; litigation support and case management services; and others.  All these services existed before e-discovery became big business.  But the volume (and profitability) of e-discovery work brought out hundreds of new firms, and created an industry that today does billions of dollars of business annually.   The industry now has all the trappings of a major business sector: tradeshows; publications dedicated to e-discovery technology and related issues; education conferences (often sponsored by vendors); and more.

What the industry has not created (as yet) is much in the way of standardized technologies, or even wholly-standardized processes for conducting e-discovery.  There is no single major (let alone dominant) market leader.  And there appears to be far less shared (or at least inter-operable) use of basic systems than appears in the computer industry as a whole.  Why?

The E-Discovery Difference

No one could expect the e-discovery industry to duplicate the history of the computer industry, exactly.  To a large extent, however, the e-discovery industry exists within the computer industry, and is affected by many of the same forces as affect the larger industry.  The differences (outlined below) do not necessarily preclude standardization of e-discovery.  Rather, they suggest that the special circumstances of e-discovery require consideration of the limits, and best methods for achieving, standardization and inter-operability.

Unlike the computer industry, e-discovery involves much more than just technology.  The impetus for e-discovery projects is the legal obligation to preserve, search and produce documents in connection with litigation (and other legal proceedings, such as government investigations, and regulatory action).  For a long time, litigation lawyers (principally law firm lawyers) supervised and controlled much of the process of e-discovery, because they were the chief intersection with opposing counsel and the court (from whom discovery requests and orders derived).  These lawyers typically had no technical (let alone computer science) background.  In-house lawyers, moreover (their clients), typically had the same pedigree, and thus were not in a position to evaluate the technology solutions proffered by counsel.  As a result, technology solutions in e-discovery might depend on the law firm’s preferred approach (or vendor ability to develop close relations with the law firm).  Cost control, efficiency and quality results might easily suffer as a result.

That era is fast nearing its end.  Companies have come to realize that they can take control of much of the e-discovery process, to improve the efficiency of the process while decreasing risks to the institution.  One essential element of this control has been development of in-house teams of e-discovery specialists (often attorneys and para-professionals who previously served in outside functions, but also often inter-disciplinary teams, including IT, records and other professionals), with the ability to create repeatable, dependable processes for handling e-discovery demands.  These processes can form the basis for negotiation with adversaries and the courts, essentially in the form of “this is what we can do (in response to discovery requests); and here’s why our process is reasonable.”

Companies, moreover, have come to recognize that good information management and record-keeping functions can greatly enhance their ability to respond to e-discovery obligations.  Elimination of unnecessary records, data-mapping and technological solutions (such as automated “tagging” of records) can reduce the burdens associated with these obligations.  Companies now understand that the “pay me now; pay me later” principle is at work in the e-discovery field.  Especially where a company’s other obligations (public disclosure, HR, data security and privacy protection, to name a few) require revision of information management practices, the benefits of periodic review and upgrading of the company’s e-discovery response capabilities have become increasingly obvious.

At the highest ends of the e-discovery project spectrum (in terms of size and cost), moreover, companies increasingly have embraced technology-based solutions.  Technology-assisted review (TAR) has proved to be the “giant slayer” response, in many instances, to management of giant collections of information, which essentially cannot be reviewed and produced through conventional e-discovery methods.  The challenge, in large measure, has been to convince courts and adversaries that a machine-only review, when developed carefully, through human quality control and open disclosure of the process, can be effective and reliable (indeed, more effective and reliable than a human-only review process).  Several landmark cases have established solid precedent for use of these systems, in appropriate cases.  More generally, companies use more effective systems of information governance (in advance of litigation) to weed out unnecessary data, and to organize their collections, placing them in a better position to respond efficiently when litigation comes.

Daunting technology problems remain.  Chief among these is the heterogeneity of e-discovery projects.  Not only do cases vary, greatly, in terms of the volume of materials, number of custodians, and location and format of data, but the actual technologies involved can vary greatly.  In some instances, large companies have various “legacy” systems in place, coupled with a variety of add-ons for special functions, and more recently acquired systems (in some instances, whole communities of additional systems, as in the case of mergers and acquisitions).  Many systems, moreover, were not designed for the preservation, search and production tasks involved in e-discovery.  Large costs can arise from the need to convert data into standard forms, to be subjected to e-discovery processing.  

These problems may only accelerate, as the computer and telecommunications industries continue to develop more extensive and more complex methods of creating, using and storing information.  Mobile devices, cloud computing, wearable devices, remote office and bring-your-own-device systems (and many more developments) will present additional challenges for e-discovery in years to come.  The volume of data, moreover, remains on a continuous path of growth. 

Meanwhile, the cutthroat competition in the e-discovery industry shows no sign of abating.  The days of re-purposed e-discovery startup services (in the beginning, often a photocopy service that claimed that it could “also do” e-discovery work) are over.  E-discovery service vendors have become ever more sophisticated in their capabilities.  At the high end, some of the providers are near-global behemoths (or at least affiliated with accounting, consulting and investigation services that rank on a global scale).  Many of these large-scale providers have grown through acquisitions (often of specialty services, or regional offices, that help complete their portfolio of resources).  New entrants continue to break into the market, however, and incumbent vendors continue to develop new technologies, and new systems for management, in an effort to lower costs and improve efficiency.  Off-shoring and fragmentation of the market (including highly-specialized vendors that do just a few specific things, but very well) mean that, in addition to competition among the biggest players, no single vendor is so dominant that its technologies and systems can be considered “standard” in the industry.

Promoting Solutions

Over the past 20 years or more, the computer and telecommunications industries have created new and ever-more-powerful systems, and e-discovery professionals have struggled to catch up.  So, will we never find what we’re looking for?  Several of the innovations and trends outlined below suggest that the industry can, and will, move toward ever-more efficient methods of operation.  No “magic wand” may appear, but certain improvements are foreseeable, and certainly to be encouraged.

Measurement:  If is often said that “what gets measured matters.”  In the modern e-discovery era, practitioners and their clients increasingly focus on budgets for e-discovery projects, before, during and after their projects are completed.  Sophisticated corporate clients, in particular, have begun to “keep score” on the results of their e-discovery projects, noting where costs may balloon, and concentrating on methods to find efficiencies in processes (often, by effectively reducing the volume of materials and decreasing the number of human hours required to complete a project).  In addition, some broad efforts at data collection concerning e-discovery (such as the Federal Judicial Center study on costs in litigation),[3] provide some sense of the principal sources of costs in litigation.  The continuing TREC and DESI studies and conferences, moreover, offer solid information on the effectiveness of various e-discovery search techniques (including Technology Assisted Review).[4]  What is missing, however, is detailed information on the actual costs of e-discovery projects, across a spectrum of businesses, including comparisons of alternative methods, aimed at developing a “bang for the buck” sense of which technologies, and especially which management systems, are most efficient, and which most appropriate for various configurations of business operations.

It is possible to conceive of methods of sharing more detailed information on the actual results of e-discovery projects.  Courts might, for example, routinely survey parties and their counsel, at the conclusion of every litigation, regarding basic facts about the case (type of case; amount at issue) and results (probably in rough ranges) in terms of the expenditures on e-discovery in the case.  Further details (length of the discovery period; volume of data and number of custodians) might also provide some insights.  The data collection might be spotty (as most parties and counsel have little interest in a case once it concludes), but some useful information could develop.

Data collection might also take the form of more detailed reporting, during the course of ongoing proceedings.  A court might, in certain cases, require that parties produce budgets for e-discovery projects in the case, and require the parties to report on their progress against the budget.  This information might guide the court’s own management of the matter, and could also provide useful guidance for other cases.  This kind of information, however, typically would not provide direct comparisons of software and e-discovery management systems (although there already are a number of sources for information about software features and capabilities).

To date, many of the comparisons between vendors have consisted of surveys of users (chiefly law firms).  Such surveys are useful, for identifying “market leaders,” but they do little to compare actual performance capabilities directly.[5]  Perhaps the most delicate information to develop would be some form of head-to-head competition between e-discovery vendors and systems.  A neutral professional might develop a body of materials, to be processed by competing vendors, and the output measured, in terms of speed, productivity and accuracy of results.[6]  Methodologies for corporations to compare vendor performance, on their own test data sets, might also be useful.  Anonymized summaries of the results of these comparisons might provide guidance to the industry as to what techniques (objectively measured) can provide better results.  Some form of feedback system from actual users of these services (in the nature of restaurant and travel reviews) might also prove useful.

Inter-Disciplinary Collaboration:  In the early days of e-discovery, lawyers (chiefly, outside counsel) often controlled the process (selecting vendors, assigning responsibilities).  Today, companies do a much better job of managing much of the process in-house (establishing internal protocols, selecting vendors and more).  Nevertheless, e-discovery remains a lawyer’s game.  Lawyers (in-house and outside) dominate the actual process of e-discovery, but (perhaps as important) lawyers also dominate discussions shaping the future of e-discovery.  Lawyers also dominate the Continuing Legal Education (CLE( programs on e-discovery, a chief source for exchange of information on e-discovery best practices. 

Large vendors of e-discovery services often sponsor the CLE conferences (and appear at booths in the halls near the conference proceedings).  Trade shows, featuring all manner of e-discovery and litigation support software and technology also tend to be dominated by large vendors.  The vendors, moreover, are often controlled by lawyers (and former lawyers).  Comparisons of systems and practices, other than to tout the “latest and greatest” innovation of the specific vendor, are rare indeed.

Most conspicuously absent from this landscape is any comprehensive system for collaboration between lawyers, their clients, and the vast array of computer science and information management professionals and researchers that operate outside the e-discovery “bubble.”  Some quite useful fora for exchange of information (chiefly, among lawyers, vendors and their clients) exist.[7]  But broad interchanges between affected constituencies and participants in arenas other than law (and e-discovery in particular) generally have not yet occurred.  E-discovery is part of a greater universe of issues: data management, storage and telecommunications; records management; data privacy and data security; compliance and reporting; human resources management; and more.  Developments affecting one of these areas may have significant implications for the others.  And development of “best practices” in one area must take into account capabilities (and limitations) in the others. 

No easy solutions appear, but one can imagine some methods for increased collaboration and information-sharing, across these groups.  Most basically, existing organizations in the various fields should develop methods of liaison, such as cross-organization, co-sponsored conferences and publications.  In particular, there would be great value in having technical representatives from various disciplines discuss commonalities of problems and solutions, outside the competitive atmosphere of sales conventions and booths at CLE meetings.  Technical conferences of this kind regularly occur, for example, in the medical field.  Reporting (ex officio) representatives might also participate in each other’s organizations, to help keep their own members apprised of parallel developments, and look for new means (and topics) for collaboration.

Another promising method to promote liaison of various disciplines might involve law schools and other professional schools sharing resources and information, and co-sponsoring events and publications.  Major universities often have both law schools and a variety of other departments (computer science, electrical engineering, management, labor relations, even psychology) that could contribute to (and host) a cross-disciplinary academic conference.  Distance exchanges between various education institutions might further leverage the input of these many experts.  Publication of relevant proceeding notes and papers from conferences of these types could greatly improve the flow of cross-disciplinary information, and development of awareness of the connections between these subjects.

Education And Market Down-Scaling:  Perhaps the most intractable problem in e-discovery remains the fact that many lawyers (and certainly many clients) do not really understand the e-discovery process well, and do not really want to spend the time to learn the “nuts and bolts” of the process.  More than twenty years have passed since the dawn of the modern e-discovery era (commercial use of email).  We are long past the “don’t ask; don’t tell” phase.  We have passed the watershed event of the 2006 amendments to the Federal Rules of Civil Procedure, and are about to witness implementation of further amendments to the Federal Rules.  Countless CLE, bar association and judicial education programs have highlighted the importance of e-discovery.  And still, most lawyers (including litigators) and their clients remain well outside the “bubble” of e-discovery cognoscenti.  The results include egregious (and often preventable) mistakes (sometimes leading to severe sanctions, and almost always leading to judicial frustration). 

No less significant is the phenomenon of the “drive by” e-discovery meet-and-confer session.  Courts and commentators have implored lawyers to identify (and solve) problems, in a spirit of cooperation.  But lawyers unfamiliar with the process, and unable or unwilling to obtain the necessary information from their clients, commonly fail to engage (risking discovery of problems long after they have mushroomed and often creating burdens and inefficiencies that could easily be avoided).   

To provide effective education and improve the participation of lawyers outside the e-discovery “bubble” in the process of managing their projects more efficiently (for the benefit of their clients, and the courts) may require a shift in focus.  The average lawyer does not need to become an expert in e-discovery arcana (mastering the “latest and greatest” TAR technology, for example).  Lawyers need “meat and potatoes” instruction in the basic “how-to” of e-discovery.  Short courses on the mechanics of preservation, search and production, coupled with hands-on instruction in conducting such things as an effective meet-and-confer session may offer the greatest impact.  In addition, very brief modules, addressing common problems in e-discovery, and offering practical solutions, would offer the “know it when you need to know it” education that most lawyers favor.  These educational materials, moreover, could help paralegals, litigation support assistants, and IT staff (in-house, and at firms) to better understand their functions, and help avoid the inefficiency and mistakes that can come from misconception of the e-discovery process.

Who can provide such instruction?  Much of the information already exists.  The real question is how to most efficiently deliver the necessary instruction.  Traditional all-day, in-seat instruction clearly offers some possibility of reaching target audiences.  But the broader answer almost certainly involves online instruction systems.  Some service vendors already offer “test drive” online experiences (chiefly as part of their marketing efforts).  These kinds of programs could be adapted, by bar and other education groups.  Law schools (and other professional schools), moreover, might partner with vendors and bar groups to develop appropriate content.  Another useful project might create an index of useful web-sites for information on significant (and common) e-discovery problems.  One might even imagine some form of prize for “best of the year” articles, pod-casts and other materials offered on these subjects.

A final challenge concerns the down-scaling of e-discovery technology and practices, to address smaller cases, typically with less money at stake and lower volumes of information (although, in many instances, no less complicated sets of issues).  E-discovery service vendors largely ignore this segment of the market, with trade shows and publications principally focusing on “big firm” lawyers and their clients.   In the process they may leave “money on the table,” as the first truly national “smaller case” vendor could pick up a very significant share of that market.  The comparison to down-scale legal and accounting operations, which enjoy a 30-year track record of development, seems obvious.[8]  Indeed, it is entirely possible that an existing operation of this sort might expand, to offer e-discovery services for moderately sized matters, on a national scale. 

Even if a national smaller-case vendor does not emerge, one can imagine consortia of smaller clients and law firms forming, which could become significant players as consumers in the market.  An array of government entities (such as education boards) in a geographic area, for example, might pool their buying power, to obtain discounts on e-discovery and related legal services, on a “preferred customer” basis.  These consortia, moreover, could demand the kinds of targeted education (outlined above) not generally available through conventional CLE and trade show programs.  Existing small-firm and solo practitioner committees at the ABA and at many state bar associations, might also help circulate information on smaller-case options, including vendors who cater to this segment of the market.


The drive to develop improved rules of civil procedure for e-discovery is admirable.  Serious and well-intentioned thinkers, in the judiciary, legal academia and the legal profession as a whole have spent years in the process.  But rules alone will not tame e-discovery.  Indeed, as new technology rapidly develops, and social and business practices rapidly change, procedural rules inevitably lag behind.  Creation and expansion of e-discovery best practices requires free-flowing information exchange, collaboration between the many technical and professional constituencies affected by e-discovery (and the larger world of information management and governance), and a constant drive to educate, at law schools, in CLE fora, and in the host of new media forms now available. 

[1] Partner at Park Jensen Bennett LLP, in New York City.  The views expressed are solely those of the author, and should not be attributed to the author’s firm, or its clients.

[2] For a brief introduction to computer history, see Michael S. Mahoney, The Histories Of Computing, 30 Interdiscip. Sci. Rev. 119 (2005).  

[3] See Emergy G. Lee III & Thomas Willging, Litigation Costs in Civil Cases: Multivariate Analysis Report to the Judicial Conference Advisory Committee on Civil Rules (March 2010),

[4] See generally

[5] One survey proponent gave up the process of ranking vendors, largely because consumers of e-discovery software and services were “using them as a substitute for the work they should be doing themselves—analyzing [their] needs and assessing what and whose services and software might meet those needs.”  See The Socha-Gelmann Electronic Discovery Survey: Time For A Change (Aug. 8, 2008), (noting that “anyone who makes buying decisions primarily on these rankings is a fool”).

[6] One example appears in the Enron litigation materials collection.  See John Wang, Cameron Coles, Rob Elliot & Sofia Andrianakou, Comparing Exclusionary And Investigative Approaches For Electronic Discovery Using The TREC Enron Corpus (2009), available at

[8] See, e.g., Legal And Tax Advisers Join To Offer One-Step Services (Sept. 2, 1982), (discussing linkage of Hyatt Legal Services and H&R Block accounting services).

For more information about LexisNexis products and solutions, connect with us through our corporate site.