The latest e-discovery opinion to cause a stir is from a federal magistrate judge well known in the field for his ability to communicate e-discovery “best practices” to practitioners. In Da Silva Moore, et al. v. Publicis Groupe & MSL Group,[1] Magistrate Judge Andrew J. Peck issued the first reported opinion dealing with a technology called “predictive coding.”

While Judge Peck's comments at the hearing preceding the written opinion were quickly heralded by vendors as officially “validating” predictive coding as a judicially endorsed software product, the opinion specifically rejects this misinterpretation.[2] Rather, Judge Peck states that predictive coding is an appropriate tool for certain cases but should be subject to the same kind of rigorous analysis and testing as other methods of document review:

[I]f the use of predictive coding is challenged in a case before me, I will want to know what was done and why that produced defensible results. I may be less interested in the science behind the “black box” of the vendor's software than in whether it produced responsive documents with reasonably high recall and high precision.

The underlying case involves claims of gender-based employment discrimination. In the course of discovery, the parties disagreed on the appropriate e-discovery protocol—not with respect to the use of any particular technology, but about such mundane matters as how many documents should be reviewed, which custodians' email should be searched, which custodians' documents should be reviewed when, discovery cutoff dates and what sources of electronic information should be searched.

Judge Peck's opinion advises attorneys that predictive coding can be an acceptable aid in conducting document review in appropriate cases, as long as it is part of a process that is defensible, and subjected to quality control testing appropriate for any document review, however conducted. These kinds of quality control methods have previously been elaborated by the Sedona Conference, in its white paper entitled “Commentary on Achieving Quality in the E-Discovery Process,” and in the New York State Bar Association's “Best Practices in e-discovery in New York State and Federal Courts.” Predictive coding is a term that has become a buzzword in e-discovery, but seems to have various meanings depending on which software vendor is speaking. Vendors often use the term in conjunction with the enticing moniker “automated review.” Judge Peck uses the term “computer-assisted coding” and defines it as “tools … that use sophisticated algorithms to enable the computer to determine relevance, based on interaction with (i.e., training by) a human reviewer.”

In Da Silva Moore, Judge Peck asked the defendants to agree on a transparent protocol that would include producing the documents used to “train” the software, i.e., the seed documents, as well as the other parameters of the process. While Judge Peck did accept predictive coding as an acceptable tool in this case to facilitate e-discovery, the underlying condition of acceptance—that this acceptance would be contingent on whether it was defensible by being “quality control verified” —would be advisable in any document review technology or process:

As with keywords or any other technological solution to e-discovery, counsel must design an appropriate process, including use of available technology, with appropriate quality control testing, to review and produce relevant ESI …

The underlying reasoning is as clear as it is incontrovertible: if you can verify that the results are accurate, which technology generated them is a moot point.

The bottom line here is that there is more than one way to skin a cat. The trick is to verify with a high degree of confidence that the cat has in fact been skinned. The application of any particular software in conducting document review has to be part of a process in which a number of elements apart from the software are used to arrive at the right results. The arrival of the e-discovery “easy button” has been delayed. Human judgment is still part of the process, and we all know that reasonable minds will differ—especially in litigation.

The views expressed herein are those of the authors and do not necessarily reflect the views of Ernst & Young LLP.

[1] 1:11-cv-01279-ALC-AJP (S.D.N.Y. Feb. 24, 2012).

[2] Judge Peck noted that the vendor of the software used by defendants had initiated a press release mischaracterizing the opinion as ordering the parties to use predictive coding. The parties had already agreed to its use, and Judge Peck accepted this agreement.

The latest e-discovery opinion to cause a stir is from a federal magistrate judge well known in the field for his ability to communicate e-discovery “best practices” to practitioners. In Da Silva Moore, et al. v. Publicis Groupe & MSL Group,[1] Magistrate Judge Andrew J. Peck issued the first reported opinion dealing with a technology called “predictive coding.”

While Judge Peck's comments at the hearing preceding the written opinion were quickly heralded by vendors as officially “validating” predictive coding as a judicially endorsed software product, the opinion specifically rejects this misinterpretation.[2] Rather, Judge Peck states that predictive coding is an appropriate tool for certain cases but should be subject to the same kind of rigorous analysis and testing as other methods of document review:

[I]f the use of predictive coding is challenged in a case before me, I will want to know what was done and why that produced defensible results. I may be less interested in the science behind the “black box” of the vendor's software than in whether it produced responsive documents with reasonably high recall and high precision.

The underlying case involves claims of gender-based employment discrimination. In the course of discovery, the parties disagreed on the appropriate e-discovery protocol—not with respect to the use of any particular technology, but about such mundane matters as how many documents should be reviewed, which custodians' email should be searched, which custodians' documents should be reviewed when, discovery cutoff dates and what sources of electronic information should be searched.

Judge Peck's opinion advises attorneys that predictive coding can be an acceptable aid in conducting document review in appropriate cases, as long as it is part of a process that is defensible, and subjected to quality control testing appropriate for any document review, however conducted. These kinds of quality control methods have previously been elaborated by the Sedona Conference, in its white paper entitled “Commentary on Achieving Quality in the E-Discovery Process,” and in the New York State Bar Association's “Best Practices in e-discovery in New York State and Federal Courts.” Predictive coding is a term that has become a buzzword in e-discovery, but seems to have various meanings depending on which software vendor is speaking. Vendors often use the term in conjunction with the enticing moniker “automated review.” Judge Peck uses the term “computer-assisted coding” and defines it as “tools … that use sophisticated algorithms to enable the computer to determine relevance, based on interaction with (i.e., training by) a human reviewer.”

In Da Silva Moore, Judge Peck asked the defendants to agree on a transparent protocol that would include producing the documents used to “train” the software, i.e., the seed documents, as well as the other parameters of the process. While Judge Peck did accept predictive coding as an acceptable tool in this case to facilitate e-discovery, the underlying condition of acceptance—that this acceptance would be contingent on whether it was defensible by being “quality control verified” —would be advisable in any document review technology or process:

As with keywords or any other technological solution to e-discovery, counsel must design an appropriate process, including use of available technology, with appropriate quality control testing, to review and produce relevant ESI …

The underlying reasoning is as clear as it is incontrovertible: if you can verify that the results are accurate, which technology generated them is a moot point.

The bottom line here is that there is more than one way to skin a cat. The trick is to verify with a high degree of confidence that the cat has in fact been skinned. The application of any particular software in conducting document review has to be part of a process in which a number of elements apart from the software are used to arrive at the right results. The arrival of the e-discovery “easy button” has been delayed. Human judgment is still part of the process, and we all know that reasonable minds will differ—especially in litigation.

The views expressed herein are those of the authors and do not necessarily reflect the views of Ernst & Young LLP.

[1] 1:11-cv-01279-ALC-AJP (S.D.N.Y. Feb. 24, 2012).

[2] Judge Peck noted that the vendor of the software used by defendants had initiated a press release mischaracterizing the opinion as ordering the parties to use predictive coding. The parties had already agreed to its use, and Judge Peck accepted this agreement.