The options available for identifying relevant and responsive electronically stored information (ESI) are increasing. Predictive coding, or computer-assisted review, can now be considered “judicially-approved for use in appropriate cases.” Da Silva Moore v Publicis Groupe & MSL Group, Case No. 11 Civ. 1279, Feb. 24, 2012 Opinion and Order. The Da Silva Moore case noted that predictive coding “should be seriously considered for use in large-data-volume cases where it may save the producing party (or both parties) significant amounts of legal fees in document review.” Id. 

In the Da Silva Moore case, Magistrate Judge Andrew Peck described “predictive coding” or “computer-assisted review” as “tools…that use sophisticated algorithms to enable the computer to determine relevance, based on interaction with (i.e., training by) a human reviewer.” Id. A human reviewer, often a senior lawyer, will review and code a “seed set” of documents, from which the computer identifies properties that it will use to code other documents. As the human reviewer codes more documents, the computer will predict the coding; or, the computer will code some documents and ask the human reviewer for feedback.

Judge Peck explained further that, “[w]hen the system's predictions and the reviewer's coding sufficiently coincide, the system has learned enough to make confident predictions for the remaining documents. Typically, the senior lawyer (or team) needs to review only a few thousand documents to train the computer.” Id. 

Judge Peck opined that computer-assisted review is an acceptable way to search for relevant ESI in appropriate cases. See id. The parties in Da Silva Moore had been discussing an electronic discovery protocol to try to cull down approximately 3 million electronic documents from the agreed-upon custodians. No party was opposed to using predictive coding, but plaintiffs wanted clarification on the way in which defendant planned to employ this technology. 

Judge Peck granted the parties' request to use predictive coding and considered the parties' positions on an appropriate ESI protocol. Plaintiffs, however, objected to the “final” ESI protocol approved by the court, after having submitted their own proposed ESI protocol which the court largely rejected. The parties thus agreed to use computer-assisted review but disagreed about how best to implement such review. Id.

The court subsequently “determined that the use of predictive coding was appropriate considering: (1) the parties' agreement, (2) the vast amount of ESI to be reviewed (over 3 million documents), (3) the superiority of computer-assisted review to the available alternatives (i.e., linear manual review or keyword searches), (4) the need for cost effectiveness and proportionality under Rule 26(b)(2)(C), and (5) the transparent process proposed by [defendant].” Id. 

District Judge Andrew Carter, Jr. approved Judge Peck's ruling, finding no basis to hold that his conclusion was clearly erroneous or contrary to law. Da Silva Moore v. Publicis Groupe & MSL Group, Case No. 11 Civ. 1279 (S.D.N.Y. April 25, 2012). Judge Carter acknowledged plaintiffs' concerns with the ruling but concluded that the “ESI protocol contains standards for measuring the reliability of the process and the protocol builds in levels of participation by Plaintiffs. It provides that the search methods will be carefully crafted and tested for quality assurance, with Plaintiffs participating in their implementation.” Id.

In another case involving disputed ESI discovery issues, Global Aerospace Inc. v. Landow Aviation, L.P., Case No. CL 61040 (Vir. Cir. Ct.), defendants requested permission to use predictive coding for the processing and production of their ESI. Plaintiffs opposed the request. On April 23, the Virginia Circuit Court granted defendants' request and issued an Order Approving the Use of Predictive Coding for Discovery.

The court ordered that “Defendants shall be allowed to proceed with the use of predictive coding . . . . This is without prejudice to a receiving party raising with the Court an issue as to completeness or the contents of the production or the ongoing use of predictive coding.” Although the court did not elaborate on or explain its order, it did allow defendants to use predictive coding to process and produce their ESI. 

Finally, plaintiffs in Kleen Products LLC v. Packaging Corp. of America, Case No. 1:10-cv-05711 (N.D. Ill.), also recently requested court approval of the use of computer-assisted review. In support of their request, plaintiffs argued that their content-based search methodology was far superior to the defendants' Boolean search methodology and was in line with the principles of the Sedona Conference, a non-partisan research and educational institute which has published a series of best practices and principles for addressing electronic document production. The court has not yet decided this issue. 

The aforementioned case law is probably too small a sample to predict how many other courts across the country will approve the use of predictive coding. The cases highlight, however, that simple search term filtering and human review are not the only way, and may no longer be the best way, to identify relevant records in all cases, particularly those involving large collections of ESI. The decisions by the District Court for the Southern District of New York and Virginia Circuit Court approving the use of predictive coding have set the stage for significant change in the legal landscape of court-accepted protocols for identifying relevant and responsive ESI. Time will tell whether this is the beginning of a “sea change” in how relevant records are identified, or only a smaller blip on the radar screen that will have little applicability to most cases anytime soon. Keep an eye out for further developments in this area!

The options available for identifying relevant and responsive electronically stored information (ESI) are increasing. Predictive coding, or computer-assisted review, can now be considered “judicially-approved for use in appropriate cases.” Da Silva Moore v Publicis Groupe & MSL Group, Case No. 11 Civ. 1279, Feb. 24, 2012 Opinion and Order. The Da Silva Moore case noted that predictive coding “should be seriously considered for use in large-data-volume cases where it may save the producing party (or both parties) significant amounts of legal fees in document review.” Id. 

In the Da Silva Moore case, Magistrate Judge Andrew Peck described “predictive coding” or “computer-assisted review” as “tools…that use sophisticated algorithms to enable the computer to determine relevance, based on interaction with (i.e., training by) a human reviewer.” Id. A human reviewer, often a senior lawyer, will review and code a “seed set” of documents, from which the computer identifies properties that it will use to code other documents. As the human reviewer codes more documents, the computer will predict the coding; or, the computer will code some documents and ask the human reviewer for feedback.

Judge Peck explained further that, “[w]hen the system's predictions and the reviewer's coding sufficiently coincide, the system has learned enough to make confident predictions for the remaining documents. Typically, the senior lawyer (or team) needs to review only a few thousand documents to train the computer.” Id. 

Judge Peck opined that computer-assisted review is an acceptable way to search for relevant ESI in appropriate cases. See id. The parties in Da Silva Moore had been discussing an electronic discovery protocol to try to cull down approximately 3 million electronic documents from the agreed-upon custodians. No party was opposed to using predictive coding, but plaintiffs wanted clarification on the way in which defendant planned to employ this technology. 

Judge Peck granted the parties' request to use predictive coding and considered the parties' positions on an appropriate ESI protocol. Plaintiffs, however, objected to the “final” ESI protocol approved by the court, after having submitted their own proposed ESI protocol which the court largely rejected. The parties thus agreed to use computer-assisted review but disagreed about how best to implement such review. Id.

The court subsequently “determined that the use of predictive coding was appropriate considering: (1) the parties' agreement, (2) the vast amount of ESI to be reviewed (over 3 million documents), (3) the superiority of computer-assisted review to the available alternatives (i.e., linear manual review or keyword searches), (4) the need for cost effectiveness and proportionality under Rule 26(b)(2)(C), and (5) the transparent process proposed by [defendant].” Id. 

District Judge Andrew Carter, Jr. approved Judge Peck's ruling, finding no basis to hold that his conclusion was clearly erroneous or contrary to law. Da Silva Moore v. Publicis Groupe & MSL Group, Case No. 11 Civ. 1279 (S.D.N.Y. April 25, 2012). Judge Carter acknowledged plaintiffs' concerns with the ruling but concluded that the “ESI protocol contains standards for measuring the reliability of the process and the protocol builds in levels of participation by Plaintiffs. It provides that the search methods will be carefully crafted and tested for quality assurance, with Plaintiffs participating in their implementation.” Id.

In another case involving disputed ESI discovery issues, Global Aerospace Inc. v. Landow Aviation, L.P., Case No. CL 61040 (Vir. Cir. Ct.), defendants requested permission to use predictive coding for the processing and production of their ESI. Plaintiffs opposed the request. On April 23, the Virginia Circuit Court granted defendants' request and issued an Order Approving the Use of Predictive Coding for Discovery.

The court ordered that “Defendants shall be allowed to proceed with the use of predictive coding . . . . This is without prejudice to a receiving party raising with the Court an issue as to completeness or the contents of the production or the ongoing use of predictive coding.” Although the court did not elaborate on or explain its order, it did allow defendants to use predictive coding to process and produce their ESI. 

Finally, plaintiffs in Kleen Products LLC v. Packaging Corp. of America, Case No. 1:10-cv-05711 (N.D. Ill.), also recently requested court approval of the use of computer-assisted review. In support of their request, plaintiffs argued that their content-based search methodology was far superior to the defendants' Boolean search methodology and was in line with the principles of the Sedona Conference, a non-partisan research and educational institute which has published a series of best practices and principles for addressing electronic document production. The court has not yet decided this issue. 

The aforementioned case law is probably too small a sample to predict how many other courts across the country will approve the use of predictive coding. The cases highlight, however, that simple search term filtering and human review are not the only way, and may no longer be the best way, to identify relevant records in all cases, particularly those involving large collections of ESI. The decisions by the District Court for the Southern District of New York and Virginia Circuit Court approving the use of predictive coding have set the stage for significant change in the legal landscape of court-accepted protocols for identifying relevant and responsive ESI. Time will tell whether this is the beginning of a “sea change” in how relevant records are identified, or only a smaller blip on the radar screen that will have little applicability to most cases anytime soon. Keep an eye out for further developments in this area!