We read with interest the recent article suggesting that the use of technology-assisted review (TAR) to cull privileged documents from voluminous materials seized from an attorney's office pursuant to a search warrant is “improper and potentially unconstitutional.” N.Y.L.J., Jan. 15, 2019, at 4. The article focuses on the Government's seizure of documents from the office of Michael Cohen, who was then serving as President Trump's personal attorney. We write to respond briefly to several points made by the authors that are inaccurate.

First, the authors assume that TAR would have been used to the exclusion of manual review and Cohen's counsel would have had no input into the process we suggested. That is wrong. The TAR tool would have been used to identify potentially privileged documents, which would, of course, have been reviewed by humans, including the Special Master and Cohen's counsel, who would have been given an opportunity to identify other documents in the production set for which Cohen claimed privilege.

Second, the authors presume that a TAR tool would have been a blunt—and likely inaccurate—instrument for privilege review. That, too, is incorrect. By programming the tool to cast a wide net, this concern would have been mitigated. In TAR-speak, the goal would have been to achieve high recall (i.e., finding as many privileged documents as reasonably possible) at the cost of relatively low precision (i.e., presenting a relatively large number of documents for review). Given the prosecution's belief that the number of privileged documents was likely to be low—since Mr. Cohen had few clients—even a precision of 25 percent or 30 percent would still have resulted in a significant savings of review time and cost over an entirely manual review. What the TAR process was intended to avoid was a needlessly lengthy first-pass review by Cohen and his counsel, who initially contended that the seized documents might include “thousands, if not millions” of privileged documents. (As the authors point out, fewer than 10,000 documents out of over 1.3 million seized were ultimately presented to the Special Master for her consideration.)

Third, the authors apparently assume that TAR is an “untested technology” that is necessarily inferior to manual review for privilege. N.Y.L.J., Jan. 15, 2019, at 10. In fact, there is ample research demonstrating that TAR is superior to manual review, including for identifying confidential documents. See, e.g., G.V. Cormack and M.R. Grossman, “Navigating Imprecision in Relevance Assessments on the Road to Total Recall: Roger and Me,” Proceedings of SIGIR 2017 (August 2017). Unlike junior associates and contract attorneys, a computer using TAR does not get tired after lunch, can work around the clock, and makes accurate and consistent determinations. Moreover, the continuous active learning (CAL) protocol we proposed involved iterative training and would have been continued until no further privileged documents could be found, by TAR or through other means, including statistical sampling and keyword search.

Finally, in Rio Tinto PLC v. Vale S.A., 306 F.R.D. 125 (S.D.N.Y. 2015)—a civil case—Judge Peck, a leading (now retired) e-discovery judge, did not “deny[] a requesting parties' attempt to force producing parties to use TAR,” as the authors assert. Indeed, in Rio Tinto, Judge Peck specifically noted that “it [was] not clear what a court might do if the issue were raised before the producing party had spent any money on document review.” Id. at 127 n.1. Judge Peck also did not express “reticence to get involved in drafting TAR protocols” in Rio Tinto. He did not have to address that question because the parties had already agreed to a TAR protocol. We suspect that the authors meant to cite Hyles v. New York City, 2016 WL 4077114 (S.D.N.Y. Aug. 1, 2016), but even in that case, where Judge Peck declined to order the producing party to use TAR, he did acknowledge the possibility that there might come a day when not using TAR would be unreasonable. Id. at *3.

In sum, we strenuously disagree with the notion that the use of TAR to assist with the privilege review in the Cohen case would have rendered it unfair or unconstitutional.

Frank Maas served for 17 years as a U.S. Magistrate Judge for the Southern District of New York, including a two-year term as Chief Magistrate Judge. Maura R. Grossman is a Research Professor at the University of Waterloo.