Michael Cohen President Donald Trump's attorney Michael Cohen arrives at federal court in Manhattan on April 26, 2018. (Photo by David Handschuh/NYLJ) |

In a late April court filing, government attorneys in the United States v. Cohen case in the U.S. District Court for the Southern District of New York officially withdrew their opposition to having a special master conduct privilege review of the documents seized from Michael Cohen, the personal lawyer of President Donald Trump.

However, they also suggested a new way forward: The review should rely on a process proposed by one of the government's candidates for special master, former U.S. Magistrate Judge Frank Maas of the Southern District of New York. That process involved using technology-assisted review (TAR) “to identify potentially privileged material for review in an efficient manner,” noted a letter attached to the filing.

In the letter, Maas, who is now at commercial dispute resolution services provider JAMS, noted he would use a TAR process known as continuous active learning, which relies on supervised learning to train a tool to identify certain attributes in documents, such as privilege. The process, Maas wrote, “has been shown in many studies to be at least as effective, as exhaustive” as manual review.

Maas added he would bring on Maura Grossman, one of the leaders in TAR protocol development and now a research professor in the David R. Cheriton School of Computer Science at the University of Waterloo, Ontario, to assist with the review. Grossman declined to comment for this article.

Ultimately, U.S. District Judge Kimba Wood of the Southern District of New York, who is presiding over the Cohen case, chose Barbara Jones, former Southern District of New York judge and current partner at Bracewell, as the special master.

But the government's support for using TAR has raised questions about how such technology could work in the current case and what potential advantages it could bring to this high-profile fracas. To be sure, using any automation technology to identify privileged content, which is defined not by a fixed standard but by subjective legal interpretation, can be difficult.

“In general, using TAR for privilege review is more challenging than using it to find relevant documents,” Maas told Legaltech News.

The TAR process Grossman and her team developed is up to the challenge, however, because it “considers metadata as part of the algorithm, so it looks at things like the 'to' field, the 'from' field, the 'bcc' and 'cc' fields, and so forth,” Maas added.

Maas noted that after finding all communications between Cohen and his clients, one can then use those documents to “train the tool through an iterative process to discriminate between privileged and non-privileged communications.”

Dave Lewis, chief data scientist at Brainspace Corp., explained that such training involves giving a TAR tool “examples of privileged and non-privileged documents. The software would then use machine learning to build a predictive model, a kind of statistical model, that gives each document in the collection a numeric score indicating how likely it is to be privileged.”

He added that the type of continuous learning proposed by Maas is “what is called relevancy feedback. That's when the top scoring documents are the ones that get reviewed for privilege and are also used to train the software further, so you have an iterative training and review cycle.”

Training TAR technology to identify privileged documents, however, is just one of several ways to use it for privilege review. “Another approach would be to flip the problem on the side and build one or more TAR models to identify types of non-privileged material to remove them from consideration,” Lewis said.

What's more, parties could also “use TAR only for quality control,” he added, “so they have essentially a manual process but build a predictive model to double-check the manual process.”

The advantage of using TAR in privilege reviews comes down to speed and accuracy. In their court filing, government attorneys noted that “such a process generally provides for the most efficient, expeditious and neutral review.”

Indeed, in his letter, Maas noted that barring complications, “the entire technology-assisted review of the electronic data could likely be accomplished in one to two weeks.”

When asked how such a review could take just weeks given the potentially large volume of material in consideration, Maas noted that “whatever the quantity of information is, it all comes from one source: Mr. Cohen. Accordingly, although I have no idea what the ultimate volume of electronically stored information is, this case certainly is a far cry from one involving the review of the materials of 400 corporate custodians.”

He added that his estimate was based on the knowledge that “the electronically stored information came from one individual with very few clients.”

Such a process might be slowed down if parties fought over the accuracy of the TAR tool used or the way in which it was trained. But unlike some advanced technology processes, there is transparency around how TAR gets to its determinations.

“For instance, you can examine the words and phrases that are used by the statistical models and what numerical weights the learning algorithm has associated with them,” Lewis said. Each party can also monitor how a TAR tool is trained through each iterative process or train the tool on its own interpretation of privilege to show how results would differ.

It's an open question whether special master Jones, whose appointment was widely praised by colleagues, will use TAR tools in her review. At the very least, Wood, responding to government attorneys' questions in court, was confident Jones was up to the task. “I think she'll be technologically well-suited to the job,” she said.

Still, Jones' history with TAR is not well-documented. “I have no idea how extensively [Jones] has used TAR in the past, if at all,” Maas said. “She is obviously a highly experienced judge with the resources of a large law firm, so I'm confident she will develop an efficient process.”

Whatever process Jones relies on will be sure to obviously catch the attention of discovery and search technology professionals across the legal world. “All of us in [the] e-discovery area are going to be fascinated to hear what it turns out to be,” Lewis said.