Document review is, of course, a crucial phase in the production of discovery. Prior to the ascendance of computers, the documents involved would, typically, be contracts, letters, reports and other such documents, typed or handwritten and kept by the client in a file regarding the matter. If the documents filled an expansion folder, they were considered a large number of documents; if they filled a banker's box, document production was considered huge.

Today, of course, the number of documents produced in discovery is considerably greater. There are two principal reasons for the increase, both due to the advent of computers. First, communications that were in years past verbal—in person, on the phone—are now done, or summarized after the verbal conversation, in emails and texts, driving up the number of responsive documents considerably. Second, since documents generated or received by computers are stored, usually on several devices and in backups, nonresponsive documents—a number usually considerably greater than that of responsive documents—must also be collected and searched to determine whether any or all are responsive are also stored.

Given the volume of documents that must be reviewed when a party is producing discovery, it is common for the party to contract with a service provider who provides managed review of documents, that is, attorneys who review files for responsiveness and redactions. In this month's column, we will look at the many issues which arise under managed review, some of which are common to any discovery production.

Data Collection and Preservation

The number of locations where data can be and is stored today has been written about voluminously. Imagine a client where 100 employees each have computer access to the company's various digital applications—email, e-docs of various kinds (e.g., Word, PDF, PowerPoint, Excel, texts, etc.). Such e-docs will be found on the computers assigned to the employees, their old computers (if preserved), their cellphones, the company's servers, whatever active back-up servers the company uses, and whatever digital “snapshots” of servers the company or its back-up provider maintains, e.g., copies of servers created every seven days, 30 days, etc. If companies permit (or do not technically prohibit) employees from using their personal computers, e-docs will likely also be found on them, as well as on all of the servers and other backups of which those employees may make use.

There will be, of course, be a tremendous number of duplicate files in all of these locations. Such “duplicates” can be exact duplicates, i.e., duplicates of the files' contents, metadata, etc., or can be near-duplicates whose differences, from the point of view of document review, are insignificant. At the same time, only slight differences in documents may be of substantial significance from the point of view of producing responsive documents. A Word document, for example, involving a key issue in the matter, may have a phrase such as “in most cases” in one version of the document but not in another; the addition or removal of that phrase may be highly relevant to a point at issue.

The de-duplication of documents, then, that are not exact duplicates, involves judgment. As well, the locations of the duplicates may also be highly relevant. If a witness claims, for example, not to know anything regarding a matter which is the subject of copies of files found on the witness' computer, the presence of such copies may be of great relevance to the matter.

There are several approaches to de-duplication. Exact duplicates of a document can be removed, with locations of the duplicates tracked, since those locations, as discussed above, may be relevant. When the content of files is exactly duplicative but metadata differs, the differences may be relevant, e.g., the differences may reveal the copying of a file that a witness claims was never copied.

How granular a look at de-duplication reviewers will take will depend upon the needs of the litigation. If the review will be granular, the reviewer will have to have an understanding of the matter that runs deeper than that of the typical reviewer. Often in such cases, the reviewer with that deeper understanding is referred to the duplicate files either by a reviewer tasked with the more basic duty to review the contents of a file and then simply to note such differences as metadata differences as captured by the review software in databases, or by reviewing such databases himself.

Content Review

Consider the following cases in which documents must be reviewed and the responsive ones produced (and redacted if need be) in discovery:

• Cases in which only a small number of documents must be reviewed;
• Cases in which only a small number of documents must be reviewed, but those documents contain arcane information requiring great expertise to understand (e.g., formulae and other information needed to create a new drug in a case where the new drug manufacturer is being sued by another drug manufacturer who claims that the “new” drug is simply a copy of the old one and so the plaintiff's patent has been violated);
• Cases in which a substantial number of documents must be reviewed;
• Cases in which a substantial number of documents must be reviewed, many containing arcane information;
• Cases in which an overwhelming number of documents must be reviewed;
• Cases in which an overwhelming number of documents must be reviewed, many containing arcane information;
• Cases in which a substantial or overwhelming number of cases must be reviewed, where the content of many files is exactly duplicative, but the metadata differs.

There are several ways to approach the review of such cases. Which is the best way may not be obvious even to the best reviewers.

Cases with a small number of documents to review can and should be reviewed in-house, after the files have been processed and uploaded to a review platform, such as Relativity. Those involving arcane information should, of course, be reviewed only by a reviewer who understands that information. The review itself should be reviewed by another knowledgeable reviewer, since opinions may vary as to the meaning of the arcane information.

Cases with a substantial or overwhelming number of files to review will likely be reviewed by managed review teams provided by an outside supplier. The client and firm will likely look to managed review providers for three reasons: such providers will charge less than will the law firm; law firms can rarely bill per hour as low as providers can, as firms usually pay even their entry-level associates more per hour; and, the firm will rarely have a sufficient number of lower-level associates who have no other work and so can devote all their time to the review. Managed review teams should have layers of supervision to check and double-check that nothing has been missed, in terms of content, other aspects of the files under review (e.g., redactions are proper and none have been missed, the significance of their location, metadata, etc.), or the aforementioned approach to review.

Artificial intelligence (AI) tools can be used in reviews of a substantial or overwhelming number of files to decrease the number of reviewers needed and spot errors or omissions. A typical AI tool can work by having a knowledgeable reviewer (or two or more, depending upon the size of the dataset and the deadline for review to be completed) review a quantity of files. The AI tool will then automatically review the next files, of the same quantity, and tag them as the reviewer did the initial set (e.g., responsive, nonresponsive, privileged, needs redaction, etc.). The reviewer reviews the set done by the AI tool and makes corrections, thereby “teaching” the AI application. The AI application will then review the third set, which the reviewer will review and correct. Each review set done by the AI application should contain fewer errors than the previous one, until AI review produces virtually no errors. Once the AI application has, then, thoroughly “learned” how to review, the reviewer can simply let the application review all remaining files. The reviewer(s) can then spot-check the entire set and, unless a problem with the AI review has been detected, complete redactions and produce responsive documents. One additional benefit of AI review is that reviewers can easily include firm counsel knowledgeable of the issues in the matter. If such counsel understands how to use AI applications, they can simply go ahead; if they need assistance, they can work with managed review provider counsel, who can initially instruct them and then oversee their work as the review progresses. Once review is complete, the production to opposing counsel can be finished by the e-discovery provider or by the firm if such is done in-house.

Conclusion

Managed review requires a great deal of expertise, to determine what review best suits the matter and to conduct that review. Opinions, of course, will vary as to how to answer these questions, as well as which providers conduct the best reviews, but opinions vary regarding virtually every topic about which people have opinions. The more experience the law firm client has regarding review, the better position it will be in to decide what type of review should be undertaken and by whom.

Leonard Deutchman is a legal consultant recently retired from one of the nation's largest e-discovery providers, KLDiscovery, where he was vice president, Legal. Before joining KLDiscovery, he was a chief assistant district attorney at the Philadelphia District Attorney's Office, where he founded the Cyber Crime Unit and conducted and oversaw hundreds of long-term investigations involving cybercrime, fraud, drug trafficking and other offenses.