Most of the hype around new technology holds the promise of a magic pill that can solve our business problems at speeds never before imaginable. Those of us who have experienced more than one technology wave concede the truth involves integrating proven tech with solid work methodology to achieve reliable results. Being the first to the party otherwise might end up being both awkward and disastrous at the same time.

Add to the mix a highly time-compressed business threat requiring all-hands-on-deck with stratospheric resource demands and who's ready to bet the farm?

In this new article series, we explore where the balance point between technology selection/use and an effective legal project management approach exists to solve critical business use case challenges.

Let's Buy a Company!

Business combinations can be powerful (particularly with slowing organic growth in many industries). The strategic decision to acquire a company can quickly expand market share, geographic reach, add new product/service capabilities and gain access to an entirely new client/customer base. It can also trigger regulatory scrutiny that can thwart completion of the deal while destroying equity value, market capitalization and goodwill all in a single blow. This potential failure can even weaken a company to the point of its own undoing.

Under U.S. antitrust law, a second request is a discovery procedure by which the Antitrust Division of the U.S. Department of Justice and the Federal Trade Commission investigate mergers and acquisitions that have anticompetitive consequences. Unlike many other business challenges, a second request is a highly time-compressed process, relegating a 90-day period in which the target is given to respond in combination with a voluminous magnitude of data it must review for responsiveness/privilege and ultimately produce.

When Cost Is No Object But Space Travel Remains Elusive

Elon Musk sending his sports car into space just because he could is one thing (albeit still a curiosity), but companies willing to write blank checks when contested don't always succeed. Most large enterprise second request legal budgets are proportional to the risk they present, inclusive of the downside risk/costs of failure to launch and the potential loss of what the business combination otherwise would amount to in revenue growth/market access (frequently in the billions of U.S. dollars).

Despite a cost-be-damned strategy to get a second request response approved, scaling to meet those demands is rarely successful without outside counsel and a capable alternative legal service provider (specializing in e-discovery technology best practices and global review core competency) partnering on some level with the company. This remains true despite the company selecting any one of an array of the most reputable outside counsel law firms who have integrated some level of captured document review services.

All Documents Are Not Created Equal

One of the initial technologies adopted by many legal departments, and ultimately by the courts, was sample based learning found within technology assisted review (TAR 1.0). TAR uses samples or "seed sets" to train the algorithm to then apply coding to the larger review universe. Statistical analysis is then run on the entire review set to confirm the accuracy of the algorithm.  If the desired accuracy isn't met, you then must train the system again, in an iterative fashion, until you meet your accuracy level.

The issue becomes that this process needs to be recalibrated each time you change search terms for the issues sought (which is a frequent need during most cases) or upon a change in the data set. This is not ideal and can be fatal in a second request scenario where a 90-day, time-compressed period for production is not flexible. This could also back you in a corner by requiring an army of attorneys/review team to boil the ocean on every issue subject to this make-or-break endeavor.

Keep in mind the end game here is to review and produce relevant, nonprivileged documents so getting to them and putting eyes on those most critical in terms of relevancy and privilege at the earliest possible time juncture (otherwise known as speed to legal intelligence). Recalibrating TAR 1.0 is not something we have the bandwidth to accomplish if we need to go back to the well each time there are material changes to the search terms or data set.

Game Changer

Technology-assisted review comes in a variety of flavors, with protocols that include simple passive learning (SPL), simple active learning (SAL) and a newer approach, continuous active learning (CAL). The "continuous" aspect of CAL refers to the ongoing process of ranking and re-ranking documents for manual review based on a constant stream of incoming coding throughout the review's lifecycle. In other words, CAL takes into account not only an initial set of training assessments to rank and prioritize documents, but continuously updates those rankings based on the most recent assessments. This removes any need to go back to the well, recalibrate document seed sets and recalibrate results until a target level of accuracy is achieved.

While we have had great success in negotiations with the government (helping them understand and become comfortable with the process and benefits of using CAL), it can also be helpful even if it isn't used to make coding decisions.

Some of the benefits of utilizing CAL:

• CAL can be run in the background and used to prioritize documents for review.
• Enablement to more effectively utilize finite time windows to review responsive rather than nonresponsive documents.
• Identification of documents for secondary workflow review (privilege, redaction, logging) earlier in the review process.
• The ability to quickly rank and prioritize new data as it is added.
• No additional cost in most (if not all) cases to use CAL (and most frequently realization of a measurable spend reduction by doing so).

CAL vs. TAR 1.0 (Flying Versus Walking)

Continuous active learning (CAL or TAR 2.0) uses support vector machine learning (SVM) to draw a line of best fit to rank documents from 1-100, with 1 being least likely to be relevant and 100 being most likely. Documents are ranked from the outset of review and are continuously re-ranked during the course of review. There are two different workflows we can use in active learning:

• Priority review—In this workflow, unreviewed highly ranked documents are given to reviewers first feeding them the most responsive documents to review.
• Coverage review—Here, documents that the model is struggling to categorize (around rank 50) are prioritized allowing the system to learn the most quickly and "stabilize" where training the system further doesn't result in greater accuracy of the model.

With CAL workflows being an ongoing process, the general assumption is that all responsive documents will be manually reviewed and that all coded documents will be incorporated into the continuously growing training set. Put in the context of a second request, this means "eyes on" the relevant documents and those that may be privileged is mission critical.

Unlike other TAR 1.0 approaches, the end goal is not to automatically classify documents either as responsive or nonresponsive. CAL, on the other hand, is optimized to route likely responsive documents to the manual review queue while curtailing inclusion of nonresponsive documents. In this context, ongoing document prioritization is the driver rather than automated one-time classification, making the notion of "seed set" largely irrelevant.

The benefits of CAL over TAR 1.0 are:

• Review of priority documents faster​.
• Less time on administration​.
• Far greater flexibility and versatility.
• Can easily handle rolling/continuously added data​.
• Validation with confidence​.
• Resistant to incorrect/contradictory coding decisions.

Next Steps

The balance between technology selection/use and effective legal project management strategy/workflow is a dance that can be mastered for each business-use case. Through a proactive and highly collaborative team approach, critical business tasks can be solved in both a cost- and process-effective manner. Our next article will dig deeper into the skill each team member needs to have for us to get there.

Dan Panitz, UnitedLex, SVP, strategic accounts, is an experienced attorney based in New York with more than 25 years of combined legal, technology and corporate advisory experience. Having worked with SEC enforcement and NASD (now FINRA) arbitration, he also holds anti-bribery and Corruption specialty certifications for the PRC, U.K and the United States.

Bruce (HB) Gordon is the manager of e-discovery in the Office of the General Counsel for The Vanguard Group. Gordon's career spans over 20 years of ESI response management and as an IT manager/liaison to legal departments including Teva Pharmaceuticals, AmerisourceBergen Corp. and the Rohm and Haas Co.