artificial intelligence touch

Legal departments are more focused than ever on finding ways to efficiently manage their contracts. At scale, contract generation and formation are the easy part. Mature forms and established workflows underpin a playbooked process augmented with automation (document assembly, escalation triggers, e-signatures). Often, the real challenge is organizing executed contracts at scale and extracting actionable information.

For example, it is labor-intensive to identify all contracts with pending expiration dates if all you have is a file folder containing nothing but scanned contracts. These files are “unstructured.” Someone would have to open and read each contract to determine if it has expired. We “structure” data by populating fields in a database with material information. In this simple example, a contract would be linked to a database containing a field for end date. We accomplish this by abstracting the end date from the individual contract and populating the database field with the date. Once populated, the field enables us to quickly sort and filter contracts by end date and generate reports on pending expirations.

But end date is only one among a myriad of potentially material data points. These data points can range from binary questions, like whether the contract contains a most favored nation provision, to verbatim extraction of text (i.e., if the contract does have a most favored nation provision, the entire provision is copied into the database where it can be searched, filtered and analyzed without needing to open and review the linked contract). Companies look to structure their contracts in this way to help make data-driven business and legal decisions, such as determining when revenue contracts are up for renewal or renegotiation, identifying contracts that need to be amended in response to a regulatory event (e.g., GDPR, Brexit) or completing due diligence in advance of an M&A transaction.

But while moving from unstructured to structured contract data can be quite useful, it is also extremely time-consuming (read: slow and expensive) depending on contract volumes and the kind of questions that the database needs to answer. Technology is a component of the long-term response to the industry's challenge. In the past few years, legal tech companies have brought to market many technologies that streamline, supplement and leverage what humans currently do for contract abstraction, using approaches like natural language processing and machine learning that fall under the umbrella of artificial intelligence. The dream is an electronic brain that can ingest contracts in seconds and subsequently answer any question posed—like Watson on Jeopardy! In the real world, however, these products are often hard to differentiate, especially through the fog of marketing hyperbole and the attendant hype hangover.

QuisLex undertook a study for a deep dive on leading AI tools built specifically to abstract text from contracts, with an expectation that introducing automated contract extraction technology would add value by reducing cost while improving speed and accuracy. Setting up a formal project internally and running tests the same way as for any live client project, the study looked at things like differences in functionality between tools, time savings from completing an AI-enabled review versus a manual review, and accuracy rates of the AI models to see how often users could rely on their results.

The findings:

1. Time savings are real, but lawyers are still needed. You can materially improve human performance by integrating these tools into your contract workflows. Time savings averaged 28 percent in our proof of concept. But the study also safely concludes that legal professionals will not be replaced en masse by robots anytime soon.

2. Cost and time savings vary. Time savings and accuracy had a high degree of variance, not just from tool to tool but within tools, from one agreement to the next and from one provision to the next—including some instances where using the tool took more time than manual review. The usefulness of each tool is dependent on the tool/project fit and the process built to support the workflow. Cost savings and ROI will also vary depending on the cost of the resource whose time is being saved. Implied cost savings for a 10,000 contract review project ranged from $33,000 to upward of $1.7M depending on whose manual review time was being reduced.

3. Training AI models is a skill. Building models to cover new concepts or to target specific agreement sets involves thinking through how you want the model to perform to ensure you are giving it the right training data and building a process to ensure that only the right training data gets added to the model.

4. Tools will get even better. Improvement opportunities exist, including for handling of amendments and functionality to better support the entire workflow.

All caveats aside, QuisLex has integrated AI-based abstraction tools into several of our workflows. We can think of no better vote of confidence for where these tools are, and where they are going, than altering our time-tested workflows at risk to our wallet and our reputation.

Chase D'Agostino is Associate Vice President of Corporate Solutions at QuisLex, a leading legal services provider. For the full details of QuisLex case study, click here.