Artificial intelligence beats human lawyers at lawyering.

That was a headline last week following the release of a report by a company called LawGeex, which recently hired a chief legal strategist and also put 20 human lawyers up against its AI software in a challenge to read and review nondisclosure agreements.

The robot, as it were, significantly outperformed the humans. While the lawyers took an average of 92 minutes to review five NDAs taken from a collection of old Enron Corp. documents, the AI took 26 seconds. The law school-trained individuals were 85 percent accurate in identifying different sections and provisions of the NDAs. The AI? Ninety-four percent accurate.

So, if robots can do it faster and better than humans: Will they?

The American Lawyer caught up with Shmuli Goldberg, a vice president of marketing for LawGeex, to discuss the competition that his company put together and what will happen to lawyers if they adopt what he calls “spell-check for lawyers.”

The American Lawyer: How did this test come about?

Goldberg: We've been wondering about this for about a year. About six months ago in our own tests our AI was performing better than a lot of the lawyers we were working with. We figured it'd be great if we could validate these results in a real-world test. We worked with several academics to develop it. We essentially worked with 20 experienced lawyers as well, all of whom had significant experience reviewing these types of contracts.

We set the identical test: We gave these lawyers five contracts each sourced from Enron. Our AI hadn't seen these before. We gave them to our lawyers and we said, “Hey, basic issue-spotting: Can you review the contract and highlight specific things.”

TAL: What did you find?

Goldberg: The interesting thing we found was twofold. First, our AI performs just as well in real-life situations as we've seen in the lab and with our customers. It's about 94 percent accurate. And lawyers themselves scored roughly 85 percent, which is in my opinion a very respectable score. But it also goes to highlighting the infallibility of human lawyers as much as the consistency and accuracy of an AI.

One thing we've always said to lawyers about the accuracy of our AI is: What's the baseline? What's the accuracy of human lawyers? We hadn't seen any real studies that showed what that baseline was. And this is something we offer ourselves. Once our AI finishes reading a contract, we'll run it past a human for a final set of eyes before we send it out.

While we intrinsically believe that humans make better lawyers than robots—and I think that's by the way still at true statement—there is still a myth of the infallibility of the human lawyer.

TAL: If software can do this type of work better than humans, do you expect lawyers will let them? Because even if humans make mistakes, we understand why they think or how they think. We don't necessarily know why AI systems make the decisions they do.

Goldberg: Some of these questions are philosophical and go down to the human condition. But start from the beginning: Are humans better than robots in this type of work? That for me, the key words in that sentence are this type of work. There is no question today with the technology we have, any complex legal matter should be handled by a human lawyer.

When you are drafting complicated contracts or putting legal arguments forward, there are good arguments to say the amount of creativity a human can bring to the table is going to significantly outperform a robot or AI. However, the study we did was on the exact opposite side of the spectrum. Looking through an NDA and issue-spotting requires little to no creativity. It's almost like the legal equivalent of a spell-check.

The AI was not told to offer solutions. It was not told to make recommendations when it found an issue. It was told: Read through this contract and tell us what's in it. Tasks like that, yes, robots have proven consistently that they can do better than humans. If you don't believe me, write a 10,000-page article in Microsoft Word and give it to a human to spell-check. We're talking about the most basic work that lawyers today do.

TAL: What about the trust issue?

Goldberg: I think in every industry over the last 30 or 40 years, there has been an inflection point where AI has taken over the humans. If you look at the travel industry, just look at autopilot compared to pilots. There is nobody now who says I don't trust the autopilot, and insist the pilot does it. Autopilot doesn't get drunk. They don't have a fight with their spouse the night before.

The same emotion behind it is true in the legal world. If I had one message to the legal world, based on a study like this, it's that these tools are not here to replace you or take your jobs the same way. No pilot has been replaced by autopilot. These are tools and software the same way Microsoft Word is a software tool. It's a tool in the lawyers' arsenal. Something they should be working with in order to outperform themselves five years ago. We're changing the fallible status quo for a better alternative.

TAL: A lot of people say that tools like this will take the sort of “grunt” work off the plate of lawyers so they can focus on high-value work. But is there enough high-value work for today's lawyers to be doing?

Goldberg: I don't see the amount of high-value work shrinking. History has shown the amount of high-value legal work will only go up. And there are two applications this type of study may have. The first is to allow smaller firms to utilize technology to take on lower value work as well. Instead of doing this themselves, they're using AIs that they manage to do their work. The same way a factory linesman now manages a lot of machines rather than standing there with a hammer and nail. That work gets more interesting and higher value.

The second thing is: If you look at the rise of [legal process outsourcing] over the last couple of years, it's pretty clear to me that the writing is on the wall when it comes to companies not wanting to outsource their lowest, everyday legal work to expensive top 100 law firms. Nobody with an efficiency mindset is going to a law firm and say, “Hey check this basic contract for me.”

All interviews are condensed and edited for style, grammar and clarity.

➤➤ Sign up here to receive The Law Firm Disrupted, a weekly email chronicling changes in the legal services industry.