Scott Reents of Cravath, Swaine & Moore

ALM's annual Legalweek conference kicks off next week in New York, and one of the major themes at the event is how artificial intelligence and other technologies are changing legal practice. In this excerpt from the latest episode of Law.com's “Unprecedented” podcast, Legalweek speaker Scott Reents—who heads Cravath, Swaine & Moore's e-discovery and data analytics practice—talks about the challenges and benefits of implementing machine learning and analytics, and why the problem of sifting through big data won't be easy to solve.

This transcript has been edited for clarity and length. Listen to the full interview here, or through Apple Podcasts, Google Play, or Libsyn. You can catch Reents at Legalweek on Jan. 31 at the Fireside Chat session.

Ben Hancock: I've heard that one challenge in implementing AI in a legal context is having the technology explain itself and being able to follow the “decision tree,” as it's called. Talk to me a little about that.

Scott Reents: You've put your finger on what I think is a true challenge for data analytics and machine learning, which is that machine learning almost definitionally creates models of the world that are difficult to explain. That's why these are models of the world that are being put together by a computer and not by a human. If it were a simple model of the world, we might be able to do it without computer intervention. But the computer allows us to build much more sophisticated, complex and therefore—hopefully—accurate models of the world. But one of the costs is the transparency of it and the explainability of it.

I think what you need to do to sort of accommodate that is to bring to your analytics practice a robust validation methodology, and you need to prioritize that. While you may not be able to say—in the e-discovery context—”We identified these thousand documents as the documents that are relevant,” you will be able to say, “We've done rigorous statistical testing and determined that across these various criteria we have found 90-95 percent of the relevant material.”

Another big theme at the Legalweek conference is the increasing amount of data that lawyers are having to deal with from different sources, such as Internet-of-Things devices and all the different messaging services. We're all generating huge amounts of data, which is more evidence to sift through.

That's absolutely right. And one of the dynamics that's sort of ironic is, there's this concept of the rebound effect. It's a concept from environmental economics, which says that when you make a device or technology more energy-efficient, you're unlikely to reap all of the gains from that energy efficiency. Take energy-efficient lightbulbs, for example. You make it cheaper to run any given lightbulb for an hour, and people are going to leave the lights on longer, [or] people may install more lights.

There's an analogy with law and our data processing needs, which is that as we make it cheaper to understand and process more of this information, it's probably just going to increase our appetite for this information. And we may not end up “solving” the e-discovery problem.

Aside from the explainability issue, what are some of the other challenges to implementing AI and other technologies in legal practice?

It's really about developing the pathways and the methods for using it. For most technologies and most applications—especially with our clients' data—there's no “easy button.” The data is different in every case, you have different needs. It needs to be cleaned up and organized and you need to try different methods. And so if your expectation going in is, “Take this huge mass of data, put it into the computer, get the answer,” you're probably going to be disappointed. You need to spend the time to understand it, develop the methodologies, and develop the modes of communication between the technologists and the lawyers so that you're actually doing the work that's going to serve the legal interest.