The Dark Side of AI in Law

Pictured, from left, are John Simek, VP of Sensei Enterprises; Ed Walters, CEO of Fastcase; Sharon Nelson, president of Sensei Enterprises; and moderator, John Mitchell. Photo by Gabrielle Hernandez/ALM.

Before artificial intelligence was the legal technology buzzword of the season, it was the stuff of science-fiction nightmares. Stories about artificial intelligence “gone rogue” proliferate in popular culture, from “Terminator” to “I, Robot” to “2001: A Space Odyssey.”

Although these stories are all fictional and most have no basis in reality, significant ethical considerations around AI remain, and perhaps deserve greater space in the conversation around the use of AI in the legal industry. Panelists John Simek and Sharon Nelson, respectively the vice president and president of Sensei Enterprises Inc., and Ed Walters, CEO of legal research company Fastcase, closed out the College of Law Practice Management's Futures Conference, “Running With the Machines: Artificial Intelligence in the Practice of Law” with a discussion of some of these concerns.

Walters used some of the ethical dilemmas highlighted by self-driving cars to highlight what the role of law could potentially be in regulating AI. Self-driving cars could theoretically at some point be forced to evaluate whose safety to prioritize in a collision—do you crash into a pedestrian, or the building that could have people?—which raises questions about how law could and should regulate those decisions.

What's more is that AI gets its decision-making capacity from past data, meaning that pre-existing biases in law or data sets could simply be reproduced in new ways. Echoing concerns about “garbage in, garbage out” that technologists often use to explain “training” data sets, Walters said, “Data validity and authentication is extremely, extremely important.”

That said, cleaning the data that AI draws from isn't always clear-cut. “Algorithmic bias is not a binary thing. You'll always be trading off completeness versus bias,” Walters said.

Simek addressed how these concerns may play out in cybersecurity, especially in parsing through the vast amount of data needed to successfully diagnose a particular cybersecurity vulnerability. “What AI is helping with in that arena is to sort of collaborate that information and then bubble it up to the surface so that human beings can deal with it,” he said. Allowing AI to automatically respond to a cyberevent or create its own fix could create a whole other set of issues, Simek cautioned.

Establishing policy or law around AI, especially when it has tangible consequences around human safety, is more easily said than done. “There's very little law that governs what that machine has to do. It's very hard to choose the right legal regime for these algorithms,” Walters said. “The laws that regulate us don't map very well onto these systems,” he later added.

As with many issues in technology and law, the technology has outstripped regulation, something Walters thinks is likely cause for concern. “We're nowhere near ready to deal with these issues, but the cars are here.”

Nelson raised concerns about the ways in which AI are potentially making their decisions outside the purview of human oversight. Citing a recent incident in which Facebook pulled the plug on an AI project that had created its own language unintelligible to humans, Nelson cautioned that oversight begins with understandings the internal thinking of machines. “Transparency is the one thing we must have. If they are hiding from us what they're doing, I begin to worry.”

The panel also raised recent examples from Saudi Arabia and Estonia granting machines some version of citizenship or personhood as a means of regulating machines. While it can sound absurd to give machines rights designed for humans, Walters said it may not be too far outside the purview of existing law. “There is a long strain of American law about artificial persons. We grant corporations all kinds of rights all the time,” Walters explained. “There are artificial peoples we grant rights in order to do things more efficiently. Machines can do things a certain way if you grant them certain rights.”

Nelson pointed to the Trump administration's current tendency toward deregulation to offer a reality check around regulatory possibilities for AI. “Regulation itself is a possible solution, but we live in an environment today where deregulation is the rule,” she said.

Walters attempted to find middle ground on AI between the alarmism of science fiction and the excitement from technologists by urging safe practice. “We're playing with fire. When you're playing with fire, when you're working with fire, you have to be humble. You have to understand it; you have to be careful. You have to understand the consequences as well,” he said.

Pictured, from left, are John Simek, VP of Sensei Enterprises; Ed Walters, CEO of Fastcase; Sharon Nelson, president of Sensei Enterprises; and moderator, John Mitchell. Photo by Gabrielle Hernandez/ALM.

Before artificial intelligence was the legal technology buzzword of the season, it was the stuff of science-fiction nightmares. Stories about artificial intelligence “gone rogue” proliferate in popular culture, from “Terminator” to “I, Robot” to “2001: A Space Odyssey.”

Although these stories are all fictional and most have no basis in reality, significant ethical considerations around AI remain, and perhaps deserve greater space in the conversation around the use of AI in the legal industry. Panelists John Simek and Sharon Nelson, respectively the vice president and president of Sensei Enterprises Inc., and Ed Walters, CEO of legal research company Fastcase, closed out the College of Law Practice Management's Futures Conference, “Running With the Machines: Artificial Intelligence in the Practice of Law” with a discussion of some of these concerns.

Walters used some of the ethical dilemmas highlighted by self-driving cars to highlight what the role of law could potentially be in regulating AI. Self-driving cars could theoretically at some point be forced to evaluate whose safety to prioritize in a collision—do you crash into a pedestrian, or the building that could have people?—which raises questions about how law could and should regulate those decisions.

What's more is that AI gets its decision-making capacity from past data, meaning that pre-existing biases in law or data sets could simply be reproduced in new ways. Echoing concerns about “garbage in, garbage out” that technologists often use to explain “training” data sets, Walters said, “Data validity and authentication is extremely, extremely important.”

That said, cleaning the data that AI draws from isn't always clear-cut. “Algorithmic bias is not a binary thing. You'll always be trading off completeness versus bias,” Walters said.

Simek addressed how these concerns may play out in cybersecurity, especially in parsing through the vast amount of data needed to successfully diagnose a particular cybersecurity vulnerability. “What AI is helping with in that arena is to sort of collaborate that information and then bubble it up to the surface so that human beings can deal with it,” he said. Allowing AI to automatically respond to a cyberevent or create its own fix could create a whole other set of issues, Simek cautioned.

Establishing policy or law around AI, especially when it has tangible consequences around human safety, is more easily said than done. “There's very little law that governs what that machine has to do. It's very hard to choose the right legal regime for these algorithms,” Walters said. “The laws that regulate us don't map very well onto these systems,” he later added.

As with many issues in technology and law, the technology has outstripped regulation, something Walters thinks is likely cause for concern. “We're nowhere near ready to deal with these issues, but the cars are here.”

Nelson raised concerns about the ways in which AI are potentially making their decisions outside the purview of human oversight. Citing a recent incident in which Facebook pulled the plug on an AI project that had created its own language unintelligible to humans, Nelson cautioned that oversight begins with understandings the internal thinking of machines. “Transparency is the one thing we must have. If they are hiding from us what they're doing, I begin to worry.”

The panel also raised recent examples from Saudi Arabia and Estonia granting machines some version of citizenship or personhood as a means of regulating machines. While it can sound absurd to give machines rights designed for humans, Walters said it may not be too far outside the purview of existing law. “There is a long strain of American law about artificial persons. We grant corporations all kinds of rights all the time,” Walters explained. “There are artificial peoples we grant rights in order to do things more efficiently. Machines can do things a certain way if you grant them certain rights.”

Nelson pointed to the Trump administration's current tendency toward deregulation to offer a reality check around regulatory possibilities for AI. “Regulation itself is a possible solution, but we live in an environment today where deregulation is the rule,” she said.

Walters attempted to find middle ground on AI between the alarmism of science fiction and the excitement from technologists by urging safe practice. “We're playing with fire. When you're playing with fire, when you're working with fire, you have to be humble. You have to understand it; you have to be careful. You have to understand the consequences as well,” he said.