Credit: Denis Simonov/Shutterstock.com

Lawyers from leading international law firms, including Allen & Overy and DLA Piper, have been beaten in an accuracy contest by artificial intelligence (AI) software developed by students.

More than 100 partners and associates took part in the weeklong competition against CaseCrunch, an AI startup founded by law students at the University of Cambridge.

The lawyers had to assess almost 800 historic insurance misselling claims and predict whether the cases succeeded. The AI predicted the outcomes with 87 percent accuracy, compared to 62 percent for the lawyers.

Rebecca Agliolo, CaseCrunch's marketing director and one of the company's co-founders, said it was the first-ever competition to directly pit lawyers against AI software.

The contest was judged by Ian Dodd, a director at legal analytics company Premonition, and Felix Steffek, a senior lecturer of law at Cambridge. Dodd said that anyone who thought the challenge was a gimmick “might like to think again.”

CaseCrunch scientific director Ludwig Bull warned against reading too much into the flesh-and-blood lawyers' defeat.

“These results do not mean that machines are generally better at predicting outcomes than human lawyers,” he said. “These results show that if the question is defined precisely, machines are able to compete with and sometimes outperform human lawyers.”

Along with lawyers from DLA Piper and Allen & Overy, the contest included participants from Eversheds Sutherland, Pinsent Masons, Berwin Leighton Paisner, Bird & Bird, DAC Beachcroft and Kennedys, among others.

CaseCrunch managing director Jozef Maruscak said the company would now market its services to clients, but stressed that lawyers and AI software providers are “not necessarily adversaries.” He added: “Systems like ours can make the legal world more effective for everyone.”

Credit: Denis Simonov/Shutterstock.com

Lawyers from leading international law firms, including Allen & Overy and DLA Piper, have been beaten in an accuracy contest by artificial intelligence (AI) software developed by students.

More than 100 partners and associates took part in the weeklong competition against CaseCrunch, an AI startup founded by law students at the University of Cambridge.

The lawyers had to assess almost 800 historic insurance misselling claims and predict whether the cases succeeded. The AI predicted the outcomes with 87 percent accuracy, compared to 62 percent for the lawyers.

Rebecca Agliolo, CaseCrunch's marketing director and one of the company's co-founders, said it was the first-ever competition to directly pit lawyers against AI software.

The contest was judged by Ian Dodd, a director at legal analytics company Premonition, and Felix Steffek, a senior lecturer of law at Cambridge. Dodd said that anyone who thought the challenge was a gimmick “might like to think again.”

CaseCrunch scientific director Ludwig Bull warned against reading too much into the flesh-and-blood lawyers' defeat.

“These results do not mean that machines are generally better at predicting outcomes than human lawyers,” he said. “These results show that if the question is defined precisely, machines are able to compete with and sometimes outperform human lawyers.”

Along with lawyers from DLA Piper and Allen & Overy, the contest included participants from Eversheds Sutherland, Pinsent Masons, Berwin Leighton Paisner, Bird & Bird, DAC Beachcroft and Kennedys, among others.

CaseCrunch managing director Jozef Maruscak said the company would now market its services to clients, but stressed that lawyers and AI software providers are “not necessarily adversaries.” He added: “Systems like ours can make the legal world more effective for everyone.”