The impact of a risk assessment tool can take on a whole new dimension when their scores are considered by judges in sentencing. But while the use of risk assessments in courts is mandatory in some states, most judges often have wide leeway in deciding how much, or how little, these scores factor into their decisions.

While some disregard or override scores entirely—rendering the risk assessment tools' impact almost nonexistent—others will use them for purposes for which they were not originally intended, which comes with its fair share of controversy and legal challenges.

To be sure, there are few, if any, judges who let a pretrial or risk and needs assessment (RNA) tool make a decision for them. "I think the biggest misconception is that the judge looks at that number and that number decides bail. That number does not decide anything, it's just more information for the judge to use," says Judge Mary Katherine Huffman, who sits on the Court of Common Plea in Montgomery County, Ohio. The court has used the Ohio Risk Assessment System (ORAS) tool for pretrial and sentencing decisions.

Assessment tools scores deciding a ruling is "exceedingly rare," adds Megan Stevenson, an assistant professor of law at George Mason University who has studied the impact of risk assessment tools in a handful of states. "That type of blind following is just not something that I have seen to be a phenomenon in criminal justice. It's way more common that you'll find the opposite—that judges will just ignore it."

In fact, in many states, including Colorado, New Jersey and New York, the use of an assessment tool is optional and up to the discretion of the judge or correctional officer. Still, in others like Ohio and Georgia, defendants or convicted offenders are required to be assessed at specific points of the criminal justice process. Judge Huffman, for instance, notes that the use of the ORAS Community Supervision Tool (CST) is mandatory for sentencing in Ohio.

But some tools, such as Georgia's The Next Generation Assessment (NGA), the Level of Service Inventory-Revised (LSI-R) and Level of Service/Case Management Inventory (LS/CMI), will come with built-in overrides that allow judges or correctional officers to change an assessment score. And there's a few reasons why that might be necessary.

Judge Huffman says that, based on her experience, scores sometimes aren't representative of a particular situation.

"Let's assume you have a 50-year-old man who is accused of raping of his neighbor and that50-year-old man has no criminal record. … His score is actually going to be very low," she explains.

Conversely, "let's assume you have that same 50 year old that had three felony convictions when he was 20 years old and they were all violent … He went to prison, no problems since then, and now he is charged with theft. Those old offenses might drive up his score. But then you look the current charge, theft [that's] nonviolent, and he hasn't had a conviction in 30 years—[the situation is] indicative of a high score not being terribly informative."

What's more, assessment risk scores can also be disproportionately influenced by a person's age. "One of things I think people have not realized is that young people get really high risk scores—you get a lot of points added to your risk score simply by being a teenager or young adult," Stevenson says. She adds, "A strict application of a risk assessment algorithm would lead to a really big incarnation rate for teenagers and adults."

Impact: Zero

Over the past few years, Stevenson has studied how judicial decisions in Kentucky and Virginia changed before and after both jurisdictions implemented a risk assessment tool. What she found was that judges didn't "use them a lot. … I would say it is just a supplement to what is still mostly a human process. The second thing I found, both in Kentucky and Virginia, is that it didn't bring about any big changes or unilateral advantages."

She explains after the implementation of the pretrial risk tool the Public Safety Assessment (PSA) in Kentucky, there was "a little bit of a reshuffling in terms of who got detained pretrial. There was an increase in the release rate for those rated low risk, a decrease in the release rate for those rated high risk."

However, "the net impact on incarceration was really actually small—only about a 4% point increase in releases, and that dissipated over time as judges returned to their previous bail setting habits," she adds. "To the extent that [some are] hoping that risk assessment use will lead to a real decline in pretrial detention, that's definitely not something that happens automatically.

There are, however, some success stories. New Jersey saw "really meaningful changes in their incarceration rate" after the implementation of the PSA," Stevenson says. But she notes that the pretrial risk assessment tool was part of a broad, comprehensive bail reform package where "they basically got rid of monetary bail [and] they implemented pretty stringent due process requirements before detaining someone pretrial." It's difficult therefore, to assess how much of an impact the PSA had outside of the broader reform.

Flexibility or Overreach?

While some judges disregard scores entirely, others will use them in ways for which they were not originally intended. The situation has been the focus of multiple legal battles across the U.S.

From late 2018 to mid 2019, for instance, the Iowa Supreme Court sidestepped the question of whether the consideration, in sentencing, of scores generated by the state's Iowa Risk Revised (IRR) assessment tool violated a defendant's due process. The IRR is primarily used by the Iowa departments of correction to determine convicted offender supervision levels.

In four cases during that time period (Iowa v. Gordon, Iowa v. Buesing, Iowa v. Guise, and Iowa v. Headley), the court said defendants did not have a due process claim because they failed to raise the issue at the district court trial, or object to the information in the presentence investigation report, which included IRR scores.

In a concurring opinion to Guise, however, Iowa state Supreme Court Justice Brent R. Appel, noted the ruling might have been different had counsel directly challenged the risk assessment.

"By way of example only," he wrote, "did the tool use arrests, or charges not resulting in conviction, as a factor in the calculation? If so, there would be a serious problem as there is authority for the proposition that a sentencing court may not consider charges that were dismissed or records of arrests without convictions."

Other states' Supreme Courts have also weighed in on the use of assessment tools in sentencing. In the 2010 Malenchik v. State ruling, the Indiana Supreme Court noted, "In considering and weighing aggravating and mitigating circumstances shown by other evidence, however, trial courts are encouraged to employ evidence-based offender assessment instruments," including the LSI-R (which the state used at the time) "as supplemental considerations in crafting a penal program tailored to each individual defendant."

In 2013, the Wisconsin's Supreme Court also ruled in Wisconsin v. Loomis that the COMPAS tool can be used in sentencing, though with certain limitations. Specifically, it noted that COMPAS "may not be considered as the determinative factor in deciding whether the offender can be supervised safely and effectively in the community."

The ruling echoes Equivant's own stance. Chris Kamin, interim general manager at Equivant notes the company, which owns and sells COMPAS, has "consistently promoted" the 2011 guidance from the National Center for State Courts (NCSC). Among other things, the guidance advises that risk and needs assessments should be used to "inform public safety considerations related to offender risk reduction and management," but should "not be used as an aggravating or mitigating factor in determining the severity of an offender's sanction."

Similarly, James Bonta, a consultant for corrections and criminal behavior who has worked with assessment tool developer Multi-Health Systems (MHS), says the manual for the LSI-R and LS/CMI makes clear the instruments "[are] not intended to be used to increase sentences. … Its purpose is to try and put people in the least restrictive environment." Still, he adds that "neither myself nor MHS can police these agencies to make sure that they are using these tools in the way they are intended to be." He also notes that the guidance is for "individual department of corrections."

Indeed, like the IRR, the LSI-R, LS/CMI and COMPAS were designed to help department of corrections craft and direct offender supervision and programming. While courts have upheld that some of these tools be tools' scores can be considered during sentencing, this strikes some as misguided.

Regarding the use of COMPAS in Wisconsin, "the whole controversy … unfortunately, [the tool was] being used to make an in or out decision, and that's not what it was designed to do," says Barbara Hankey, manager of Oakland County Community Corrections in Michigan. "It was designed to [determine], if someone is placed in community supervision, in probation—what are the things that need to be in place to help that person be successful?"

Still, some argue that the simplicity of many of these tools means there can be some flexibility with how their scores are applied. Stevenson says there is some validity to these type of concerns because tools are validated on a particular data set corresponding to a specific population.

But in her opinion, such concerns also "come from a misperception that these tools are more accurate than they are and that there's more to the prediction process than there is. These tools—you should think of them as really simple tools. Any decent tool is going to put a lot of weight on age, it's going to put a lot of weight on past offense, if they're trying to future likelihood of convictions, prior convictions are going to be important."

She adds, "If you think about a risk assessment tool as a very specific mechanical tool, like something that is designed to pull the transmission out, you wouldn't want to use that tool to also change the oil. Risk assessment is not like that. Think of it kind of like a sharp stick. That's the kind of level of tool we're talking about. No one would ever say that a sharp stick designed for digging holes can't also be used for scratching your back."