Statues and busts of the 19th century U.S. Chief Justice Roger Taney have been taken down recently in his native Maryland, but at the U.S. Supreme Court, depictions of the author of the notorious Dred Scott decision are still visible—and not likely to disappear anytime soon.

Before and after the Charlottesville, Virginia, violence this month, Taney has been lumped together with Confederate generals and leaders as symbols of racism and slavery. Taney statues were removed this month from public places in Annapolis and Baltimore, while a bust was removed from Frederick, Maryland.

In the 1857 ruling, Taney wrote in Scott v. Sandford for a 7-2 majority that African-Americans were not citizens, and had been viewed as “so far inferior, that they had no rights which the white man was bound to respect; and that the negro might justly and lawfully be reduced to slavery for his benefit.”