The U.S. Court of Appeals for the Second Circuit said Monday that it would not lift state prison restrictions that a group of Quaker inmates argued had interfered with the practice of their religion.