Conservation groups filed suit Tuesday against a U.S. government agency challenging its decision to allow a mining project to move forward without federal permits near the edge of the Okefenokee Swamp's vast wildlife refuge.