A federal judge has remanded the case of a national mortgage servicer who might have taken too long to remove a case to federal court.