A federal appeals court upheld the U.S. Securities and Exchange Commission's decision to deny a Bear Stearns broker a whistleblower award.