While financial institutions are finally beginning to bounce back from the economic downtown, some banks can stand to be a little more cautious when it comes to how they spend their money.

Take the Barclays for example, the second largest bank in the U.K. that was charged 330 million pounds (almost $450 million) for regulatory penalties and lawsuits in Q4 of 2013. According to a recent Bloomberg Business Week report, regulators are investigating Barclays for the possible manipulation of foreign-exchange markets and after the bank is reviewing its trading over a “several-year” period.

Barclays CEO Antony Jenkins has taken responsibility for the carelessness spending of the bank and said he will forego his 2013 bonus, acknowledging that regulatory penalties and lawsuits have continued to impose costs on the bank after it raised 5.8 billion pounds ($9.5 billion) from shareholders.

“I am aware of the very significant costs which have been required to address legacy litigation and conduct issues in 2013, as well as to exit assets and businesses we no longer wish to participate in,” Jenkins said in a statement.

Jenkins isn't the first leader of a bank to take the fall for negative implications on spending in a financial institution. Just this past August, Ross McEwan, CEO of The Royal Bank of Scotland Group Plc, said he wouldn't take a bonus for 2013, as the U.K. government-owned lender faces its biggest pretax loss since 2008. On the other hand, JPMorgan Chase CEO Jamie Dimon was granted a 74 percent raise to $20 million last year, bringing his pay closer to where it stood before he was penalized for faulty oversight of botched derivatives bets.

Last year, Jenkins noted that he is working to remove 1.7 billion pounds of annual costs by 2015. One strategy that has become apparent in the last couple of months is by eliminating jobs at its corporate bank. The lender plans to cut hundreds of jobs including directors and managing directors at its investment bank in order to cut back on costs this year.

According to Business Week, Barclays is also being probed over whether it properly disclosed 322 million pounds of payments to Qatar's sovereign wealth fund as part of a 7 billion-pound fundraising during the financial crisis. The bank was fined 290 million pounds for Libor manipulation in 2012, which led to the departure of then-CEO, Robert Diamond.

For the latest legal reports in the financial industry, read these related stories: