Warren Buffett, the world’s most admired billionaire, faced a lot of criticism on his home turf at this year’s annual meeting of Berkshire Hathaway BRK.A . Some of it was self-inflicted.

Buffett said that he wouldn’t have predicted that interest rates could have stayed this low for this long without a problem. “So far, I have been wrong on interest rates,” said Buffett. “It is so hard for me to believe that you can drop money from a helicopter and not have inflation, but we haven’t.”

Two years ago, the Berkshire CEO said he was worried about the Federal Reserve’s efforts to stimulate the economy. In particular, Buffett warned that the end of the Fed’s so-called quantitative easing program, in which the U.S. central bank bought billions in bonds to drive down interest rates, would end badly.

It hasn’t. The Fed stopped adding to its bond portfolio in the past year, though it still owns a lot of bonds, and the market and the economy have continued to hum along. The European Central Bank is now doing its own form of quantitative easing.

Shareholders hit Buffett with questions about predatory lending practices at Berkshire-owned Clayton Homes; his partnership with 3G Capital, which is notorious for its layoffs and cost cutting; and the fact that many of his investments are in companies that produce products that are high in processed sugars, like Coke and ketchup, at a time when many are worried about obesity.

Buffett was also pressed on his investment in IBM IBM , which has recently disappointed investors. Buffett’s chief lieutenant and investment partner Charlie Munger responded that he believes IBM is a wonderful company and that, much like every other firm, some of Berkshire’s investments have “reversals.”

Buffett often faces criticism at his annual meeting. But this year seemed more contentious than usual, and more of the heat was directed at Berkshire’s operations. Why? In the past decade, Berkshire has morphed from an insurance/investing vehicle for Buffett to a major conglomerate that owns everything from Dairy Queen to Fruit of the Loom to giant railroad company BNSF. Investors may have more angst about Berkshire’s sprawling growth. If you are Buffett, and you and your companies make some mistakes here and there, you can get away with it. But now that Buffett is 85, some shareholders are likely worried if the company will be viewed as favorably after he is gone.

On predatory lending, Buffett said an investigation of his manufacturing housing business Clayton Homes revealed “many mistakes.” He said Berkshire had no incentive to make loans to people who couldn’t repay them. Buffett said he was proud of the many people who have been able to buy a home with Berkshire’s financial assistance, and that the default rates on its loans during the financial crisis were lower than those of other banks.

On 3G, which Buffett was pressed on more than once, he said that the Brazilian partnership has bought some companies that have more employees than it needed. He also said that 3G has done quite well cutting costs including at Heinz, which it bought with Berkshire. He further commented that no company should be run with more employees than it needs.

On contributing to the obesity epidemic, Buffett mostly fielded the question with zingers. Buffett cited his own diet, which is famously horrible, and said that he sees few people smiling at Whole Foods. The line got a laugh from a crowd that seemed more wary of Berkshire than in the past but was still in love with Buffett.