A new housing development in Phoenix

FORTUNE — In his annual letter to Berkshire Hathaway brk.a shareholders, which was released on Saturday, Buffett says he still believes a real estate recovery is on the horizon. Yet, the Oracle of Omaha admits that on housing his crystal ball has been cloudy. Last year, Buffett said the real estate market would recover in 2011 or early 2012. While home sales have risen recently, it would be a stretch to call the past year a recovery, and Buffett doesn’t try. “I was dead wrong,” he writes.

Nonetheless, Buffett says a turnaround for housing is in the works. Buffett’s argument mostly rests on demographics and, in part, on sex. He says the housing bust was created by the fact that home builders put up houses faster than there were new families to fill them. Recently, that trend has reversed.

“People may postpone hitching up during uncertain times, but eventually hormones take over,” he writes. “And while ‘doubling-up’ may be the initial reaction of some during a recession, living with in-laws can quickly lose its allure.”

MORE: Buffett beats the S&P for the 39th year

On Friday, the Census Bureau reported that home builders are on pace to add just under 700,000 housing units a year. Buffett says in normal times we need about 1 million or more homes to keep up with household formation. Housing economists have been talking for a while about how rising household formation and the relatively slow pace of construction of new homes could lead to a recovery. “The prospects for household formation are stronger than people think,” says housing economist Thomas A. Lawler.

The question is when the number of buyers will catch up with the supply of houses already on the market, many of which are sitting vacant because of foreclosures. Buffett seems to think that the oversupply will be exhausted soon. When that happens, Buffett believes we could see a rebound in the economy as well. In fact, Buffett says the weak housing market is a major reason the economy has created so few new jobs since the end of the recession.

As a result, many have said that Washington and the Federal Reserve should do more to try to boost the housing market. Buffett disagrees. He says new stimulus measure probably won’t work, and anyway aren’t needed. “Fortunately, demographics and our market system will restore the needed balance [in the housing market] – probably before long,” writes Buffett.

More on Berkshire’s letter:

Don’t believe every “Buffett buys…” headline

Geico’s profits plunged in 2011

Warren Buffett: Why stocks beat gold and bonds