FORTUNE — One of the few things Mitt Romney and Barack Obama seem to agree on in this acid election season is that they want to rebuild the middle class. It has long been Washington policy dogma that one of the surest gateways to securing that dream is home ownership. But that may have to change. The bursting of the housing bubble wiped out many working class families that had thrown their savings into home purchases, pushing ownership rates to the lowest level in fifteen years.

Worse, the rate appears likely to continue dropping to levels not seen since the 1960s, in part because both parties, no matter how divided on other policies, advocate cutting government mortgage support to remove the risk of more bailouts, and possibly eliminating the mortgage interest tax deduction. That raises a thorny question: Can the government expand the middle class without more lawns to mow?

For some, the costs from the housing meltdown outweigh the idea that the government should support this ownership bridge into the middle class, at least as much as in the past. Some characterize the old policy as having turned into a trap that stripped working families of what little savings they had and ruined their credit.

MORE: Goldmanites fall out of the top 1%

“We destroyed the wealth of millions of households,” says David Stevens, who headed the Federal Housing Administration for the Obama administration and is now the president of the Mortgage Bankers Association, a trade group. “We over-promoted home ownership.” He says he does not believe there should be an ownership rate target but that people should just get used to entry being more restrictive.

That is already happening. After a decades-long rise in the ownership rate, from 62.9% in 1965 to 69% in 2005, according to the US Census Bureau, it dropped to 65.5% in the second quarter of this year.

Laurie Goodman, a senior managing director at Amherst Securities Group, says the figure should really be discounted even further; if you remove the 2.8 million borrowers who have not made a mortgage payment for over a year, meaning they are likely to lose their homes, the ownership rate is actually 63.3%, she says.

A closer look sheds an even harsher light on the decline and the policy implications. Hardest hit have been the types of households many politicians have most wanted to usher into the middle class, families with children and minority households.

MORE: Forget four years ago: We’re worse off than in 2011

The ownership rate for married couples with children plunged 5.1 percentage points from its peak, according to an annual survey by Harvard’s Joint Center for Housing Studies, compared with a 1.3 percentage point drop for married couples without children.

Minority families are finding this “gateway” pinched and perilous. In the second quarter of this year the home ownership rate for whites was 73.5%; for African Americans it stood at 43.8%, and 46.5% for Hispanics, according to the Census Bureau.

The rates for minorities would have tumbled even further had it not been for extraordinary government support. In 2004, 78% of black home buyers received conventional, private sector loans; the figure in 2010 was 19%, with the rest mostly relying on FHA or other government-guaranteed mortgages, Stevens says.

“I consider that unsustainable,” he says.

Officials of both parties have said they want to wind down and possibly eliminate the government-supported mortgage giants, Fannie Mae and Freddie Mac. Already, borrowers are required to have higher credit scores and records, one reason why extraordinarily low interest rates have not spurred much new buying.

MORE: Is Wall Street being bamboozled by Romney?

Reducing the federal subsidies would attract more private capital in theory, but it would also likely set even stricter lending standards, reducing mortgage availability, or at least making it more expensive and putting this traditional means of wealth building out of the reach of more families.

The demographic trends suggest that the ownership decline could be a long downward spiral. About two-thirds of net new household formation from 2007 to 2011 was among minorities, groups that have traditionally had much lower home ownership levels. The “echo boom” generation is also burdened with student loans, according to Goodman, which will postpone or eliminate ownership for many, and on top of that real incomes are flat, which is likely to prevent accumulating the savings required for down payments.

“People haven’t looked at the reality here,” Goodman says. “It will go lower.”

The ripples go beyond individual families and their class aspirations. People who own homes tend to not just have more money to draw on for their retirements, they rely less on federal programs, like Medicare, thus easing the fiscal pressure on the government, says Christopher Mayer, a Columbia Business School professor. “The point is, home ownership helps the taxpayer, not just the retirees.”

MORE: Stop beating up the Rich

Robert Edelstein, a professor at Berkeley’s Haas School of Business, warns that any hasty reduction of the government’s support could not only remove one of the rungs of the financial ladder for aspiring low income families, it could send a fragile economy back into recession by sending prices down again. He insisted that the country does benefit from prudent policies that help lower income families buy homes. The trick, he says, is to focus on needy households and deny the subsidies to the wealthy.

“You can argue that the wealthy don’t need it, but this is a real way that lower income families can accumulate wealth and move up,” he says. “Don’t get rid of the policy, just focus it.”