Reform always brings unintended consequences, and Washington’s efforts to protect retail banking customers are no exception. Banks have responded to new restrictions by boosting checking account fees, prompting millions of customers to flee. That means trouble for banks — but also new opportunities for Wal-Mart, American Express, and others that see a chance to benefit from what analyst Meredith Whitney calls the “de-banking” of America.

The biggest catalyst for much of this movement has been the Durbin Amendment to last year’s Dodd-Frank financial reform bill. As of July 21, it will reduce the amount banks can charge merchants as “interchange” fees for debit cards from an estimated 44¢ per transaction to 21¢, a move that Brian Foran, a banking analyst at Nomura Securities, thinks will cost the industry $5 billion. The Federal Reserve last year also blocked banks from charging overdraft fees on debit cards unless the customer “opts in.”

As a result, most checking accounts are no longer profitable for larger banks. To make up for the lost income, they are imposing monthly account charges. A recent survey of the top 25 banks showed that the average monthly fee is now $4, and Foran expects that to hit $8 within a year.

Those fees, in turn, are causing customers to leave. Mike Moebs, CEO of Moebs Services, a Lake Bluff, Ill., financial services research firm, estimates that 4 million customers left the biggest 30 banks last year because of fees; an additional 11 million are expected to bolt this year.

Where are they going? Many are headed to local banks and credit unions, which don’t charge for checking. But others are choosing prepaid debit cards. Research firm Mercator Advisory Group says that prepaid charges should hit $201 billion by 2013, up from just $28 billion in 2009.

So far the prepaid market leader has been Green Dot Corp., based in Monrovia, Calif., and 9% owned by Wal-Mart (WMT). But others are getting in too: On June 13, American Express (AXP) announced a no-fee prepaid card that Mercator senior analyst Brent Watters calls a “game changer.” (Green Dot and others charge maintenance and reload fees, prompting Florida’s attorney general to investigate whether the industry imposes hidden charges. Green Dot says its fees are among the industry’s lowest and that it is cooperating with the investigation.)

Former bank customers are also heading in record numbers to payday lenders. Dan Feehan, CEO of Cash America International, says business at the company’s online lending site has soared. Profits at the firm were $115 million in 2010, up about 20% over 2009, and the company’s stock is at an all-time high.

Ironically, the same banks that are booting customers by raising fees are trying to push into these new profit centers as well. BB&T (BBT), of Winston-Salem, N.C., has just issued its own prepaid card, and Wells Fargo (WFC) is offering payday loans at below-market prices. They may think their former customers won’t hold a grudge.

This article is from the July 25, 2011 issue of Fortune.