“Inversion,” of course, is a euphemism for “desertion.” It happens when a U.S. corporation takes over a foreign company and then—for tax purposes only—pretends to be based in the taken-over company’s country. That allows the U.S. company to pay lower income tax rates—such as 12.5% (or even less) in Ireland rather than 35% in the U.S.

Inverters and would-be inverters say their U.S. tax rate will remain the same, which is true. What they don’t say is that inversion makes their taxable U.S. income lower than it would otherwise be. Inverted companies reduce their federal and state income tax bills by siphoning income out of the U.S. in a variety of ways, but they continue to be run from here and benefit from what our country has to offer: deep and liquid financial markets, rule of law, intellectual infrastructure, great places to live, and military protection. Repellent, at least to me.

But today let’s not get into questions of morality and social responsibility, or whether proposals by the Treasury or Sen. Chuck Schumer (D-N.Y.) to rein in inversions will work. Instead, let me show you how regular people—especially prudent, long-term retail investors—pick up the tab for Wall Street’s inversion feast.

Wall Street firms get rich advisory fees and financing fees from inversions. Hedge funds, takeover trolls (who call themselves “activist investors”), and other institutional investors—all part of Wall Street—can get a quick boost to their investment performance, which helps them generate bigger fees and attract more investor money.

Board members and top executives of corporations that succumb to inversion mania get rich subsidies from the firms’ shareholders, who typically give them special payments to cover the excise taxes they owe on their restricted shares and options when the inversion happens. For example, Medtronic, a big medical-device company that wants to invert, estimates that it will give $63 million of nondeductible payments to its top executives and directors to cover their excise taxes and the taxes they will owe on their excise tax subsidy.

Meanwhile, retail investors who hold Medtronic in non-retirement accounts will pay taxes on shares they own directly, and will pay taxes indirectly on shares owned by mutual funds they hold.

Here’s why. Although a company pays no exit tax when it moves its tax domicile, tax law requires shareholders to pay capital gains tax as if the company were sold for cash at its market price on Inversion Day. (This is an anti-inversion regulation that hasn’t worked at all.) The holders continue to own their Medtronic shares but have to dig into their own pockets to pay the tax.

That can be serious money. For example, a high-bracket Minnesota resident who has owned shares of Minneapolis-based Medtronic for 20 years would have to pay about $20 a share in federal and state tax if the company inverts at current prices. But such taxes don’t affect mutual fund and hedge fund managers, who are judged on pretax performance and who pass on tax obligations to their investors.

“It’s one thing for investors to have to pay tax if a fund manager is selling a stock at a profit, but it’s a whole other thing to have to pay on a stock the fund is keeping,” says Daniel P. Wiener, editor of a newsletter for Vanguard investors. The same, of course, is true for people who own stock directly.

Once again, Wall Street gets its way, while responsible Main Street folks get screwed. Let’s stop this inversion farce now, fix the damn corporate tax code, and get back to the business of growing our economy and improving our lives.

See our continued coverage of corporate tax inversions at Fortune.com

This story is from the October 6, 2014 issue of Fortune.