Bill Miller had a great run. But did his investors?

December 7, 2011, 3:00 PM UTC

Legg Mason’s superstar is closing out a legendary career. Too bad his investors can’t afford to call it quits.

FORTUNE — There are certain stock market lessons that most investors never learn. Among the biggest is “Beware of throwing money at a popular fund.” That’s one thing we should all take away from the career of Bill Miller, once the mutual fund industry’s brightest star. Miller, who announced recently that he’s stepping down next April, became a star money manager as his Legg Mason Capital Management Value Trust (LMVTX) outperformed the Standard & Poor’s 500 (SPX) for an astounding 15 straight years. The longevity of that streak, which ran from 1991 through 2005, will probably never be matched. It was a fabulous performance.

So what’s the problem? It’s that Miller’s fund attracted so much money as his fame grew that many investors bought in late, as I’ll show you, and missed much of the upward ride. People usually look only at funds’ average annual return, and Miller’s was excellent. But you can also calculate the fund’s average investor’s return, which is often much lower. For instance, if a fund rises 50% in a year that it had $100 million of assets (making $50 million for investors) but falls 10% when it had $1 billion (losing $100 million), the fund is way ahead — up 40% — but its investors have lost money.

Morningstar crunched Miller’s numbers for me, showing that his average investor had a considerably lower return than the fund posted during his long hot streak. The fund made 16.44% a year in gains and reinvested dividends during that period, but the average investor made only 11.34%. Miller’s average investor actually underperformed the S&P (which returned 11.51% annually during his streak), even though his fund way outperformed the index. (See the table at the bottom for all the numbers.) “It’s human nature for investors to act this way,” says Don Phillips, Morningstar’s president of fund research. “When stocks are popular and the market is rising, everyone wants to invest.” Then, when the market hits a bad patch, many fund investors sell near the bottom, giving them the worst of both worlds: buying high and selling low.

Interestingly, the presumably non-starstruck investors in Vanguard’s plain-vanilla S&P 500 index fund (VFINX), which I also asked Morningstar to analyze, fell into the same trap. During Miller’s 15-year hot streak, the index fund returned 11.41% — but its average investor made only 7.96%. That’s because money flooded into the fund in 1999 as it neared its high, and flooded out near the market bottom in 2002. In other words, index investors weren’t any more disciplined than Miller’s investors. Quips Phillips: “Indexing is just a lower-cost way of producing a bad investor experience.”

When it comes to bad experiences, it’s hard to top what has happened to Miller’s investors since his hot streak ended in 2005. From Jan. 1, 2006, through this past Oct. 30, the last date for which average-investor results are available, his fund lost 7.40% a year; his average investor, buying high and selling low, lost 8.31%. By contrast, the average Vanguard index investor made 2.52%.

Thanks to the six-year underperformance, the return of the average Miller investor from the start of his streak in 1991 through October has fallen below the average index investor’s return: 6.06% to 6.61%. The reason: fees. Legg Mason’s management and marketing fees totaled more than 1.6% a year during this period; Vanguard, by contrast, has no marketing fee, and its management fee ranged from 0.15% to 0.20%. That cost disparity far exceeds the performance differential.

Just to let you know, I’m not immune to chasing the hot hand. From 2004 through early 2006, I bought shares in Bruce Berkowitz’s then little-known Fairholme Fund (FAIRX). I did great. As the fund’s reputation soared, I bought more shares in 2010, both before and after Fortune wrote about Fairholme, and bought more early this year — just in time to get whacked. Through November, Fairholme was down a sickening 29% for the year, 30 percentage points below the S&P. I sold in October, having given back almost all the profit I’d made since 2004.

The moral? If you buy a hot manager, don’t be shocked if you get singed.

This article is from the December 26, 2011 issue of Fortune