They argue that Ben Bernanke is pulling back the Federal Reserve’s stimulus for a reason investors should like: The economy is improving. And a better economy is good for stocks. Right?

Maybe not. It certainly hasn’t been the case lately that a rising stock market and a good economy have gone hand-in-hand. In fact, it’s been nearly the opposite. The stock market is up 72% in the past four years, even with the recent drop. Have the past few years felt like we are in an economy 72% better than it was?

MORE: Don’t believe Alan Greenspan’s bullish case for stocks

In all, the Dow Jones industrial average (INDU)  has more than doubled, up 132% since it bottomed out in March 2009. The unemployment rate, however, is less than halfway back to where it was before the recession.

Earlier this year, a number of commentators howled about how the Dow Jones industrials hitting an all-time high made no sense given the continued poor state of the economy. And yet, now everyone expects the relationship between stocks and the economy to somehow become coherent in the next year.

Going back a little further, the link between the economy and the market has held up a little better, but it’s still far from perfect. The best year for the U.S. economy in the past four decades was 1984. Gross domestic product rose 7.2%. But that year was a bummer for stocks. The S&P 500 (SPX) rose a mere 1.4%. The recent recession produced the worst year for the economy in the past four decades — 2009. How did the stock market do that year? It soared — up nearly 25%, a great year for investors.

Overall, since 1975, the stock market has risen 11.2% in years in which the GDP has risen more than 3%. That compares to a 7.9% rise in years when the economy grew less than that. But remove one year, 2008, and the stock market return of the less-than-stellar economic years rises to 10.6%. That suggests the strength of the economy makes very little difference to the direction of the market, or at least much less than we suspect.

MORE: Obama should have fired Bernanke a lot time ago

And there is reason to believe the economy matters even less to the stock market these days than usual. Part of the problem is always about timing. And the market and economic metronome seem more off this time around than usual.

What does seem to matter to stocks is corporate profits and interest rates. Neither is going the market’s way. Earnings of the companies in the S&P 500 rose just 3.3% in the first quarter of 2013. Analysts expect that to have slowed to 1.3% in the second three months of the year. And with profit margins at an all-time high, even a slightly higher GDP might not be able to improve bottom lines that much. Then there are interest rates, which seem headed up faster than most people expected, another drag for the market.

So by all means, carry on. The economy is improving. But don’t expect the stock market to stay calm.