FORTUNE — The stock market probably isn’t in a bubble, but there’s a bigger risk you need to consider: That buying now will mean much lower returns over the next few years.
The valuation on the S&P 500 is still reasonable enough – a P/E of 16.6, based on trailing earnings, which is only slightly higher than average.
That’s good news if you’re worried about a crash. But if you want 10% long-term returns? Or even 8%? You’re going to be disappointed.
MORE: Stocks not in a bubble? Hold on there Yellen
In fact, try 4%.
On average, going back to the 1930s, the market has risen just 4% a year in the following decade when stocks have traded for a similar P/E as they are now, according to Birinyi Associates.
And among a growing group of forecasters, 4% is becoming something of a consensus.
Bridgewater’s Ray Dalio last week said that’s what he expects from the market for the next decade.
MORE: Why should JPMorgan and Twitter have to pay more taxes?
Robert Shiller says his calculations suggest stocks will rise about 2.5% a year for the next decade, plus inflation, which has recently been averaging 1.5%. Cliff Asness, who runs AQR, which manages one of the largest hedge funds in the world, says they will rise 4.5% annually on average.
4% is not horrible. After the two big stock drops of the 2000s, the market basically went nowhere for the following decade. In a 4% market, with compounded returns, the money you put in the market now will be worth nearly 50% more in a decade.
So while it might not be time to be scared of the market — the economy is improving — it’s certainly a time to be less excited about it.