This is the single biggest risk to the stock market right now

Throughout the first half of the pandemic, interest rates went in one direction—down. After bottoming out at 0.5% toward the end of the summer, 10-year Treasury yields are finally on the upswing.

No one knows the precise reason for the increase in rates, but inflation expectations are rising now that there is light at the end of the tunnel for the pandemic. When rates are rising because investors expect economic growth to be higher in the future, that’s a good thing.

But investors always worry about risk, and there is fear this rise in rates could derail the stock market. That is certainly possible, but history shows the stock market holds up well in rising rate environments.

I went back to 1950 to find each instance where the 10-year Treasury yield rose by 1% or more to see how U.S. stocks performed when bond yields were moving higher:

There have only been two instances where stocks fell during a rising rate environment. In fact, the annualized returns in each of these periods where rates rose 1% or more was 10.5%, which is right around the average long-term return for the U.S. stock market.

It is worth noting some of these rising rate environments did precede some nasty stock market falls. Rates raced higher in 1987 right before the biggest one-day crash in history in October of that year. And rates rose in the final couple of years in the late-1990s dotcom bubble as the Fed was trying to snuff out a speculative mania. Eventually, it worked, and stocks got cut in half.

But we certainly don’t know the exact level of rates in which investors would determine things have gotten out of hand. There is no line in the sand on these things. It’s more of an exercise in psychology than anything.

So if it’s not rising interest rates investors need to worry about in the current environment then what should you be looking for?

Inflation is far more important to stocks than interest rate levels.

Since 1928, the average annual inflation rate in the United States is right around 3%. Based on the average returns for the S&P 500 in that time, the stock market would prefer the inflation rate stays below that level and that it’s falling not rising.

The average annual return for years in which the inflation rate is rising saw stocks up 6.7%, while the average annual return when inflation is falling was nearly 17%. A similar relationship exists when looking at inflation from one year to the next. Historically, when inflation has been higher than the previous year, stocks have returned an average of 6.3% against an average annual return of almost 16% when the inflation rate is down year over year.

Inflation is still quite low by historical standards today, at around 1.4% over the past year. But inflation expectations, as measured by the five-year break-even inflation rate, are now at their highest level since 2013 at 2.4%. That’s still below the long-term average, and actual results don’t always line up perfectly with expectations.

Rising interest rates and inflation expectations are a good thing for the economy. Each of these variables is pricing in higher economic growth from the potential end of the pandemic and the ensuing spending boom it could bring about from consumers.

But investors need to be aware of the potential risks involved as well. It felt backward when the stock market was doing so well in the face of a weakening economy last year. It may feel equally backward if the stock market pauses for a bit in the face of a strengthening economy this year. Yet this is how the stock market works, and it doesn’t always make sense in the moment.

Ben Carlson is the director of institutional asset management at Ritholtz Wealth Management.He may own securities or assets discussed in this piece.