It’s a growing concern among consumers that companies are taking advantage of them by keeping prices high for no good reason. But is that really happening in the grocery industry?

The full picture of the current state of grocery is more complex. Today, most American grocers are stuck between a rock and a hard place, unable to do what Biden is asking even if they wanted to.

Painting a fuller picture

To better understand grocers’ challenges, consider changing consumer behavior. American households are shopping around more, making regular purchases from multiple stores to get the best prices. In fact, 77% of grocery shoppers say they compare prices across stores, more than any other retail category.

Additionally, industry competition is stronger than ever. Mergers and acquisitions, like the pending deal between conglomerates Kroger and Albertsons, threaten to consolidate market share and push smaller regional grocers out of the picture.

Grocers face challenges from dollar stores, too: In late January, Dollar General announced it would begin selling produce at its 5,000 locations. And so grocers are clearly under pressure to compete on price.

In June 2023, The Feedback Group’s nationwide poll of 1,200 consumers revealed that shoppers thought their primary grocery store generated a 35% net profit margin on average–up from 33% the year prior.

However, consumers’ estimations are far off base. Supermarkets’ actual net profit has averaged closer to 1% historically. At its pandemic high, when grocery sales exploded amidst restaurant closings and quarantines, the margin rose to about 3%. That’s a far cry from the mid-30s assumptions shoppers are making. With such low profit margins across the board, reducing prices any further would force many businesses to close.

That’s because a significant chunk of gross profits–about 70%–goes towards paying off assets (mortgages, interest, rent) and store operations (staffing, accounting, utilities). Over the past few years, grocery store costs have increased faster than the prices they charge. Since January 2020, grocery prices are up by about 25%, while costs have increased more than 28%, according to the Bureau of Labor Statistics. Increasing rent, interest rates, and wages create major challenges that stand in the way of decreasing prices.

It’s also important to consider that not all grocery stores are the same. Biden’s sentiments are directed at grocery behemoths such as Walmart, Kroger, Costco, and Albertsons–the large brands that make up 34% of domestic grocery sales. This ignores the remaining two-thirds of grocers, many of which are regional or independent retailers. These grocers don’t have the same scale that enables large conglomerates to lower costs, in turn lowering prices for consumers.

A more current view of inflation

Many people assume that since inflation has fallen, grocery prices should, too. That first point is true–inflation has indeed cooled. In 2023, grocery prices increased by only 1.3%, compared to 11.8% in 2022.

Those 2022 price increases primarily benefited food suppliers and CPG companies, many of which posted eye-popping profits in 2022 and early 2023. As inflation has settled, those profits have come back to earth, but just because inflation has fallen compared to previous months doesn’t necessarily mean that prices can come down.

To help visualize this distinction, imagine you’re hiking a mountain. After a good bit of climbing uphill, you reach a flat landing. While you’re not climbing in that moment, your altitude is still far above when you started climbing.

In this analogy, price is your altitude, and inflation is your uphill slope. When President Biden says inflation is down and thus prices should come down, it’s like saying that hiker’s altitude should fall now that they’ve encountered a level path. It would take deflation–not just reduced inflation–to facilitate lower prices.

Most grocers simply don’t have the flexibility to reduce prices and remain comfortably in business. It’s important to remember that they hate inflation, too. In a recent Upside survey of 188 grocery leaders, the top business challenges they reported all revolved around cost: led by supply chain challenges and costs, inflation, and hiring and labor costs.

So yes, inflation has leveled out, but costs still remain inflated from prior years. And most regional grocers don’t have the scale to consolidate costs and make room for price reduction.

Everybody would like to see grocery prices come down–even grocers, because that would mean costs have leveled out. It’s essential to focus on the right challenge. Targeting grocery stores for making “suckers” of American shoppers lacks the nuance this topic requires.

Tyler Renaghan is the VP of Grocery at Upside, where he uses his 15 years of experience in grocery retail and e-commerce to help grocers profitably grow their businesses. The companies named in this article are not Upside clients.

More must-read commentary published by Fortune:

• Here’s how the U.S., Europe, and China are faring in the post-pandemic race for economic growth
• Working fathers are the new target of microaggressions–and they are worried they could be getting ‘daddy tracked’
• WEF president: ‘It’s time to revitalize trade—and reverse the trend of Slowbalization’
• The anti-DEI movement has gone from fringe to mainstream. Here’s what that means for corporate America

The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.