The biggest AI players have effectively imposed a tax on the entire economy — and most people have no idea it’s happening.

How the once-affordable memory chip became a luxury good

Modern computing relies on several types of memory. SRAM is the fastest and most expensive; it’s used in small amounts inside processors. DRAM is the workhorse of the group: cheap, abundant, found in everything from laptops to cars to refrigerators. Then there’s High Bandwidth Memory or HBM. This is a specialized, premium form of DRAM that stacks chips die-to-die to achieve dramatically faster data transfer speeds. The cost for this premium memory is quite steep: a single silicon wafer provides 3x as much commodity DRAM as HBM. Fab processing time for HBM is significantly longer too, making the supply problem worse. As a result, producing more HBM equates to fewer total memory chips produced.

For AI training and inference, HBM has become the essential ingredient. It’s the jet fuel that powers the GPUs running today’s largest and most advanced models. 

Memory manufacturers have a limited number of wafers they can produce from each fab, or silicon factory. The same production lines that churn out commodity DRAM for the devices consumers use every day are being allocated to building HBM. It’s a rational business decision: HBM commands premium prices in a volatile industry and comes with massive guaranteed purchase orders. In fact, AI firms and their peers have already locked up HBM supply well into 2027. The result is a tightening of commodity memory supply, rising prices, and longer lead times with ripple effects that touch almost every industry. And right now, the industry’s biggest players have cornered the supply, creating the core tension driving the memory shortage. The AI industry is effectively taxing the entire economy in order to build its own.

The memory wall explained

The scale of AI’s memory appetite is staggering. As model sizes have grown from millions to billions to trillions of parameters and context windows have grown from thousands of tokens to tens of millions of tokens, memory requirements have increased in step, and the architecture of data centers has struggled to keep pace. This is the industry’s “memory wall”: a fundamental bottleneck where memory bandwidth and capacity can’t keep up with the processors demanding it.

HBM was an elegant solution to an earlier version of this problem. When models were smaller (such as GPT-2 and GPT-3), placing memory adjacent to the processors and delivering data at extreme speeds worked well. But we’ve since blown past that era. Today’s frontier models exceed two trillion parameters and the next generation will be over five trillion. A single HBM stack holds about 24 gigabytes. That’s roughly one percent of what today’s workloads actually need and far less than that for the next generation of workloads.

The result is that data centers must now scale out exponentially. They chain together hundreds of processors across servers and racks. At that point, HBM’s killer feature of extreme local bandwidth gets strangled by the comparatively slow links connecting all of these machines. The industry has built a gold-plated solution to the problem: AI companies pay the HBM premium while realizing only diminishing returns on performance.

The AI gold rush leaves most behind

The prevailing narrative frames AI infrastructure investment as broadly good: better for memory makers, better for chip companies, better for innovation everywhere. The reality is more lopsided.

Memory manufacturers may profit in the short term. But the true winner is concentration itself. When the HBM supply is locked up by a handful of hyperscalers, it functions as a moat. Startups, enterprises, and established industries all face higher hardware costs and more limited access to the advanced AI capabilities they need to compete. The companies that can afford to stockpile chips don’t just win today; they entrench advantages that could become impossible to dislodge.

Everyone else is caught in the crossfire. Consumers will pay more for devices with less capability. Businesses face a hardware cost environment that has become more taxing and volatile. And the broader technology ecosystem is competing for memory resources against an industry that has essentially unlimited capital to outbid them.

Charting a more sustainable path for AI and memory

The AI industry loves to talk about democratization: open models, accessible tools, intelligence for everyone. That story is increasingly disconnected from the hardware reality being constructed underneath it.

The current trajectory isn’t sustainable. Pouring more investment into HBM capacity addresses a symptom while ignoring the underlying disease. The industry needs to move beyond its fixation on a single memory architecture designed for an earlier era of AI and invest seriously in new approaches—ones that can meet AI’s demands today, and as they grow a hundredfold in the next few years.

Solving this requires more than ramping up additional HBM fabs. It requires a fundamental rethinking of how memory is architected for AI. What’s needed are memory systems that are smart, fast, and compact — architectures that can scale alongside model growth without requiring brute-force resource consumption. Most importantly, the emerging architecture must make AI accessible to more than just a handful of companies.

The memory wall isn’t just an engineering footnote in AI’s rise. It’s the defining infrastructure challenge of this era. The industry’s current answer of “more HBM, faster, at any cost” is a perilous road that risks eroding competition, innovation, and consumer trust. As an industry, we must find a way to do better, and quickly, before it’s too late. The most immediate relief available is a pivot away from HBM dependency toward commodity DRAM architectures engineered specifically for AI’s requirements. The window to act — before the gap between AI haves and have-nots becomes unbridgeable — is closing fast.

The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.