The U.S.’s retirement crisis is about to be further exacerbated by another uniquely American financial malady: student loan debt.

Many workers made the most of the past three years of paused payments to beef up their retirement contributions. But that trend will reverse, at least for some, in the fall once the monthly payments come due and workers have to reprioritize their budgets, according to Fidelity Investments’ Q2 2023 retirement report.

“Retirement is one of the first savings goals to feel the impact of those shifts,” Jesse Moore, head of student debt at Fidelity Investments, told Fortune in an email. Households that already have little financial wiggle room will need to make up the money somewhere, and that typically comes at the expense of robust retirement savings.

Fidelity previously found that the share of borrowers contributing at least 5% of their pay to their 401(k)s grew from 63% to 72% since the COVID-era payment freeze began in March 2020. The share of those taking loans from their 401(k)s—a sign of financial hardship—also decreased during the pause.

It’s why young workers have made significant ground in their retirement contributions. Gen Z saw a 66% increase in average 401(k) balances compared to a year ago, according to Fidelity’s Q2 report, which analyzed the more than 45 million retirement accounts it manages, while millennials saw a 24.5% increase. Other reports have found similar trends: Young people are contributing more today than older generations did at similar ages.

Once payments, which average around $400 per month, according to Jeffries, resume in October, those increased retirement contributions will likely be dialed back. In fact, research has found employees with student loans—regardless of balance size—save significantly less than those without the debt.

Recent graduates will “certainly be highly impacted,” says Moore. A separate Fidelity report published earlier this month found that 65% of recent college graduates who currently have their federal student loan payments pause “have no idea how they are going to start repaying their student loans once the emergency pause is lifted.” A similar share say their loans are preventing them from participating in financial milestones like saving for retirement or getting married.

These young workers have less of a financial cushion than older generations while grappling with the inflated costs of everything from food to housing. They also take on more student debt, on average, than their older counterparts (baby boomers boast the largest average balances, thanks to compounding interest).

That said, “this is an issue that is going to affect all age groups,” says Moore. “Whether you are carrying debt for your own education or a child’s, people of all ages are facing higher costs and adding yet another expense will make retirement savings a challenge.”

And while not being able to save for retirement is bad enough, Moore foresees other financial challenges for the more than 40 million people who will have payments come due soon. That could result in even bigger impacts on retirement accounts, which workers may turn to as a last resort in emergencies.

“It will be important to watch 401(k) loan rates amongst student loan borrowers once payments resume,” says Moore. “Not only will these borrowers have fewer alternatives to turn to in difficult times, but they could also fall back into a pattern of saving less and withdrawing at a higher rate.”

Borrowers have some options

While workers may not be able to contribute as much themselves, employers will be able to make matching contributions into retirement accounts for employees who are making student loan payments starting in 2024, thanks to the SECURE Act 2.0 that was signed into law late last year.

Those struggling to afford their payments have a few options. They can opt to enroll in an income-driven repayment plan, like the new SAVE plan, which can cut payments to as little as $0 per month. Borrowers can sign up for that plan now.

And, as a last resort, they can put off paying their student loans a little longer. The Biden administration announced a one-year grace period for borrowers, from Oct. 1, 2023, to Sept. 30, 2024, during which missed payments will not be reported to credit bureaus, and borrowers will not be considered delinquent if they don’t pay. Additionally, interest will still accrue on balances but it won’t capitalize, meaning it won’t be added to the principal.

Retirement savings is just one area that could suffer once the payments restart again in the fall. Some experts have predicted a coming student loan “cliff” for the economy, leading to less spending in other areas.