While more workers are back in office than ever since the pandemic started, companies have struggled to drag all of them to their desks. Not even the lure of lunch boxes and pizza parties wrangled them in.

But the power dynamic could hit a head if the recession we’ve been worrying about comes to be: 78% of American workers fear that remote employees are more at risk of losing their jobs during a recession that leads to layoffs than the folks who’ve returned to the office full-time, according to employment background check company GoodHire’s second annual “State of Remote Work” report.

Whether there’s any real reason to fear is unclear at this point, but the contention between employers and the workers holding on to working from home has led to threats of firing or moving remote workers’ names to the top of the list if layoffs become a reality.

After surveying roughly 3,500 Americans over the age of 25, the GoodHire report found that the number of people who prefer to work from home dropped to 44% from 68% last year. That said, 89% believe most of their colleagues still wanted to work from home.

There’s a disconnect. How can it be that the majority thinks most of their peers want to work from home, yet there are increasing worries that those remote workers will be first on the chopping block come layoff time? It likely stems from proximity bias in the workplace, in which bosses tend to give workers who are in their line of vision more regularly preferential treatment.

According to the survey, 68% of American workers are concerned their manager views full-time office workers as more productive and higher performers in general than full-time remote workers—even though research shows the opposite is true. A recent report from Microsoft found that productivity might actually be higher than it was before the pandemic.

It’s not even the case that most people long to work remotely all the time forever. A staunch dichotomy in the remote work vs. in-person work has been created and cultivated, and the outcome—whatever that may be—is striking fear in the workforce.

“[The report] serves as a wake-up call for employers who offer hybrid in-person and remote work models: A battle between in-office and remote workers is here—how will you structure your organization’s work-from-home policies and compensation strategies to quell fears and keep employees engaged and satisfied?” GoodHire wrote in a news release announcing the survey.

Talk has been rampant about how a looming downturn would take the wind out of the sails of remote work—or boost it. Predictions put the possibility of a “severe, long, and ugly” recession hitting the U.S. by the end of 2022, economist Nouriel Roubini said in an interview last week.

Even as a recession incessantly stalks America like it’s Halloween 13 and workers seem to be losing their edge over employers, remote work—even if it’s part-time—is still a rather fervent sticking point. More than 60% of workers surveyed by GoodHire said they felt companies would have a hard time attracting talent if they didn’t offer remote work. And 33% said they would straight-up quit, or start a remote job search, if they were forced to return to the office full-time.