The fear is that the rollout of booster shots will lead some people to question the effectiveness of the vaccine in the first place.

“Many of my patients are already saying, ‘If we need a third dose, what was the point?’” said Dr. Jason Goldman, a physician in Coral Springs, Florida.

The average daily count of Americans getting a first dose of vaccine has been falling for six weeks, plummeting more than 50% from about 480,000 in early August to under 230,000 by the middle of last week, according to the most recently available federal data.

An estimated 70 million vaccine-eligible Americans have yet to start vaccinations, despite a summer surge in infections, hospitalizations and deaths driven by the Delta variant.

This is the case despite a growing number of businesses announcing vaccination requirements for their employees, including Google, McDonald’s, Microsoft and Disney. Also, big cities such as New York and San Francisco are demanding people be vaccinated to eat at restaurants or enter certain other businesses.

Separately, President Joe Biden announced sweeping new vaccine requirements for as many as 100 million Americans on Sept. 9. Employees at businesses with more than 100 people on the payroll will have to get vaccinated or undergo weekly testing. But the mandates have yet to go into effect; the necessary regulations are still being drawn up.

While some workers have gotten the vaccine under pressure from their employers, many holdouts remain.

Experts have long said the key to ending the pandemic is vaccinating the vast majority of the American public—perhaps as much as 90%. But of the more than 283 million Americans age 12 and older who are eligible for shots, only about 65%—184 million—are fully vaccinated, according to Centers for Disease Control and Prevention data. Children under 12 are not yet eligible to get vaccinated, meaning only about 55% of the U.S. public is fully protected.

CDC Director Dr. Rochelle Walensky said Tuesday that health officials have not lost sight of that problem. The booster effort “will not distract us from our most important focus—to get as many people as possible vaccinated with a primary series,” she said.

White House officials said they doubt the need for boosters is a real concern among the vast majority of the unvaccinated, who continue to resist getting their shots despite nearly a year’s worth of data showing their lifesaving potential.

They argue, too, that as the pool of unvaccinated Americans gradually shrinks, there will be declines in the numbers of new people getting shots. And they said the latest numbers should not be read as a sign that mandates are not working, noting that most businesses haven’t yet put in place the Biden administration’s vaccinate-or-test policy.

Noel Brewer, a University of North Carolina professor of health behavior, said there is good reason to be optimistic.

“I think we’re heading into a great season for vaccinations. Everything is lining up in a good way,” said Brewer, who advises the CDC and the World Health Organization on COVID-19 vaccination policy.

Last week, the Food and Drug Administration and the CDC authorized booster doses of Pfizer’s vaccine for tens of millions of Americans who are 65 and older, have underlying health conditions or work in jobs that put them at high risk. The extra dose would be given six months after the two-shot regimen.

Regulators have yet to take up the question of booster shots for people who got the Moderna or the Johnson & Johnson vaccine.

More than 400,000 Americans got boosters at drugstores over the weekend, and 1 million more have signed up for them, according to White House officials.

But some members of an expert panel that advises the CDC worried last week that the booster discussion was a distraction from the more pressing need to get more Americans vaccinated.

“We have a very effective vaccine, and it’s like saying, ‘It’s not working,’” said Dr. Pablo Sanchez of Ohio State University.

At that meeting, a CDC official presented unpublished data from a recent 1,000-person survey that suggested that offering boosters would make 25% of unvaccinated Americans much less likely to get a shot. This week, a Kaiser Family Foundation poll of more than 1,500 adults found that 71% of the unvaccinated say the recent news about boosters is a sign that the vaccines aren’t working.

Some outside experts saw this coming.

Dr. James Conway, a pediatric infectious diseases expert at the University of Wisconsin, said last week that if vaccine-hesitant people “start to get the idea that this is only going to last for six or eight or 10 months,” they may be further soured on the whole idea.

Scientists have emphasized that the vaccine remains highly effective against serious illness and death from COVID-19, but they have seen signs that its protection may be slipping.

Meanwhile, the pandemic’s summer surge has shown signs of easing. Deaths are still running high at over 2,000 a day on average, but cases and hospitalizations are trending down.

While any ebb in COVID-19’s toll is welcome, it also may undercut health officials’ effort to instill a sense of urgency in the unvaccinated. That could be a difficult task even when cases were exploding.

Dr. Alex Jahangir, director of orthopedic trauma surgery at Vanderbilt University Medical Center and the head of a coronavirus task force in Nashville, Tennessee, recalled operating on an elderly man injured in a car wreck over the summer. The man survived those injuries but ended up dying from COVID-19.

Jahangir said he was struck by how the man’s family seemed to absorb the facts about the COVID-19’s dangers only at the very end.

“Only when they were negatively impacted did they seek the truth,” Jahangir said.

—Associated Press reporter Zeke Miller contributed to this story from Washington.

More health care and Big Pharma coverage from Fortune:

• Health insurers justify passing COVID treatment costs on to patients despite massive profits
• Why pregnant people are now eligible for COVID-19 vaccine boosters
• Portugal leads the world in COVID-19 vaccinations
• A WHO-approved Novavax vaccine could upstage mRNA jabs—if it can solve its manufacturing delays
• Duckworth, Raimondo: Congress must back the president’s historic investment in home care workers