For die-hard political pundits, observers, and undecided voters, every Sunday and Monday, from now through Election Day, will be a must-watch.
That’s usually when the latest political polls from one or several of at least two-dozen organizations are released.
But are voters overreliant on these polls? It depends on whom you ask.
“I think the American public is being misled,” says Jake Novak, a New York–based political and economic analyst, and a former national cable television network producer who has written at length in his scrutiny of political polls. “We’re looking at polls the wrong way.”
We’re too focused on national polls, Novak says, and state polls tend to be less accurate overall.
Novak’s case in point: the unexpected cancellation of the final installment of the highly anticipated Iowa poll, one of the nation’s most respected political surveys, on Feb. 1, just days before the Iowa Caucus, after a supporter of former Democratic presidential candidate Pete Buttigieg claimed the candidate’s name was omitted by a poll interviewer during a call. Buttigieg’s campaign notified the pollster and its media partners.
“There were concerns about what could be an isolated incident,” J. Ann Selzer, whose company runs the 76-year-old poll, said in a statement. “Because of the stellar reputation of the poll, and the wish to always be thought of that way, the heart-wrenching decision was made not to release the poll. The decision was made with the highest integrity in mind.”
But even when no mistakes are made, how accurate are political polls really?
Quality vs. quantity
Dozens of polls related to the upcoming presidential election have already been released in the first eight days of April.
“I’m of the opinion that more is better,” says Lynn Vavreck, professor of American politics and public policy at UCLA, and author of the book, The Message Matters: The Economy and Presidential Campaigns. “Every new poll is information, and that’s good.”
Vavreck contributes to Nationscape, a series of political surveys conducted by the Democracy Fund and UCLA. Nationscape publishes detailed information on how its surveys are conducted, including field dates, including mode of interview, length, languages offered, and participation rates, for a more detailed picture of the results.
The key to evaluating a poll is to not just look at results, but the methodology behind it: What is the diversity of the sample makeup? Is it too focused on one group of people?
Patrick Murray, director of the Monmouth University Polling Institute, also says don’t misinterpret the purpose of a poll, which is to capture voters’ feelings at a particular moment in time.
Aspects of polling that are the least important, like “the horse-race question” of who has the lead in an election, tend to be the main focus of polls in the media, and that can convey a sense of precision that does not exist in polling, he says.
“Polls do best when they give you a broad sense of the issues of the day. That’s what the science of polling is designed to do,” Murray says. “Trying to forecast an election outcome is not what polling is designed for, but it gets used that way.”
How polling works
While Tim Malloy, the Quinnipiac University poll’s assistant director, couldn’t provide specifics about how it conducts polling, he did share that they make calls every couple of weeks.
During the four days leading up to the New Hampshire primary, Quinnipiac randomly called thousands of self-identified registered voters nationwide, either by cell phone or landline, before narrowing the poll down to 1,519 respondents, with a margin error of 2.5%.
The poll included a mix of approximately 665 Democratic and independent voters, who lean Democratic with a nearly 4% margin of error.
“We break it down by male, female, and party, straight down the middle and as fairly as you can make it,” Malloy says. “It’s part science, part math. All of the good polls do it; Gallup, Pew—you want to have an accurate representation of the country.”
John Della Volpe, director of polling at the Harvard Kennedy School’s Institute of Politics and the founder and CEO of SocialSphere, agrees. The school is currently preparing the spring edition of the Harvard Youth Poll to send to students in April.
Its methodology is similar to Quinnipiac’s in that it polls between 2,000 and 3,000 people, but in this case, those polled are students from age 18 to 29.
“We make sure we have a sample that is robust enough to analyze important subgroups of voters, based upon not just gender, race, and party ideology,” Della Volpe says. “It’s a more rigorous look at a subgroup of voters in the electorate.”
Also, getting people to participate in a poll is a challenge, Malloy and Della Volpe both say. Many potential respondents don’t have the time, patience, or desire to answer questions about their political preferences.
Yet some feel that participating in polls is their patriotic duty, and they want to be heard, Carl Cannon, a longtime D.C. insider, and the Washington bureau chief for polling aggregation site, RealClearPolitics says.
But even polls can get it wrong—really wrong.
The 2016 presidential election
Cannon recalls what happened in Wisconsin in 2016 when Trump won the swing state by less than 1% over his heavily favored Democratic challenger, Hillary Clinton. Respected polls conducted by Monmouth and Marquette universities, Emerson College and YouGov, and aggregators such as RealClearPolitics and FiveThirtyEight, projected Clinton was winning handily.
“The polls missed Trump winning pretty badly in the general election,” Cannon says. “We all did.”
In the crucial swing state of Pennsylvania, many of the last polls before the general election had Clinton ahead of Trump by less than 2%, well within striking distance of overtaking her, Cannon recalls.
“He was closing in, and her campaign knew it,” Cannon says of Clinton losing the critical Keystone State. “Clinton tried to rally, but if you were paying attention all along, you know that state was going to be close.”
Cannon says Trump did particularly well in Luzerne County, which traditionally had been a Democratic staple and hadn’t voted Republican since George H.W. Bush in 1988. But Trump persuaded the mostly blue-collar working-class area struggling with poverty, job shortages, and opioid abuse to vote for him.
“These are people who voted twice for Obama in droves, but they also voted for Trump the same way. [Trump] got people who hadn’t voted in a long time to vote for him,” Cannon says. “The polling was pretty good at forecasting that, but there were a lot of people in our circles who couldn’t imagine Trump winning there, and the entire state of Pennsylvania for that matter.”
In the end, Clinton did win the popular vote, but she didn’t earn enough delegates in the Electoral College to beat Trump for the presidency. Cannon hopes voters are a bit more educated these days.
“It’s easy to forget. We do not have one national election—we’re having 51 different elections, state by state, and the District of Columbia,” Cannon says. “That’s the formula, and it’s not going to change anytime soon.”
More politics coverage from Fortune:
—Americans face hunger crisis as SNAP benefits are harder for some to get
—Joe Biden’s health care plan for the coronavirus and beyond
—World War II offers lessons—and warnings—for the coronavirus fight
—COVID-19 upends learning and cancels spring tests for millions of U.S. students
—Listen to Leadership Next, a Fortune podcast examining the evolving role of CEO
—WATCH: As unemployment skyrockets, the labor market’s future looks grim
Get up to speed on your morning commute with Fortune’s CEO Daily newsletter.
