Our mission to help you navigate the new normal is fueled by subscribers. To enjoy unlimited access to our journalism, subscribe today.
Three days before the 2016 election, the Princeton Election Consortium website declared that Hillary Clinton had a “more than 99% chance” of winning. Princeton neuroscientist Sam Wang, who runs the website, had called 49 states correctly in 2012 and said he’d “eat a bug” if Donald Trump won.
A man of his word, he later ate a cricket on CNN.
Such stories give Trump supporters hope and Biden supporters jitters. The parallels this year are eerie. The Economist’s prediction model, which is recalculated every day, recently gave Biden a more than 99% chance of winning, for example. Could today’s data-rich, computer-powered, Ph.D.-filled prediction industry blow it again?
The only honest answer is maybe. In evaluating this year’s polls and predictions as the campaign nears its end, it pays to remember a few basic facts that can be surprising but aren’t often mentioned in reports of the latest numbers.
The national polls in 2016 were actually quite accurate
Their average error in the campaign’s final 21 days was 3.1%. That’s much better than the average error in all presidential elections since 1972, which is 4.1%. In some years, the polls were much worse. The pollsters’ worst performance by far in that 44-year period—and neither you nor anyone you know will guess it —was in 1980, when the average error was 8.9%. No one remembers it because the error grossly underestimated Ronald Reagan’s margin of victory; the polls showed him ahead by just 0.8%, but he won by a landslide 9.7% margin. Because the polls correctly predicted the winner, the public didn’t care about getting the margin wrong. But pollsters are just the opposite: To them, errors in either direction are bad. So in the public’s mind, an error of 3.1% in 2016 was a debacle while a far worse error in 1980 is forgotten.
But the polls weren’t as accurate—and have never been as accurate—as most people seem to think they are
That average polling error of 4.1% would strike most of us as quite large. After all, the popular-vote margin was less than that in four of the past five elections. The lesson is that polling is hard, and we should regard all election polls warily. Is the sample properly weighted to reflect the people who will actually vote? What’s the best way to reach them: random-dialed phone, email, text? Do they tell the truth about themselves (age, income, voter registration status) and about which candidate they favor?
But wait, you’re saying, who cares about national preference polls? That’s not how we elect Presidents. So don’t forget…
National polls don’t count—and state polls aren’t as accurate
In the electoral college system, you don’t know much if you don’t know which states are leaning which way, and that was a problem in 2016. While the national polls were more accurate than their long-term average, the state polls were less accurate—off by 5.2% vs. a long-term average error of 4.8%.
The largest lesson that pollsters learned from 2016 is that they were overweighting college graduates in state polls, apparently because graduates are more likely to respond to phone polling than nongraduates are. That’s one reason pollsters overestimated Clinton’s prospects. This year many pollsters are making sure they get the balance right—which is good, but they’re fighting the last war. Whether they’re committing new, unimagined errors this year remains to be seen.
Polling isn’t the same as predicting—and the predictions vary more widely than the polls do
Well-known prediction models from FiveThirtyEight, The Economist, and the Princeton Election Consortium use sophisticated mathematical equations to transform polling data and sometimes additional data into predictions. All those organizations have access to the same data, yet they produce widely varying forecasts. In 2016, when the Princeton Election Consortium was giving Clinton a 99% chance to win, FiveThirtyEight pegged it at only 71%. The Economist wasn’t running its own model then; the New York Times (which isn’t running its own model this year) said 85%.
Exactly how those numbers are produced is incomprehensible to the average citizen. The Economist, for example, loads polling and economic data into equations and then simulates the election 20,000 times using different scenarios. FiveThirtyEight runs 40,000 simulations. The predictions reflect the number of simulations in which each candidate wins, though this description vastly oversimplifies the process.
The result is that these prediction engines are black boxes for most of us. For what it’s worth, as of this writing, FiveThirtyEight gives Biden an 89% chance of winning; The Economist, 95%. The Princeton Election Consortium no longer publicizes candidates’ percent chances of winning. Instead it calculates a “meta-margin,” which is the change in national popularity necessary to produce an electoral-vote tie; it shows that Biden’s lead in the national polls would have to plunge by 5.9% for that to happen. Such a hefty margin is a reminder of something else to keep in mind…
2020 is different
The simplest and strongest reason to believe this year’s predictions is that the polling differences between the two candidates nationally and in key states is far greater than it was in 2016. The polls are guaranteed to be wrong—they never get each number exactly right—but this year they’d have to be very, very wrong in order to miss a Trump victory. As Fortune’s Lance Lambert showed, if the errors in this year’s state polling numbers were the same as in 2016, Biden would win easily.
And finally, please remember that…
Predictions are probabilities, not certainties
If a model says Candidate X has a 94% chance of winning, and Candidate Y wins, it doesn’t follow that the model got it wrong. If you flip a coin four times, there’s a 6% chance it will come up heads every time—unlikely, but it happens.
The larger point is that, especially today, we all crave certainty. But it’s the uncertainty—the possibility of an unbelievable outcome, whether you dread it or yearn for it—that makes Super Bowls, World Series, the Oscars, and above all, presidential elections so irresistibly compelling.
