Consumer confidence is at its lowest level since 2014, yet spending rose in August. Kari Butcher, Burson’s Chair for Intelligence & Insights for the Americas, argues that business leaders make the same mistake with election polls: treating what people say as a prediction of what they will do.
Business leaders are making consequential decisions based on a category error: treating what people say in a poll as a reliable forecast of what they will do next.
Consumer confidence falls, and a retailer cuts inventory. A majority supports a social issue, and a CEO assumes speaking out will strengthen customer loyalty. A company’s reputation score holds steady, and its leadership team concludes customers aren’t going anywhere. In every case, the poll may be accurate. The business conclusion can still be wrong.
Polls measure what people say at a particular moment. On their own, they cannot tell a leader how that sentiment will translate into purchases, loyalty, employee behavior or investor reaction. Get that distinction wrong, and a company can pull back while demand remains strong, enter a debate its customers did not ask it to join, or mistake the friction of switching for genuine loyalty.
The midterms make this harder. The economy is voters’ top issue: 84% of registered voters told Pew it is very important to their vote. Polls are everywhere, and leaders want to know what the election will mean for spending, investment, and what customers and employees expect. A poll showing who’s ahead may help a business anticipate the political environment. It cannot tell them how consumers, employees or investors will respond to it. CNN’s average of recent generic-ballot polls has Democrats ahead 52% to 42%, but that is a measure of preference, not of what any customer will do next.
As a former pollster, I don’t think polling is broken. The question I’d ask of any poll is what decision it gives me enough evidence to make. That means knowing whose views the number captures, what’s behind them, and whether people will act on them.
Take the economy. The Conference Board’s consumer confidence index fell 6.7 points in September to 81.9, its lowest level since 2014. The next day, the government reported that spending rose 0.9% in August while personal income rose 0.2%. Households made up the difference by saving less: the saving rate fell to 4.1%. Sentiment says people are deeply pessimistic. Spending says they’re carrying on. Both are partly true, because people can feel one way and spend another. A retailer watching only the sentiment number would miss that.
Elections show how quickly stated preference can be mistaken for predicted behavior. About a week before Michigan’s Democratic Senate primary, a poll had Abdul El-Sayed ahead by 15 points, 54% to 39% among likely primary voters. He won by less than a point. One possible factor is the assumption about who would actually vote. In early October, YouGov’s chief scientist found that the same survey could show a Democratic lead on the national House ballot anywhere from 6 to 16 points depending on how likely voters were identified. He called the high end almost certainly an overstatement and said defensible choices gave 8 to 14 points. The answers didn’t change. The assumptions about behavior did.
That’s why one number is never enough. The Washington Post made a similar point in late September: a single poll is a snapshot, and trends across polls tell us more.
Companies run into a version of this, and I should disclose that the evidence here comes from Burson’s own research. In the Burson Pulse, a survey of 1,601 U.S. adults fielded in late April, 61% said economic anxiety was making them save more and pay closer attention to what companies say and do. Four months later, national data showed households saving less. The two measures come from different months, so they are not a direct test, but they show the risk of treating a stated intention as a forecast. Burson’s Pulse study is designed to track how confidence moves over time. In the first wave, 48% of Americans said they lost confidence in a company gradually, without being able to point to a single moment. Another 52% said a single event did it. Confidence can erode quietly, well before it shows up in a headline or a metric, so an overall score may not reveal slipping confidence until the damage is done.
Don’t throw out the polls. Stop asking one source to do everything. Pair what people say with what they search for, share, buy and do. Read what is being said online and continue asking people directly. Before acting on any poll, leaders should ask one question: What decision does this give us enough evidence to make? With the midterms days away, that question matters most for the polls everyone is reading: they will tell you who is ahead, not what your customers will do next. If the answer is unclear, find the missing evidence before making the decision.
The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.