• Home
  • Latest
  • Fortune 500
  • Finance
  • Tech
  • Leadership
  • Lifestyle
  • Rankings
  • Multimedia

Trendingnow

1

Despite a $500 million net worth, Shaq just finished his fourth degree. He warns graduates: 'Your character will take you further than your resume'

2

Bolt CEO says he let go of his entire HR team for creating problems that didn’t exist: ‘Those problems disappeared when I let them go’ 

3

Jeff Bezos wants the bottom half of earners to pay zero income tax—he says nurses making just $75K should save $12K a year

1

Despite a $500 million net worth, Shaq just finished his fourth degree. He warns graduates: 'Your character will take you further than your resume'

2

Bolt CEO says he let go of his entire HR team for creating problems that didn’t exist: ‘Those problems disappeared when I let them go’ 

3

Jeff Bezos wants the bottom half of earners to pay zero income tax—he says nurses making just $75K should save $12K a year
Tech

What the Latest Nate Silver Controversy Teaches Us About Big Data

By
David Z. Morris
David Z. Morris
Down Arrow Button Icon
By
David Z. Morris
David Z. Morris
Down Arrow Button Icon
November 6, 2016, 5:20 PM ET
Inside The South By Southwest (SXSW) Interactive Festival
Nathaniel "Nate" Silver, editor-in-chief of ESPN's FiveThirtyEight blog, speaks during a panel discussion at the South By Southwest (SXSW) Interactive Festival in Austin, Texas, U.S., on Saturday, March 8, 2014. The SXSW conferences and festivals converge original music, independent films, and emerging technologies while fostering creative and professional growth. Photographer: David Paul Morris/Bloomberg via Getty ImagesPhoto courtesy of Bloomberg via Getty Images

Early on Saturday, Nate Silver, renowned poll analyst and editor of FiveThirtyEight.com, unleashed a Twitter tirade directed at the Huffington Post. His target was an article that criticized Silver’s use of a modeling technique known as trend line adjustment to adjust poll results in light of each poll’s changes over time.

HuffPo Washington Bureau Chief Ryan Grim characterized such adjustments as “merely political punditry dressed up as sophisticated mathematical thinking.” In part, Grim seemed motivated to assert the primacy of the resolutely left-leaning HuffPo’s own election projection, which gives Hillary Clinton a 98% chance of winning, over FiveThirtyEight’s, which paints the election as much closer, giving Clinton only a 64.9% shot.

Get Data Sheet, Fortune’s technology newsletter.

Silver responded yesterday with a long series of tweets. That included several that point to an important lesson for business:

Not just an issue in elections models. Failure to understand how risks are correlated is part of what led to the 2007/8 financial crisis.

— Nate Silver (@NateSilver538) November 5, 2016

We constantly write about our assumptions and **provide evidence** for why we think they're the right ones. https://t.co/IhLKXdxGGK

— Nate Silver (@NateSilver538) November 5, 2016

That's what makes a model a useful scientific & journalistic tool. It's a way to understand how elections work. Not just about the results.

— Nate Silver (@NateSilver538) November 5, 2016

When you go low, I go high 80% of the time, and knee you in the balls the other 20% of the time.

— Nate Silver (@NateSilver538) November 5, 2016

And yes, even that last one is important. Because this isn’t ultimately a debate about who’s ‘right’ and who’s ‘wrong’—it’s a debate about how we think about statistical prediction itself.

In industry, big data has been a buzzword for years now, with existing and potential applications including fraud detection, securities trading automation, supply chain management, and market trend prediction. All of these, in various ways, attempt to use current data to predict the future.

But Silver, despite his position as the reigning king of predictive analytics, is in essence calling for a very cautious, even skeptical, approach to its applications. Numbers, he is arguing, are not magically correct because they’re numbers—they can be incomplete or wrong. Silver’s trend line adjustments are, at least in part, an attempt to account for errors and noise in data by smoothing them against a historical pattern.

For more on analytics, watch our video.

But even that historical data is limited, with thorough and modern polls only going back to the 1970s. The same can be said for a lot of business data. As Silver references, the housing market bets that led to the 2008 collapse were based on a very shallow pool of data about default rates on high-risk mortgages.

That limited data pool is one possible reason that Silver has come up with a model that bakes in a great deal of uncertainty—in modeling terms, it’s conservative in its predictions. Giving Donald Trump a 35% chance at the White House, even with a lot of national polls showing a big Clinton lead, is, in essence, a message that the world is very complex and unpredictable.

That seems to frustrate people who don’t like uncertainty—which, right now, includes a lot of Democrats like Grim.

And of course, uncertainty isn’t a big selling point for big data applications in business, either. The entire point of data analytics has always been to make decisions easier—or even, as in the case of trading algorithms, take decision-making out of human hands entirely. And maybe, as data pools get deeper and we have longer and more robust historical patterns, the numbers will speak to us more clearly, about everything from polling to disaster risk to people’s taste in toothpaste.

But until that day comes, both Silver and Grim are at least partly right—but mostly Silver. Numbers don’t simply speak for themselves, and interpreting them still leaves room for human fallibility, and even bias. And building models that reflect uncertainty isn’t a crime. Instead, those models merely reflect reality.

The real question is whether we’re comfortable acknowledging that.

About the Author
By David Z. Morris
See full bioRight Arrow Button Icon

Latest in Tech

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025

Most Popular

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Fortune Secondary Logo
Rankings
  • 100 Best Companies
  • Fortune 500
  • Global 500
  • Fortune 500 Europe
  • Most Powerful Women
  • World's Most Admired Companies
  • See All Rankings
  • Lists Calendar
Sections
  • Finance
  • Fortune Crypto
  • Features
  • Leadership
  • Health
  • Commentary
  • Success
  • Retail
  • Mpw
  • Tech
  • Lifestyle
  • CEO Initiative
  • Asia
  • Politics
  • Conferences
  • Europe
  • Newsletters
  • Personal Finance
  • Environment
  • Magazine
  • Education
Customer Support
  • Frequently Asked Questions
  • Customer Service Portal
  • Privacy Policy
  • Terms Of Use
  • Single Issues For Purchase
  • International Print
Commercial Services
  • Advertising
  • Fortune Brand Studio
  • Fortune Analytics
  • Fortune Conferences
  • Business Development
  • Group Subscriptions
About Us
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • Facebook icon
  • Twitter icon
  • LinkedIn icon
  • Instagram icon
  • Pinterest icon

Latest in Tech

Elon Musk puts one hand to his chin and he looks up. He is in front of a blue "World Economic Forum" background.
InvestingSpaceX
‘SpaceX is his new baby at the expense of Tesla’: Elon Musk’s IPO could be bad news for his EV maker, investors warns
By Sasha RogelbergMay 21, 2026
7 hours ago
matthew prince
AILayoffs
Cloudflare posted record revenue, then cut 20% of its workforce. CEO Matthew Prince says AI has made an entire category of workers obsolete
By Jake AngeloMay 21, 2026
7 hours ago
Prakash Arunkundrum, HP’s first-ever chief strategy and transformation officer, bets edge AI will ‘bring the token cost down’
AIConsumer electronics
Prakash Arunkundrum, HP’s first-ever chief strategy and transformation officer, bets edge AI will ‘bring the token cost down’
By Angelica AngMay 21, 2026
8 hours ago
malaysia
CybersecuritySocial Media
Malaysia slams ‘grossly offensive, false, menacing and insulting’ TikTok memes about its king
By The Associated PressMay 21, 2026
9 hours ago
Meta laid off 10% of its workforce as Mark Zuckerberg warns that in the AI race ‘success isn’t a given’
AILayoffs
Meta laid off 10% of its workforce as Mark Zuckerberg warns that in the AI race ‘success isn’t a given’
By Marco Quiroz-GutierrezMay 21, 2026
9 hours ago
bock
Cybersecurityfraud
Minnesota fraudster at center of $250 million scam, controversial ICE crackdown sentenced to 42 years
By Tim Sullivan and The Associated PressMay 21, 2026
10 hours ago

Most Popular

Despite a $500 million net worth, Shaq just finished his fourth degree. He warns graduates: 'Your character will take you further than your resume'
Success
Despite a $500 million net worth, Shaq just finished his fourth degree. He warns graduates: 'Your character will take you further than your resume'
By Preston ForeMay 20, 2026
2 days ago
Bolt CEO says he let go of his entire HR team for creating problems that didn’t exist: ‘Those problems disappeared when I let them go’ 
Workplace Culture
Bolt CEO says he let go of his entire HR team for creating problems that didn’t exist: ‘Those problems disappeared when I let them go’ 
By Preston ForeMay 19, 2026
2 days ago
Jeff Bezos wants the bottom half of earners to pay zero income tax—he says nurses making just $75K should save $12K a year
Success
Jeff Bezos wants the bottom half of earners to pay zero income tax—he says nurses making just $75K should save $12K a year
By Preston ForeMay 21, 2026
13 hours ago
Pay transparency is exposing a bigger problem: Most companies can't explain why they pay what they pay
Workplace Culture
Pay transparency is exposing a bigger problem: Most companies can't explain why they pay what they pay
By Sydney LakeMay 20, 2026
1 day ago
Meet a 21-year-old community college student who's going to China as the first American woman welder in the trades Olympics
Future of Work
Meet a 21-year-old community college student who's going to China as the first American woman welder in the trades Olympics
By Mike Householder and The Associated PressMay 17, 2026
5 days ago
A 'proudly autistic' workplace expert says putting neurodivergent employees in a typical office is like dropping a polar bear in Austin, Texas
Conferences
A 'proudly autistic' workplace expert says putting neurodivergent employees in a typical office is like dropping a polar bear in Austin, Texas
By Tristan BoveMay 20, 2026
1 day ago

© 2026 Fortune Media IP Limited. All Rights Reserved. Use of this site constitutes acceptance of our Terms of Use and Privacy Policy | CA Notice at Collection and Privacy Notice | Do Not Sell/Share My Personal Information
FORTUNE is a trademark of Fortune Media IP Limited, registered in the U.S. and other countries. FORTUNE may receive compensation for some links to products and services on this website. Offers may be subject to change without notice.