• Home
  • Latest
  • Fortune 500
  • Finance
  • Tech
  • Leadership
  • Lifestyle
  • Rankings
  • Multimedia

Trendingnow

1

Ex-PepsiCo CEO Indra Nooyi worked from midnight until 5 a.m. as a receptionist to pay for her Yale degree—and she says ‘respect went up’ because of it

2

Shark Tank's Kevin O'Leary says if he were 25 today, he'd chase these two booming opportunities in the world of AI

3

China’s birth rate just hit its lowest point since 1949—and Trip.com cofounder James Liang thinks that’s a threat to innovation

1

Ex-PepsiCo CEO Indra Nooyi worked from midnight until 5 a.m. as a receptionist to pay for her Yale degree—and she says ‘respect went up’ because of it

2

Shark Tank's Kevin O'Leary says if he were 25 today, he'd chase these two booming opportunities in the world of AI

3

China’s birth rate just hit its lowest point since 1949—and Trip.com cofounder James Liang thinks that’s a threat to innovation
TechInternet of Things

The Internet of Things Will Make Big Data Look Small

By
Tom Krazit
Tom Krazit
Down Arrow Button Icon
By
Tom Krazit
Tom Krazit
Down Arrow Button Icon
March 3, 2016, 1:40 PM ET
TIME Summit On Higher Education
NEW YORK, NY - OCTOBER 18: Associate Professor of Computer Science at Stanford University Andrew Ng makes a presentation during the "Changing Landscapes: From the Digital Classroom to the Global Campus" panal during the TIME Summit On Higher Education on October 18, 2012 in New York City. (Photo by Jemal Countess/Getty Images for TIME)Jemal Countess/Getty Images
Add Fortune on Google for similar content.

It’s kind of amazing that we all settled on the term “big data” before the “Internet of things” really arrived. That pending revolution, in which we’ll see all kinds of new objects connected to the Internet thanks to the cheap hardware provided by the smartphone boom, will generate information on a scale we can’t even really comprehend yet.

Most businesses have woken up to the reality that data analysis is the lifeblood of a 21st century corporation. Yet they are not prepared for the incoming flood of data that these connected objects will provide. It’s one thing to track your customers as they flow through your website, or observe that the yellow button gets way more clicks than the red button. It’s another thing to track data created inside and outside a company about their employees, customers, and partners.

We need new technologies to make sense of this glut of information. And this is going to change the way that big data companies like Hortonworks (HDP), Cloudera (CLOUDERA), IBM (IBM), and others sell products and services.

This looming problem is something we’re sure to discuss at Structure Data, scheduled for March 9th and 10th in San Francisco. We’re featuring speakers such as William Ruh of GE, who will talk about the impact the industrial Internet will have on the manufacturing sector; Jerome Dubreuil of Samsung, who will illustrate just how much data connected home devices generate; and a panel of healthcare experts will sort through the dual challenges of the retiring baby boom generation and an explosion in quantified-self health apps.

This may sound like a buzzword salad to many of you. But those in charge of the massive players in this market are making moves to get themselves ready for the data deluge from a realized Internet of things. Hortonworks, for example, bought a company last year called Onyara that focused on finding smarter ways to analyze data than just throwing a bunch of servers at the problem. (Hortonworks CEO Rob Bearden will be speaking at Structure Data.)

For a long time, this was a tried-and-true approach to data analysis: Find a problem, collect or obtain a data set, and run a smart piece of software on a gazillion servers (if you want results before the end of the year). But the Internet of things will overwhelm this line of thinking. The sheer number of events will be too much for companies to parse.

Get Data Sheet, Fortune’s technology newsletter.

This exposes a problem in the nascent world of “big data.” Industry-wide best practices for some of the trickier problems are hard to come by, since most companies have data issues and opportunities that are unique to their business. Data scientists and engineers tend to build data infrastructure in ways they’ve seen work in the past, and when the underlying assumptions behind those methods change dramatically—like, say, when you dump hundreds of millions of additional inputs into the system—they’re going to need a new method.

That method likely involves machine learning techniques that can help data analysis systems learn which inputs to prioritize, the same way you use context clues to pick out the handful of relevant emails in your inbox each morning against the hundreds of extraneous ones.

Herein lies the opportunity for data analysis companies like Hortonworks as the Internet of things evolves. IBM, which has put most of its eggs in Watson’s basket, launched a product earlier this month that helps application developers prioritize extremely important —such as oxygen levels in a mine—over the reams of other data produced by sensors in that environment.

If the budding giants of big data—Hortonworks, Cloudera, and their ilk—don’t adjust to meet this challenge, someone surely will. And even if the Internet of things arrives bit by bit, instead of as a tsunami, the data scientists, engineers, and developers who get this right over the next few years will be in prime position for both profits and influence over the development of our connected world.

Find out how that world will evolve at Structure Data in March.

Correction (3/9/16): The original version of this article gave an incorrect last name for Hortonworks CEO Rob Bearden. The story has been updated.

About the Author
By Tom Krazit
See full bioRight Arrow Button Icon
Add Fortune on Google for similar content.

Latest in Tech

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025

Most Popular

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Fortune Secondary Logo
Rankings
  • 100 Best Companies
  • Fortune 500
  • Global 500
  • Fortune 500 Europe
  • Most Powerful Women
  • World's Most Admired Companies
  • See All Rankings
  • Lists Calendar
Sections
  • Finance
  • Fortune Crypto
  • Features
  • Leadership
  • Health
  • Commentary
  • Success
  • Retail
  • Mpw
  • Tech
  • Lifestyle
  • CEO Initiative
  • Asia
  • Politics
  • Conferences
  • Europe
  • Newsletters
  • Personal Finance
  • Environment
  • Magazine
  • Education
Customer Support
  • Frequently Asked Questions
  • Customer Service Portal
  • Privacy Policy
  • Terms Of Use
  • Single Issues For Purchase
  • International Print
Commercial Services
  • Advertising
  • Fortune Brand Studio
  • Fortune Analytics
  • Fortune Conferences
  • Business Development
  • Group Subscriptions
About Us
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • Facebook icon
  • Twitter icon
  • LinkedIn icon
  • Instagram icon
  • Pinterest icon

Latest in Tech

Presidents aren’t supposed to pick winners, former White House ethics lawyer says. Trump keeps choosing Dell
PoliticsDonald Trump
Presidents aren’t supposed to pick winners, former White House ethics lawyer says. Trump keeps choosing Dell
By Mia OsmonbekovJuly 7, 2026
7 hours ago
Palantir CEO Alex Karp with his arms outstretched while making a point on stage.
NewslettersEye on AI
Palantir CEO Alex Karp is wrong about the threat Anthropic and OpenAI pose to most enterprises. That doesn’t mean he doesn’t have something to lose
By Jeremy KahnJuly 7, 2026
8 hours ago
Scott Wu, in front of a blue background, sits in a gray chair and speaks to a person out of frame.
AIProductivity
Cognition CEO says tech companies got ‘carried away’ with token leaderboards and should measure employees on output instead
By Sasha RogelbergJuly 7, 2026
9 hours ago
Tech worker walks to office
SuccessJobs
AI start-ups are snubbing entry-level talent in favor of Silicon Valley men with top degrees, research shows
By Emma BurleighJuly 7, 2026
11 hours ago
South Korean law targeting ‘fake news’ takes effect, but journalists say it discourages critical reporting and can lead to self-censorship
AsiaSouth Korea
South Korean law targeting ‘fake news’ takes effect, but journalists say it discourages critical reporting and can lead to self-censorship
By The Associated Press and Kim Tong-HyungJuly 7, 2026
11 hours ago
Trump is building a helipad with ‘carved granite’ White House seal to handle new presidential choppers
PoliticsDonald Trump
Trump is building a helipad with ‘carved granite’ White House seal to handle new presidential choppers
By The Associated PressJuly 7, 2026
12 hours ago

Most Popular

Ex-PepsiCo CEO Indra Nooyi worked from midnight until 5 a.m. as a receptionist to pay for her Yale degree—and she says ‘respect went up’ because of it
Success
Ex-PepsiCo CEO Indra Nooyi worked from midnight until 5 a.m. as a receptionist to pay for her Yale degree—and she says ‘respect went up’ because of it
By Preston ForeJuly 6, 2026
1 day ago
Shark Tank's Kevin O'Leary says if he were 25 today, he'd chase these two booming opportunities in the world of AI
AI
Shark Tank's Kevin O'Leary says if he were 25 today, he'd chase these two booming opportunities in the world of AI
By Marco Quiroz-GutierrezJuly 5, 2026
3 days ago
China’s birth rate just hit its lowest point since 1949—and Trip.com cofounder James Liang thinks that’s a threat to innovation
Asia
China’s birth rate just hit its lowest point since 1949—and Trip.com cofounder James Liang thinks that’s a threat to innovation
By Nicholas GordonJuly 7, 2026
20 hours ago
Current price of oil as of July 6, 2026
Personal Finance
Current price of oil as of July 6, 2026
By Joseph HostetlerJuly 6, 2026
2 days ago
Even as Elon Musk calls philanthropy ‘very hard,’ everyday Americans gave a record $617 billion—despite feeling the squeeze over the cost of living
Success
Even as Elon Musk calls philanthropy ‘very hard,’ everyday Americans gave a record $617 billion—despite feeling the squeeze over the cost of living
By Preston ForeJuly 4, 2026
4 days ago
The man who ran Bernie's campaign says Democrats are still making the same mistakes with Democratic Socialists, and they should laud Mamdani's win
Politics
The man who ran Bernie's campaign says Democrats are still making the same mistakes with Democratic Socialists, and they should laud Mamdani's win
By Catherina GioinoJuly 6, 2026
1 day ago

© 2026 Fortune Media IP Limited. All Rights Reserved. Use of this site constitutes acceptance of our Terms of Use and Privacy Policy | CA Notice at Collection and Privacy Notice | Do Not Sell/Share My Personal Information
FORTUNE is a trademark of Fortune Media IP Limited, registered in the U.S. and other countries. FORTUNE may receive compensation for some links to products and services on this website. Offers may be subject to change without notice.