It’s hard enough staying on top of the latest developments in the technology industry. That’s doubly true in the fast-growing area known as big data, with new companies, products and services popping up practically every day.
There are scores of promising big data companies, but Fortune sought to cut through the noise and reached out to a number of luminaries in the field to ask which big data companies they believe have the biggest potential. Which players are really the ones to watch?
That question, we learned, is rather difficult to answer.
“A list of ‘big data companies’ is interesting because of the definition,” said Dean Abbott, co-founder and chief data scientist of Smarter Remarketer. “Is a ‘big data’ company one that is pure-play analytics with big data? A company that does predictive analytics often with big data but not always—like Beyond the Arc or Elder Research? A large company that has a group that usually does ‘big data analytics’ but it’s only a small part of what they do as a company—like HP (HP), SAP, Dell, even GE (GE)?”
‘One of the most interesting ones I’ve seen’
There was certainly consensus on some of the big data companies that industry experts said were notable. At least two of 10 experts we polled named MapR, MemSQL, Databricks, Platfora, Splunk, Teradata, Palantir, Premise, Datameer, Cloudera, Hortonworks, MongoDB, and Trifacta as shining examples in the space.
MemSQL, for example, is “an in-memory relational database that would be effective for mixed workloads and for analytics,” said Svetlana Sicular, an analyst at Gartner. “While SAP draws so much attention to in-memory databases by marketing their Hana database, MemSQL seems to be a less expensive and more agile solution in this space.”
Splunk, meanwhile, “has an excellent technology, and it was among the first big data companies to go public,” Sicular said. “Now, Splunk also has a strong product called Hunk—Splunk on Hadoop—directly delivering big data solutions that are more mature than most products in the market. Hunk is easy to use compared to many big data products, and generally, most customers I spoke with expressed their love to Splunk without any soliciting on my side.”
Palantir Technologies, which focuses on data analysis for public sector clients, also received high marks. “I’d have to put Palantir at the top of the list” of the startups in the big data space, said Tom Davenport, an IT management professor at Babson College and author of the book, Big Data @ Work.
DJ Patil, vice president of product at RelateIQ, said San Francisco-based Trifacta—which makes a “data transformation platform” promising increase productivity for data analysts—was a company to watch. “One of the most interesting ones I’ve seen,” said Patil, who serves as a technical advisor to the company.
Two miles away, cross-town peer Datameer is also remarkable, said Carla Gentry, founder of Analytical-Solution. “There are lots more companies out there, but most of the them just end up being a BI platform that anyone with an engineer could have started.” Datameer is different, she said.
‘Graphs have a great future’
Some of the less well-known companies received the highest praise.
Tamr, for instance, is “an exciting startup in data curation, so that would be my nomination,” said Gregory Piatetsky-Shapiro, president and editor of KDnuggets.com.
Neo Technology, the company behind open source graph database Neo4j, is another that Gartner’s Sicular pointed out. “I think graphs have a great future since they show data in its connections rather than a traditional atomic view,” she said. “Graph technologies are mostly unexplored by the enterprises but they are the solution that can deliver truly new insights from data.” (She also named Pivotal, The Hive and Concurrent.)
DataStax, WibiData, Aerospike, Ayasdi and ClearStory were all part of analyst Curt Monash‘s “obvious inclusion” list, he said, while Automatic, Planet Labs, Sight Machine, DataPad, Interana, Wise.io, LendUp, Declara, Sentinel Labs, Fliptop, Sift Science, Import.io and Segment.io were among those named by data scientist Pete Skomoroch.
Paxata and Informatica were both cited by Ovum analyst Tony Baer; IBM (IBM), Syntasa, Actian and Tableau were four named by George Mason University professor and data scientist Kirk Borne.
“There are a number of startups in security and machine learning that are emerging,” Baer said. “What’s missing right now are startups that look at data governance, stewardship, lifecycle management for big data. Right now IBM is largely alone, but I’m expecting there will be more startup action to come.”
‘Most of these companies will go away’
If you’ve reached this point in the article, you will have read 42 recommendations by our panel of experts. All of them are foremost technology companies; most exist specifically to perpetuate big data technology.
But some experts said that the most interesting big data companies aren’t big data companies at all. Established companies with traditional products and services are starting to develop offerings based on big data, Davenport said. Those include agriculture giant Monsanto (MOO), back-office operations stalwart Intuit (INTU), and the trucking company Schneider.
“To me, it’s much easier to create a startup than it is to change your entire way of thinking about information and how it relates to your business operation,” Davenport said. “One of the really exciting things about big data is when you use it to create new products and services.”
He added with hesitation: “It’s early days still, and we don’t know how easy it will be for companies to make money off these things.”
It is inevitable that there will eventually be a thinning of the big data herd, experts said.
“Most of these companies will go away because the most important part of the big data movement will be how to use data operationally—to make decisions for the business,” Smarter Remarketer’s Abbott said, “rather than who can merely crunch more data faster.”
