FORTUNE — The Obama Administration thinks it can prevent terrorism by collecting and linking immense volumes of private phone call, credit card, and Internet data. Unfortunately, as the administration trades privacy for intelligence insights, President Obama is exchanging a cherished American value for an unproved theory.
The Obama Administration loves data. It argues that the president was reelected because of its enormous people database and use of polling and social media. And the Affordable Care Act is built in part on a belief that massive troves of electronic health records will pave the way for predictive algorithms that will prevent costly hospital readmissions, or identify medical practices that lead to better patient health.
But the evidence for big data is scant at best. To date, large fields of data have generated meaningful insights at times, but not on the scale many have promised. This disappointment has been documented in the
Wall Street Journal
,
Information Week
, and SmartData Collective. Yet, for years now, corporations and public organizations have been busy buying huge servers and business intelligence software, pushed by technology providers and consultants armed with sales pitches with colorful anecdotes such as the Moneyball story in which general manager Billy Beane triumphed by using player statistics to predict the winning strategies for the Oakland A’s baseball team. If it worked for Billy Beane, it will work for your global multinational, too, right? Well, no.
MORE: Spying: Good for business — as long as it stays secret
The worship of big data is not new. Twenty-five years ago, technology salespeople peddled data using an old story about a retailer that spotted a correlation between diaper purchases and beer drinking, allowing a juicy cross-promotion of the two products for young fathers. Today, most data warehouses are glorified repositories of transaction data, with very little intelligence.
Working with multinationals as a management consultant, I have chased big data insights all my life and have never found them. What I have learned, however, is that local data has a lot of value. Put another way, big data is pretty useless, but small data is a rich source of insights. The probability of discovering new relationships at a local, highly contextual level and expanding it to universal insights is much higher than of uncovering a new law from the massive crunching of large amounts of data.
According to my firm’s research, local farmers in India with tiny fields frequently outperform — in productivity and sustainability — a predictive global model developed by one of the world’s leading agrochemical companies. Why? Because they develop unique planting, fertilizing, or harvesting practices linked to the uniqueness of their soil, their weather pattern, or the rare utilization of some compost. There is more to learn from a local Indian outlier than from building a giant multivariate yield prediction model of all farms in the world. The same is true for terrorism. Don’t look for a needle in a giant haystack. Find one needle in a small clump of hay and see whether similar clumps of hay also contain needles.
You need local knowledge to glean insights from any data. I once ran a data-mining project with Wal-Mart (WMT) where we tried to figure out sales patterns in New England. One of the questions was, “Why are our gun sales lower in Massachusetts than in other states, even accounting for the liberal bias of the state?” The answer: There were city ordinances prohibiting the sale of guns in many towns. I still remember the disappointed look of my client when he realized the answer had come from a few phone calls to store managers rather than from a multivariate regression model.
MORE: Robots with your face want to invade workplaces and hospitals
So, please, Mr. President, stop building your giant database in the sky and quit hoping that algorithm experts will generate a terrorist prevention strategy from that data. Instead, rely on your people in the field to detect suspicious local patterns of behavior, communication, or spending, then aggregate data for the folks involved and let your data hounds loose on these focused samples. You and I will both sleep better. And I won’t have to worry about who is lurking in the shadows of my business or bedroom.
Francis Gouillart is the president of Experience Co-Creation Partnership, a management education and consulting firm in Concord, Mass. and author of The Power of Co-Creation.
