Skip to Content

Why Big Data will never beat business intuition

(TheMIX) — Big Data is big business. Sensors, GPS tracking, math modeling, and artificial intelligence offer companies real-time market insights at massive scale and open the door to unprecedented ways of monitoring, targeting, and measuring employees and customers. Analyst firm Gartner predicts that enterprises adopting Big Data technologies will “outperform competitors by 20 percent in every available financial metric.”

Big Data might well be “the new oil,” but I would caution us not to worship it as the new religion. Amidst all the data frenzy, we are not only losing a more holistic view of business but also a part of our humanity. How much space do we leave for creativity if we equate better living with better algorithms?

I am not a dataphobe, but I am concerned about relying only on data. I am not against quantitative metrics, but I question their authority as the main indicators of business performance, prosperous societies, and meaningful lives.

Big Data comes with many benefits, but let’s complement it with Big Intuition. Here are six reasons why:

Big Data = Big Brother? The New York Times’ Steve Lohr describes Big Data as a descendant of Taylor’s “scientific management.” Instead of performance in the workplace, which was the focus of Taylorism, we are now measuring happiness and well being, our consumption preferences, social interactions, physical activities, our attitudes, moods, emotions, behaviors, and bodily functions — in other words, we are measuring our lives.

Sure, to some degree, “quantified self” apps may empower people to take more control over their decisions. However, by doing so, we are opening up once-private terrain to the business world, all under the mandate of self-improvement.

Big Data is not social. We humans are social animals. Research shows that relationships, especially friendship and marriage, are key factors of happiness and fulfillment. Our brains are wired to care, and our hearts and minds have developed an astounding capacity to empathize and sympathize with fellow humans. We can show compassion, sense mood swings, detect subtle non-verbal cues, tolerate or embrace, accept and reject, love and hurt, experience with all of our senses, act irrationally, and lose our self-control. These key traits of our humanity are threatened by the “mathematization of subjectivity,” as Leon Wieseltier calls it.

Recent social genomics studies suggest that not only our productivity, but also our evolutionary capacity to connect with others is diminished by digital overload.

Big Data creates small worlds. Morality is gained by way of empathy. Paradoxically, in our age of hyper-connectivity we are increasingly facing the challenge of connecting with people whose opinions, values, beliefs, faith, and culture may be unlike ours. As digital technology customizes our social experiences, online and offline, based on our preferences, we are increasingly stuck in our own worlds — the “Filter Bubble,” as Eli Pariser called it, designed by smart algorithms to serve us with content, culture, and company that we are already familiar with and that fall squarely within our comfort zones. We don’t “like” the people and things that are unlike us, feeding a vicious cycle of social and cultural narrow-mindedness.

Big Data makes us smarter, not wiser. Our data-driven worlds are not only becoming smaller, they are becoming faster. The real-time flow of information persuades us to react to feedback constantly and instantly. Playing on the title of Alvin Toffler’s 1970 book Future Shock, Douglas Rushkoff calls our current state-of-mind Present Shock, lamenting “a diminishment of everything that isn’t happening right now — and the onslaught of everything that supposedly is.”

Data might give us information fast, but for quick but profound decisions, intuition is much better. Prasad Kaipa and Navi Radjou, in a recent book, urge business leaders to move “from smart to wise.” They have a point. Smart organizations and leaders thrive on constant feedback. Smart is fast. Wise, however, is slow. Wise organizations and leaders need time and take it.

Big Data is (too) obvious. “You can only manage what you measure”—really? The financial crisis has shown that we manage poorly what we measure. And failed mergers, failed product launches, reputational crises, and social media disasters, indicate that we need to get better at managing what we cannot measure.

Leaders need to have “opposable minds,” as design thinker Roger Martin puts it. The business leader of the 21st century will no longer be judged by how much uncertainty he or she can eliminate but by how much uncertainty he or she can tolerate.

Big Data doesn’t give (or forgive). Data might be able to predict new problems or find new solutions to existing problems, but only human intuition and ingenuity can come up with groundbreaking new ideas. That is a uniquely human gift—one that goes beyond merely fixing a problem or meeting a functional need.

By the same token, if we quantify all of our relationships, we will not leave any wiggle room for human discretion. Because we often have mixed feelings about people and their behavior, our judgment can be more than just binary. This means we can assess and respond to ambivalent behaviors with more nuance, and we can choose to accept failure as a prerequisite of innovation. It is hard to see how we can make progress towards any goal without an ability to forgive.

Let’s resist the rush to data and take the time to lean back so we can be fast when it matters. Let’s grant ourselves a data moratorium from time to time that we can use to reflect on what really is important. Let’s use data to tell our stories, but let’s not allow data become our only story.