In July 2011, Floyd Yager, then senior vice president of analytics at Allstate Insurance, huddled in a conference room with Suren Gupta, the company’s chief information officer. Yager had an idea. He had heard about open source software called Hadoop that could process millions of datasets in amazing speed on a just a dozen computers. Yager had a small team that could mine this information but he needed Gupta’s buy in: the computers, the storage, the network and the capacity to go forth. Could the company’s information technology department work with him to get it done?
Gupta wasn’t sure Hadoop was necessary. For decades Allstate (ALL) used smaller chunks of data to dissect its customers to analyze risk and pricing. “Why do you need so much data?” he asked. “And what would you even do with it?” The potential, Yager explained, could be big.
The faster processing technology meant Allstate could look at millions of data points instead of thousands and sort through them in a matter of hours instead of months. The insurance giant could dig through troves of customer information—policies, claims, quotes—to uncover patterns and make more precise predictions about what customers really wanted. It would be able to pinpoint which customers might switch to the competition, which price points worked to attract new ones, which home policies required physical inspections or which claims were fraudulent. The list could be endless.
At the time, Allstate executives faced big problems. The Northbrook, Ill. insurer shed customers after cutting coverage in hurricane-prone regions. Fraud was rampant in New York and Florida and agency compensation in flux. That year, the company lost $625 million, its market share slipped and its stock hit a low of $22.70 per share, marking a steep slide from $65 in late 2005.
Investors criticized CEO Thomas Wilson despite efforts to right the ship with a cost-cutting reorganization. Yet much of the problems were out of Wilson’s control: Allstate had been hit by payouts for big storms, agency compensation changes were controversial, and the planned retirement of senior executives looked to outsiders like a management exodus. “That criticism of Wilson wasn’t entirely fair,” says Paul Newsome, an analyst at investment banking brokerage Sandler O’Neil and Partners.
Then came another bit of bad news: the abrupt departure of Joseph Lacher, the president in charge of turning around the home and auto insurance operations. The departure stunned investors. Says Meyer Shields of the investment banking firm Keefe, Bruyette & Woods: “Everyone saw a company in disarray.”
That was three years ago, about the same time buzz began percolating over this new idea of big data and the dizzying amounts of terabytes and zettabytes proliferating inside companies and from social media and mobile phones. Then, and even now, few companies fully understand how to best use those bits and bytes. They furiously collect and store the information, only to do nothing with it.
“Everyone is saying big data is going to change the world,” Yager says. “But companies have to figure out what’s important to them and get out of the hyped world.”
Culling random customers’ rants or raves on social media wouldn’t be very informative to Yager’s team. (After all, Allstate had an entire team dedicated to handling social media chatter.) The real gold was already piled up inside company databases and data storage facilities: years’ worth of customer information, transaction history, policy pricing, quotes and claims. Yager had spent his 25-year career at Allstate crunching that kind of data but it was always to develop rating plans and make underwriting decisions—in essence determining who to insure and at what price.
The rise of Hadoop and big data offered something new: Instead of looking at small samples of data, the insurer could for the first time look at the entirety of its customer base at once. It could segment it and precisely know what Allstate customers look like and what kind of risk the company was really taking on. Allstate could now use that data to drive decision-making elsewhere in the business, from identifying fraudulent claims to developing better marketing and sales campaigns. Instead of sending direct mail to every customer, for instance, Allstate can use customer data to target only those people most likely to act. “The growing processing power allows us to see so much more,” Yager says.
After that first conversation in 2011, Gupta agreed to provide the IT assistance for a few small data tests. They would pull 50 million price quotes and try to pinpoint the price elasticity of customers. If the information didn’t yield insights, there would be nothing lost; they’d move on.
Yager went online and bought a book about Hadoop. His team started traveling to conferences to meet vendors and consultants and find out how they were using new data science to improve their businesses. A six-person team then pulled a few unused computers out of storage (to cut down on costs), wired them together, and used Hadoop software to mine insurance quotes. They looked for patterns in customer behavior: How did 30- to 40-year-olds in the Midwest respond after getting an auto quote on a 2010 model minivan? How did customers with newly built homes react when Allstate raised their home insurance rates? Allstate already knew from small data samples that some variables made a difference to close a sale and keep a customer, but Yager wanted to know if the bigger data sets offered even more knowledge. Which ones were especially price sensitive? Which ones more loyal?
Eventually, Yager’s team uncovered the price point and policy features that drove prospective customers to the competition, which policy features offered more value, and which prices led to agents signing a customer. Even more surprising to Yager and his team? The speed. Information that would have ordinarily taken months to analyze could be done in less than one day.
That was the first of many tests. By 2012, big data had become a big deal at Allstate. Wilson touted the potential of big data on stage at CEO and investor roundtables. He also promoted Yager to chief data officer. Instead of reporting to Gupta in IT, Yager worked under Matthew Winter, the new president in charge of auto and home insurance (which comprises 90% of Allstate’s customers), and offered mounds of useful data.
Yager started giving presentations to senior executives and talking about this vague idea of big data, what having more data actually meant to the company and how new algorithms and predictive modeling could lead to more precise information and better business decisions. In the past, executives might have taken faith in customer surveys showing that customers would refer friends to Allstate. Yet those surveys don’t tell whether people actually referred their friends. Analytics can show execs in real time why new customers chose the insurer, offering a game plan for Allstate to replicate. For Allstate to take advantage of data, executives needed to start asking how analytics could answer their questions and drive their decision-making.
So executives started asking. How can claim investigators figure out which claims are most likely fraudulent? It turns out, computers could look at thousands more claims than any individual claims adjuster might see in a year. Using software called Tableau along with Hadoop, investigators could speed up data surfing to see suspicious activity. They could easily find when a clinic provider billed for procedures on a Sunday—a day the clinic was closed. (Red flag.) They could spot when the same patient was treated the same day at multiple clinics that were separated by distances too great for a person to physically travel between appointments. This kind of data helped Allstate investigators turn in one Florida clinic owner who was prosecuted for fraud, one of hundreds of cases caught, identified and confirmed with the help of analytics.
Another question: Which homeowners policies needed an inspector to see if a home qualified for coverage? Allstate had that information too, hidden in three years worth of data like where a home is located, its features (like an unprotected swimming pool), when it was built, and past claims on the property (such as a damaged roof). With predictive algorithms using that kind of historical data, Allstate cut down unnecessary inspections by 20%, a move that saved $3 million. The financial savings was small but it will likely grow as analytics improve, says Julie Parsons, senior vice president of product operations. More important, she says, is that it cut out a huge hassle for many customers who didn’t have to take time to meet an inspector at their house. “It’s a better customer experience,” she says.
Today, Allstate has an entire data science department staffed with 60 people who work hand-in-hand with some 200 managers across the company in claims, sales and marketing on other projects. Allstate is still recruiting and training and on track to double the department’s size again this year.
The overall company appears to be slowly be turning the corner, thanks to Wilson’s broader restructuring, which involved reorganizing agencies, changing compensation plans and selling its Lincoln Benefit Life insurance company. Allstate grew revenues 3.6% last year to $34.5 billion and grew operating income by 24.3% to to $2.7 billion. In its most recent quarter, it reported revenues of $8.94 billion, nearly a 6% increase from the year-ago quarter. “They decreased their risk profile quite a bit, and they’re a significantly stronger company than they were in 2011,” Newsome says. The company’s stock price has nearly tripled, to $63.68 a share, from $22.22 in September 2011.
Yet with a compound annual growth rate of 3.3% over the past five years, Allstate still lags behind the industry’s 10% average. The company also faces fierce competition from the fast-growing Progressive (PGR) and Berkshire Hathaway Inc.’s Geico unit (BRK-A), which sell auto policies over the Internet and advertise lower rates than Allstate’s agent-sold policies. And virtually all insurers are focused on big data: Nationwide Insurance hired a chief data officer last fall, while MetLife intends to hire 1,000 technologists this year and to put $300 million into technology to better understand customers.
Allstate executives contend their predictive algorithms are just beginning to make a difference; they’ve already led to better rating plans, faster growth in auto policies and better customer retention. Shields, who was skeptical of Allstate in 2011, believes that big data helped the company’s turnaround. “It means much less guesswork,” he says. “It’s now the lifeblood of the business.”
