Hero Images—Getty Images

Why health insurance companies are doomed

Oct 20, 2014

It’s that time of the year. No, not Halloween, but something almost as scary—open enrollment season.

It’s time to choose among the many plans offered through the various health exchanges as part of Obamacare, among the variety of Medicare Advantage and prescription drug plans offered by private insurers as part of Medicare (for those who are age-eligible), and, for the 62% of employees who are have the opportunity, time to sign up for an employer-sponsored health insurance plan.

As we struggle to make sense of the health insurance landscape, it’s a good time to consider why health care costs in the United States are so high and outcomes so relatively poor and, more importantly, what the future is likely to bring.

For now, the U.S. is unique among advanced industrial economies in its reliance on private insurers to administer much of the health care payment system. But this situation is almost certainly going to change. Cost containment and competitive pressures will transform, if not doom, health insurance companies. Here’s why.

We have a performance problem

Let’s put the U.S. health care system in some comparative context. U.S. healthcare is exceedingly expensive. According to OECD data released in 2014, among 34 advanced industrialized countries, the U.S. spends $7,662 per person (adjusted for purchasing power parity differences), which is more than 2.6 times the OECD average. The U.S. devotes 16.9% of its GDP to health care, 1.8 times as much as the average. In the case of health care spending measured any way you want, the U.S. is No. 1 by a large margin.

Despite all that spending, America’s health system does not perform particularly well. That same OECD report shows that the U.S. ranks 27th for life expectancy at birth. This comparatively low ranking is not merely a consequence of higher infant mortality, where the U.S. ranks a dismal 53rd in deaths per 1,000 live births. Even considering life expectancy for men aged 65 places the U.S. in 23rd place.

The U.S. ranks so poorly on health outcomes partly because it is the only advanced industrialized economy that has not provided health care to everyone, a situation that persists even after the passage of the Affordable Care Act. Not having health insurance adversely affects access to health care, which in turn affects mortality and morbidity. One Urban Institute analysis estimated there were about 22,000 excess deaths annually because of a lack of universal access to care, while a study published in the American Journal of Public Health calculated that there were approximately 45,000 excess deaths in 2005 because of the absence of universal health coverage. Numerous research studies have analyzed some of the pathways that lead to these excess deaths, including reduced use of preventive screenings among the uninsured, which means that disease is detected later when it is more difficult and expensive to treat.

U.S. health care costs are not only high, but they continue to rise and more of those costs are being shifted to individuals. For instance, even though overall health cost inflation has been curtailed by the recent recession, in 2015, the cost for covering my spouse and myself through Stanford-provided health insurance will go up by an astounding 25.5%. Although this is an unusually large increase, the Kaiser Family Foundation reports that in the last decade, employee premiums have increased more rapidly than total health insurance costs as employers—using a combination of increasing premiums, deductibles, and copayments—have relentlessly shifted health care costs to their workers.

A bureaucracy that would make Kafka blush

It’s June 2014, and unfortunately I need back surgery. My back surgeon, having carefully reviewed my MRI, turns me over to his “scheduler,” a full-time employee who spends all day every day on the phone talking to insurance company functionaries (or waiting on hold to speak to them) to get authorization to deliver medical care. This staffing is typical of most medical offices, particularly for specialists, who confront endless forms, prior authorizations, and other manifestations of bureaucracy. And then there are the other people doctors and hospitals have to hire to hound the insurance companies for payment once the care is delivered.

The time spent dealing with insurance intermediaries costs money and aggravates physicians and their patients. As one doctor lamented in a column in The Wall Street Journal, “U.S. doctors spend almost an hour on average each day, and $83,000 a year … with the paperwork of insurance companies.” And for every call coming from a doctor’s office or hospital to an insurance company, there is someone at that insurance company on the other end of the line to process the call.

In 2012, more than 460,000 people were working in the health insurance industry, and employment growth in health insurance is much higher than for the providers of actual health care. Of course, managing all these people is expensive—very expensive. In 2011, the CEO of Blue Shield of California made $4.6 million and the organization’s top 10 executives earned $14 million in total, although of course none of them did any medical research or delivered any care to real patients. The Affordable Care Act mandated that health insurers had to spend at least 80% of their collected premiums on medical care. The very inclusion of that provision implies that at least some health insurers had overhead rates in excess of 20%. All of this seems expensive and wasteful, and it is.

There are other models. Kaiser Permanente, a health care provider that combines both a health insurance plan and care delivery in a single organization, serves more than 9 million members and employs more than 17,000 physicians. Without a separate insurance company intermediary, Kaiser saves money, which it then uses to offer lower health insurance rates. For instance, for 2015, the total cost (Stanford’s portion plus my contributions) of purchasing the Blue Shield-administered plan in which my wife and I participate is 39.7% higher than the cost of getting health insurance through Kaiser Permanente. Although many factors might explain this huge difference in price, the simplification of access and the reduction of administrative overhead is a big part of the story. Kaiser, by being both health plan administrator and care provider, has eliminated the insurance company intermediaries that raise administrative costs.

To be clear, that’s precisely what insurance companies are—intermediaries. Health insurers receive payments from employers, individuals, and governments and then send that money to health care providers such as pharmacy benefits managers, doctors, and hospitals, of course keeping some for themselves to cover overhead and, in some instances, profit. To take one informative example, Stanford University, which has its own medical school, hospital, and doctors, sends money to Blue Shield on behalf of those employees who use Stanford Medical Center services (and others). And then Blue Shield sends that money back to Stanford for the services Stanford renders to its own employees! The obvious questions are: how much does this intermediation cost and what valuable purpose does it serve?

Let’s consider evidence on the cost issue first. In 1991, Steffie Woolhandler and David Himmelstein, two Harvard doctors with an interest in health policy, published a paper in The New England Journal of Medicine in which they estimated that health care administration constituted somewhere between 19% and 24% of total spending on health care, an amount that was 117% higher than what it was in Canada and much more than in the U.K.

About a decade later, the researchers decided to revisit their earlier examination of U.S. healthcare administrative inefficiency. They wanted to determine whether changes in the medical care system—including the rise of managed care and numerous hospital mergers, and changes in technology, including the growing use of computers and the Internet—had changed the administrative burden.

They found that things were worse.

Their updated estimate, once again published in The New England Journal of Medicine, found that administration accounted for about 31% of health care spending and that more than 27% of all of the people employed in health care worked in administrative and clerical occupations.

This large administrative expense is not surprising. It costs money for health care providers to deal with multiple insurers, each with its own protocols, forms, and requirements. And it costs money for insurers to be able to transact with multiple providers and to furnish the oversight—which many would consider more annoying than helpful—of health care delivery.

What do we get for all that spending?

You may be thinking, “All right, insurers certainly cost money. But don’t they and their health insurance brethren help control costs as they efficiently administer care delivery?”

Not really. First of all, health insurance companies often improperly reject claims and deny coverage. As just one example, a recently reported audit of Medicare Advantage plans conducted by the Center for Medicare and Medicaid services found that “in 61 percent of audits, insurers ‘inappropriately rejected claims’ for prescription drugs” and “in more than half of all audits, ‘beneficiaries and providers did not receive an adequate or accurate rationale for the denial’ of coverage when insurers refused to provide or pay for care.”

In some cases, such as the use of advanced imaging services, there is evidence that health insurers have helped constrain inappropriate overuse. Of course, any savings come at a cost. One physician, complaining about prior authorization as “a wasteful administrative nightmare,” noted that “prior authorization requests consumed about 20 hours a week per medical practice.” And studies of prior authorization for prescription drugs tend not to find savings. For instance, a study of the effects of prior authorization for Type 2 diabetes prescriptions reported higher costs for patients who requested authorization for a medication and did not receive it. The article concluded that “failure of a member to take a medication deemed necessary by his or her physician could translate into inadequate control of the diabetic condition and result in an excess of resource utilization and costs for treating the disease.”

Why has nothing changed? Lobbyists.

If insurance intermediaries frustrate doctors, vex patients, drive up costs, and provide few consistent benefits in administering health care, a reasonable person might wonder how they have maintained their central role in health care for so long? The answer: particularly when health care reform gets debated, the health insurance industry spends a lot of money to make sure its interests are well-served. After all, health spending in America is approaching $3 trillion, and 30%—the estimated overhead rate—of $3 trillion is an enormous sum.

When the Affordable Care Act was proposed and debated, a report carried by CNN noted that between lobbyists, political donations, and television advertising, the various health care constituencies including hospitals, doctors groups, and health insurance companies spent $375 million. Another news report noted that America’s Health Insurance Plans, an industry association representing the health insurance industry, funneled more than $100 million through the U.S. Chamber of Commerce in an effort to defeat Obamacare. Consulting a database of reported lobbying expenditures (which understates the money spent attempting to influence policy) reveals that in 2013, some $154 million was spent by insurance lobbyists. The two biggest spenders were Blue Cross/Blue Shield and America’s Health Insurance Plans.

This pot is about to boil over

Although the health insurance industry is powerful, betting on their survival seems like a long shot. That’s because increasing levels of competition among health systems, particularly in large metropolitan areas, will eventually cause many large health care providers to do what Kaiser Permanente has already done—offer their own health plans and disintermediate the insurance intermediaries.

In June 2014, I gave a talk to a group of health care executives from Sutter Health as part of a leadership development program. Sutter is a large, multi-unit health care organization in Northern California that owns both hospitals and some physicians’ practices. The people in the room, including one member of the board of directors, were stressed. Known as a “high-cost” provider, Sutter has been losing market share to Kaiser Permanente and was worried about the emergence of other potent competitors. For instance, Stanford’s medical center had begun to offer Stanford employees (and, in the future, probably the public) access to Stanford hospital and doctors and at Kaiser prices. Although initially Stanford relied on Blue Shield, because Stanford Medical Center was not yet licensed by the state to offer health insurance, that could change, particularly as the medical center purchased physician practices throughout the Bay Area.

Many people of all political stripes have long advocated for more competition in health care. That competition is coming, particularly in large metropolitan areas.

The Sutter group’s conclusion: if they are to compete, they have to offer their own health plan to see the full patient picture and reduce overhead costs. They have such a plan, Sutter Health Plus, which began earlier this year in the Sacramento and San Joaquin Valley areas. It now has more than 7,300 members, a number expected to double by 2015. The plan includes 1,800 physicians and a multi-hospital network. Sutter needs to expand this offering throughout Northern California, and many Sutter employees know it. To take one example of why this is a prudent move, a person from their Palo Alto Medical Foundation, a doctor’s group serving the South Bay, reported that Blue Shield negotiated lower prices from PAMF and then did not fully pass those reduced prices on to employers, leaving Sutter disadvantaged in its pricing during open enrollment.

To be sure, Sutter’s story concerns just one health care system in one region of the U.S. And while Kaiser Permanente operates in many locales throughout the U.S., it is far from ubiquitous. But the evidence demonstrates that one important source of excess health care costs in the U.S. is administrative overhead, as physicians and pharmacy staff spend more and more time learning various insurance company rules and procedures. As cost containment pressures increase and as more health care systems introduce their own insurance offerings, those that rely on intermediaries will be at a major disadvantage.

This should not come as a big shock. After all, in industry after industry, ranging from travel to publishing to finance, we have witnessed the disruptive effects of disintermediation or its beginnings. Travel agencies shrank as airlines and hotels moved distribution to the Internet and user ratings permitted people access to information on quality. Self-publishing grows as companies such as Book Baby, Lulu, and others make it possible for anyone to produce and sell books online, reducing the role of traditional publishers. Bloomberg recently reported on disintermediation in finance, with banks left out as lenders and borrowers connect directly. There are websites and services that match car drivers with passengers and eBay matches buyers of merchandise with sellers. There is absolutely no reason to believe that the $3 trillion health care industry will be immune to pressures to eliminate or reduce the costs of intermediaries that do not make the process any easier or cheaper.

The late economist Herbert Stein once said, “if something cannot go on forever, it will stop.” The ever-expanding administrative costs of health care will eventually reach a limit, and this will happen sooner than many people expect. Health insurers’ dismal customer service and high overhead costs make them easy and inevitable targets. These companies’ survival depends on fundamentally reinventing their business models. Otherwise, they are doomed.

Jeffrey Pfeffer is Thomas D. Dee II Professor of Organizational Behavior at Stanford’s Graduate School of Business. His latest book, Leadership B.S.: Fixing Workplaces and Careers One Truth at a Time will be published in September, 2015 by HarperCollins.

Editor's note: A previous version of this story incorrectly stated that the Sutter Health Plus plan was initially offered just to Sutter Health employees. Sutter's employees were not given initial access to this particular plan.

All products and services featured are based solely on editorial selection. FORTUNE may receive compensation for some links to products and services on this website.

Quotes delayed at least 15 minutes. Market data provided by Interactive Data. ETF and Mutual Fund data provided by Morningstar, Inc. Dow Jones Terms & Conditions: http://www.djindexes.com/mdsidx/html/tandc/indexestandcs.html. S&P Index data is the property of Chicago Mercantile Exchange Inc. and its licensors. All rights reserved. Terms & Conditions. Powered and implemented by Interactive Data Managed Solutions