When the history of sugar is written, 2016 may go down as the year its image turned. Sure, we always knew those sweet white crystals could rot teeth and cause people to pack on the pounds. Obesity and diabetes were already national emergencies, with the latter representing 10% of U.S. health care costs in recent years.
But now an increasing number of researchers and a buzzy book, The Case Against Sugar, have also begun linking our favorite natural sweetener to such dreaded conditions as heart disease, Alzheimer’s, and cancer. (Those conclusions are far from universally accepted to this point.) Adding a dark undercurrent, revelations in the fall suggested that the sugar industry paid Harvard scientists in the 1960s to trivialize its role in coronary problems and instead play up saturated fat as the culprit—which helped shape the direction of nutrition research to this day.
“Sugar is the new tobacco” in the minds of the public, says David Turner, a global food and drink analyst at market research firm Mintel.
And a public health war has followed. A handful of cities have responded with taxes on sugary drinks, and next year the Food and Drug Administration will start requiring companies to reveal the amount of added sugars on product packaging.
Consumers seem to be hearing the message. Research firm NPD Group has found that sugar is now the No. 1 substance they are trying to cut or eliminate from their diets. Of course, “try” is the operative word here. Whatever their aspirations, people keep right on gorging. Americans now eat a total of 76 pounds in various sugars every year, up 8% from 1970.
That’s the problem for Big Food: It’s built on the stuff. Some 74% of packaged foods and beverages in the U.S. contain some form of sweetener, according to a recent study in The Lancet, making it a more than $100 billion market. Companies “use hedonic substances, and sugar is the most ubiquitous hedonic substance,” says Robert Lustig, a professor at the University of California at San Francisco’s School of Medicine and a leading critical voice on the topic. That’s an academic way of saying that food companies know customers crave a sugar rush.
All of this has infused new urgency in the multi-decade quest to find low-calorie sweeteners. And it’s occurring just as the incumbent alternatives face ever more skepticism. According to Mintel, 39% of consumers think it’s best to avoid products containing artificial ingredients like aspartame and saccharin because of perceptions of health risks. Sales of such substitutes fell 13% between 2011 and 2016.
That brings us to the final factor that is pressing heavily on packaged food companies: the ever-more-ravenous appetite for “natural,” unprocessed products. “Health and wellness used to be synonymous with reduced caloric intake,” says Bernstein analyst Ali Dibadj. By that standard, as he puts it, referring to the iconic early diet cola, “Tab used to be damn healthy.” No longer.
Think of food companies’ plight this way: The finest scientists in industry have spent decades trying to find or invent a no-calorie sweetener that tastes and feels as good as the stuff extracted from pure cane. And now, after they largely failed to master that complex, arduous task, the level of difficulty is being raised even higher: This improbable concoction cannot appear to have been engineered by scientists.
Let us pause here to acknowledge the sugar-frosted codependent embrace of Big Food and the American consumer. You could rightly fault consumers for their insistence on an oxymoronic product. But who has been indulging their fantasies for decades now, promising sweet, satisfying taste and no calories? Big Food, of course. Now customers are upping the stakes—and it’s not at all clear that companies can pass the test.
It’s no accident that sugar has been a fixture in food since around 10,000 years ago. Historians trace the first use of sugarcane to New Guinea. By 500 BC, some farmers in India were transforming the cane into raw sugar. Sucrose, as it’s known to scientists, is a nearly perfect compound. It preserves. It ferments and caramelizes. It provides viscosity and mouthfeel, texture, and bulk. It enhances the flavors of other ingredients. As even a toddler could tell you, it is the gold standard of sweetness.
That hasn’t stopped multiple pretenders from trying to one-up it. A succession of sweeteners—saccharin, aspartame (best known today as Equal), and sucralose (Splenda)—have all swaggered onto the scene over the decades, each claiming maximal sweetness and minimal calories. Each product has followed a similar trajectory: Initially presented as a scientific miracle, with assurances that it tastes just like the original, it enjoys a stretch of popularity before eventually disappointing consumers with strange aftertastes or, in some cases, concerns about its effects on health.
The most recent low-cal marvel to reach the market is stevia—the best-known brand is Truvia—and it’s hailed as natural because it is derived from the leaf of a plant. Stevia’s story illustrates the opportunity—and the conundrum—of the sugar-replacement mission, which is why I’ve traveled to Illinois to visit PureCircle, the largest producer in the world.
The winter weather does not agree with stevia, which thrives in warm climates. “You’ll have to forgive our plants,” Faith Son, head of global marketing and innovation, says apologetically as she eyes a struggling seedling on a conference room table. “It’s January in Chicago.” Her company has built its entire business on the small green leaves that Son has placed in front of us.
Stevia’s sweetness is unexpected. For starters, it’s extracted not from a fruit, but from the plant’s leaves. More surprising, though, is the way the sweetness lingers. Even a few minutes after I bite, chew, and spit, the essence remains in my mouth. “It’s almost like a freak of nature,” Son says.
The compounds that give the plant its magical properties are called steviol glycosides. They are 100 to 350 times as sweet as sugar and make up only a fraction of the leaf’s weight. It’s what the industry calls a high-intensity sweetener, which delivers dulcitude but not any of the other attributes of sugar, like its mouthfeel or texture.
Stevia was supposed to be the answer to the great dilemma. But like its predecessors, it’s burdened with a tragic flaw: A persistent bitter undertaste conferred by some of the most commonly used steviol glycosides, along with a metallic and licorice quality. “Tastes like sucking on a penny!” is not the type of selling point food manufacturers are looking to plaster on a label.
Scientists believed that an antidote to the bitterness lurked inside the more than 40 different glycosides that have since been discovered in the leaf. At Cargill, one of PureCircle’s stevia competitors, scientists initially believed there was one bad actor, a single glycoside it could remove to solve the taste issues.
See more: ‘Healthy’ May Be Getting a New Definition
It turned out to be more complicated. Each chemical compound has a unique sweetness and bitterness. Cargill’s scientists started categorizing the taste profile of each individually and in combination to create a complex model of what works best together. “It wasn’t intuitive,” says Andy Ohmes, who runs the company’s high-intensity sweetener business. In 2014 Cargill rolled out ViaTech, a sweetening system based on this work that combines permutations of the nine glycosides approved for use. Several, including some of the best tasting, such as one known as Reb M, are present in less than 1% of the leaf. Cargill and PureCircle are trying to build them up through breeding practices.
Stevia has other deficiencies. Some product formulators say it’s harder to use than artificial sweeteners, which taste more like sugar and can therefore more easily be used as a substitute. PepsiCo CEO Indra Nooyi, for example, has stated that stevia doesn’t work well in colas.
Like other sweeteners, stevia enjoyed an early moment of exuberance. Companies, not recognizing that the then-known glycosides can replace only 70% to 80% of the sweetness in a typical soda, rolled out some truly terrible products. At a certain point, adding more stevia leads to diminishing returns and makes its flaws more prominent. Formulators have caught on, using stevia to help reduce a product’s calorie count rather than eliminate it completely. Even then, glycosides perform differently depending on the product. Reb A works well in tea, for example, but can clash with the taste of citrus. There’s no single solution for even one product category. “It’s a tinkering type of ingredient,” says John Martin, PureCircle’s head of technical development and innovation.
A whole series of companies are now experimenting with ways to bolster or modify sweeteners. MycoTechnology in Colorado, for example, uses the root system of mushrooms to block the off-taste of stevia. It can also reduce the need for sugar by masking the bitterness in products like wheat bread. Sensient in Milwaukee looks for properties in natural ingredients like tree roots or bark that can enhance the sweetness of sugar so formulators can use less of it. Chromocell, which works with Coca-Cola, is doing similar work, as is Senomyx, which collaborates with PepsiCo.
Most people in the business believe that a “systems approach”—a blending of ingredients rather than a single molecule—is the future of the natural-sweetener industry. “I don’t think you’re going to see just one thing,” says Mike Harrison, senior vice president of new product development at British ingredient giant Tate & Lyle.
For some people, tinkering or blending will never be enough to make stevia palatable. When I visit Tate & Lyle’s U.S. office outside Chicago, I sample a cucumber lime drink with stevia cooked up by a team there. I find it light and refreshing, though a bit sweet, with a few hints of bitterness at the end. But I glance over at my host, Harrison, and catch him grimacing. “I can’t drink this,” he says. “It makes me go bleh.” An hour later, it’s still bothering him. Harrison cracks open a can of root beer to try to drown out the sensation that lingers in his mouth.
It’s not the fault of his team. Harrison’s high degree of sensitivity is unusual, but in truth, nearly half the population has an aversion to stevia’s bitterness. That’s untenable for a seller hoping to reach a mass market. “If you’re making a food product and some 40% will out and out reject it,” Harrison says, “that’s a defect that you can’t really live with.” Another 20% of the population doesn’t detect any bitterness at all. It just depends on your genetic makeup. “Where that threshold is differs for everybody,” says John Smythe, Tate & Lyle’s head of sensory.
See more: Justin Timberlake Invests in Beverage Startup Bai Brands
Tate & Lyle has been working on new incarnations of stevia, mixing glycosides in different variations. Its second generation, due on the market next year, is not as bright as sugar; it’s “a little papery,” explains Smythe (who used to work for a wine company). Tate & Lyle’s next generation, currently in development, gets closer. But don’t expect to see it on the market anytime soon. Right now it’s simply too costly.
Still, even that pricey version has a sweetness that is slow to dissipate, suggesting that stevia’s problem simply may not be fixable. That’s partly because the sugar experience consists not only of intensity but also of more subtle aspects. Sugar’s sweetness spikes quickly, then fades almost as fast. Replicating it requires replicating that curve. “To have the exact same profile would be something closer to a miracle,” Smythe acknowledges.
PureCircle is addressing that issue by setting expectations about stevia. “People will say to us, as we come up with different innovations, ‘Well, it doesn’t taste like sugar.’ No, it tastes like stevia,” Son says. “We acknowledge that they taste different.”
Still, every time I eat something sweet that’s not sugar it tastes artificial to me. That’s just the way my brain and taste receptors are wired. “The way we think about it is, Tastes like sugar, good,” says Smythe. “Tastes like anything else, not good.”
In the decades after World War II, “scientific” food production was seen as the height of progress. Today, of course, consumers want their food to be “natural” and simple.
That has greatly complicated the search for a low-calorie sweetener. Shoppers view products with fewer ingredients as healthier than those with lengthy assortments. “The goal is not to add anything,” says consultant Alex Woo, who has worked at Kraft and PepsiCo. If ingredients must be inserted, he says, the fallback position is “to claim no artificial flavors or sweeteners.”
It’s the crux of the current dilemma. How does one engineer a natural solution? Cargill, for example, is trying to produce some of the rarest and best-tasting steviol glycosides through a fermentation process. The product, called EverSweet, has been delayed because of production costs, but there’s also the looming issue of whether consumers will accept the product as natural if it doesn’t come from a leaf.
The same can be said of erythritol, a zero-calorie sugar alcohol that’s found in fruit, which ingredient companies can also produce by fermenting yeast. Erythritol is often coupled with stevia, especially in tabletop sweeteners like Cargill’s Truvia, to replicate the bulk and weight of sugar. “Our biggest challenge is that the consumer doesn’t understand how wonderful erythritol is,” says A.J. Aumock, Cargill’s global marketing leader for Truvia. Some are wary of its chemical-sounding name and its classification as a sugar alcohol, some of which are associated with what’s known in polite circles as “gastrointestinal distress.”
Even stevia is not exempt from the question of naturalness. Sure, it comes from a plant, but it’s then processed into an extract. “There’s a large debate about whether those products are natural,” says Bernstein analyst Dibadj. “Some would argue that sugar is more natural than stevia.” (“Natural” sugar, too, is made by applying chemicals to pure cane or beets.) The FDA does not define the term “natural,” so in the end it’s left to the opinions of the buyers and sellers.
Some consumers clearly prefer what they perceive as pure substances—one reason that cane sugar has been making a minor comeback. They would rather accept the perils of too much sugar than the perceived risks of an artificial alternative.
“Whatever tastes sweet, we’re going to grab and send in for analysis,” says an executive of the search for a new sweetener.
It’s irrational to assume that natural sugar substitutes are better than artificial ones, argues Paul Breslin, a professor who specializes in taste perception in the department of nutritional sciences at Rutgers University. Calorie-free natural sweeteners, for example, have not been cleared of all of the risks some researchers attribute to artificial ones—messing with gut bacteria, causing metabolic dysfunction like glucose intolerance, and leading people to overeat.
“We just don’t know one way or the other,” says Dana Small, the deputy director of Yale’s John B. Pierce Laboratory, which studies physiology and health. “We know enough to know that we don’t know enough.” For example, what does it mean that sweetness used to always indicate the presence of calories and now it frequently does not? And what are sweetness receptors doing not only in the mouth but along the intestinal tract? Even today, scientists don’t fully understand what it is we’re looking to replace.
When Grant DuBois joined Coca-Cola (KO) in 1992, the Type 2 diabetes crisis was looming large. “Quickly my focus became finding alternative sweetener systems,” says DuBois, an organic chemist who would go on to become the company’s director of ingredient and product sciences. It was his job to find a way to make Diet Coke taste like the real thing. By the end of the decade, the additional caveat was that it had to be natural. “They had already reached the conclusion that they had to get artificial flavors and sweeteners out of their products,” he says.
DuBois and his team started combing through the plant-based materials that had potential. They examined more than 50 possibilities. All had their flaws—some of them pretty major. Monatin, for example, comes from a South African plant and is 3,000 times as sweet as sugar, but it produced a nasty fecal odor when exposed to light. “It was horrible,” DuBois says. Finding that perfect alternative is “a fool’s errand,” he contends. “There’s a lower probability in my view than prospecting for gold. The probability is essentially zero.”
Even if DuBois and his team had found something that tasted right, it would also have to be cost-effective. It’s hard to beat the price of artificial sweeteners. “They’re dirt cheap,” he explains—even cheaper than sugar. When DuBois left Coca-Cola in 2011, he says, it cost about 60¢ to sweeten a case of 24 eight-ounce bottles with sugar, 50¢ with high-fructose corn syrup, and just 5¢ with aspartame. Companies “love selling it because of the margins they make,” he says. By contrast, DuBois says, natural sweeteners are “very expensive and always will be.” That would cut significantly into profit margins or lead to higher soda prices for consumers.
Natur Research Ingredients has developed a product called Cweet made from the brazzein protein, which is 2,000 times as sweet as sugar and comes from the African oubli plant. Miraculex in Davis, Calif., is working with a sweetener protein found in a berry called the miracle fruit that binds to taste buds to make sour things taste sweet.
Tate & Lyle already has the rare sugar allulose on the market. The body does not retain it, so it can’t cause weight gain, and it has a pure and bright taste with 70% the sweetness of sugar. But it suffers a huge marketing impediment: Since allulose is chemically a sugar, it would have to be identified as such on the nutritional facts label even though it has almost no calories—a confusing proposition for consumers. Tate & Lyle is currently petitioning the FDA to grant an exception.
One plant-based alternative that has gained some traction is an extract from monk fruit, or luo han guo, which is part of the melon family and is grown in China. Its fruity taste is often coupled with stevia to soften its bitterness, but it’s rarely used as the sole sweetener in part because it costs about five times as much as sugar. Elaine Yu, president of the U.S. subsidiary for monk fruit and stevia producer Guilin Layn Natural Ingredients, says the company is trying to lower its cost. Guilin Layn is attempting to double the output of its active ingredient, Mogroside V, through breeding practices, as well as getting two harvests a year out of its crop rather than just one.
Some scientists believe fixing sugar is a better idea than replacing it. For example, a startup called DouxMatok, based in Israel, is taking a lesson from the pharmaceutical industry’s research in targeted drug delivery. If a chemotherapy drug is encapsulated so it releases only when it hits the tumor, it will cause less damage on the way to the target. DouxMatok is trying a similar approach. It coats minerals with sugar, which keeps most of it from being released until it hits the sweetness receptor. Since very little material is lost on its journey, the company says it can achieve the same level of sweetness by using up to 50% less sugar.
Yet even DouxMatok’s product, which is more than 99.5% sugar, did not always behave exactly like sugar. Though it tasted the same, the company had to come up with different versions that would, say, react properly in a fatty solution or if cooked at high temperatures.
Nestlé, meanwhile, is trying to change sugar’s structure, essentially by hollowing it out. Stefan Catsicas, the company’s head of innovation, describes a sugar crystal as being like a box. We taste only the outside of it in our mouths, but we swallow the contents of the entire thing even though the sugar on the inside is not essential to the sense in our mouth. “We can structure it so whatever we put on the tongue will be perceived and will represent most of what we swallow,” Catsicas says. This could potentially cut down on sugar by up to 40%. The tradeoff—and you should know by now there’s always a tradeoff—is that the structure is destroyed in water, which is present in most foods. Luckily for Nestlé, chocolate is one of the only foods that are not aqueous.
There seems to be an obvious solution to all of this that would be much easier for everyone: Why not just eat less sugar? “As we move away from sugar, we are facing this dilemma that nothing tastes like sugar,” says consultant Woo. We know, after all, that our expectations are not set by nature. In the U.S. products tend to be sweeter than in Europe. For example, a liter-size bottle of American Dr Pepper has 108 grams of sugar, vs. about 73 grams for the U.K. equivalent. Why not just drop the threshold in the U.S. market too?
Several of the big food and beverage manufacturers have pursued this path, vowing to cut sugar in their products. Coca-Cola says it has already reduced it in more than 200 of its sodas. For its part, PepsiCo has committed that by 2025, at least two-thirds of its volume will have 100 calories or fewer per 12 ounces. (A can of Pepsi has 150 calories, for example.) General Mills has begun slashing sugar in its cereals and yogurt. Nestlé and Dr Pepper Snapple have made pledges of their own.
The challenge stems in large part from what the rest of the market is doing. “They’re afraid that consumers will taste 20% lower sweetness and go to a competitor,” says DuBois. Paul Bakus, Nestlé’s president of corporate affairs, told me that the company has to walk a narrow line between being nutritionally superior to the rest of the market and not sacrificing taste. “We want to reduce sugar where possible as long as we don’t put ourselves at a competitive disadvantage,” he says. “How do you compete if your competitors aren’t following the process or rules or guidelines?”
Some experts believe the industry should follow the template of the U.K.’s sodium reduction program. In 2005 the country’s food industry pledged to reduce sodium in key categories by as much as 50% over eight years. By 2011, national sodium consumption had dropped 15%, and deaths from stroke and heart disease fell about 40%. The U.K. is now considering a similar coordinated attack on sugar.
A version of this article appears in the March 1, 2017 issue of Fortune with the headline “Sugar Rush.”
