In an airy life sciences lab on the fourth floor of Danone’s gleaming new research facility in Utrecht, the Netherlands, amid an array of sleek technology, one machine seems slightly out of place. A marvel of plastic tubing and colorful valves, it looks like an amateur tinkerer’s contraption, or the leftovers from a game of Mousetrap. Even its name sounds a little slapdash: Throughout the lab, it’s known as TIM.

But modest as it appears, TIM is a highly sophisticated piece of medical technology: an artificial gastrointestinal tract. (TIM stands for “TNO intestinal model,” after the Dutch research institute that helped design it.) The machine mimics the extraordinarily complex, dynamic process of human digestion. After foods or ingredients are sent down its chute, they snake through a biochemical obstacle course—confronting the punishing acids of the stomach and the erosive enzymes of the small intestine, monitored all along their daylong trek by watchful researchers.

TIM gives Danone’s scientists a glimpse of how its products will fare inside humans—and how humans might fare with the products inside them. It’s a crucial element of Danone’s quest to understand the “microbiome,” the teeming mass of 100 trillion or so microorganisms that live in, and on, each of us—most of them in the gut. It’s there, scientists have recently come to understand, that the average person’s 4.4 pounds of bacterial partners do their vital work: manufacturing vitamins and essential acids, building and stimulating the immune system, and regulating digestion. Stanford microbiologist Justin Sonnenburg goes so far as to call the microbiome “a control center for human biology.”

Damage to this control center can have serious consequences. Researchers have found links between imbalances in the gut microbiota and an America’s Most Wanted list of costly health problems, from obesity and diabetes to autoimmune diseases, autism, and depression. They’ve also found evidence that First World eaters may have brought some of these problems on themselves—by rooting out too many “good” microbes through overzealous use of antibiotics, processed foods, and antibacterial sanitizers.

Nutricia Laboratory
Scientist Jan Knol checks on TIM, the artificial GI tract that helps Danone test probiotics.Photograph by Michel Porro for Fortune Magazine

But that same body of research offers the tantalizing possibility that science could restore balance to the microbiome by reengineering what we eat. Once dismissed as inconsequential, this microzoology is now seen as a key to the development of health-promoting superfoods. Researchers have a huge pool of new data that could help them isolate the little-understood effects of “probiotics” (beneficial microorganisms that we ingest) and “prebiotics” (indigestible foods, generally fibers, that nourish the good bugs already inside us)—additives once regarded as mere “digestive aids.”

These are the kinds of compounds that TIM digests all day, and the hope is that more of them will reach our supermarkets and kitchens, in products better targeted to help our bodies and even our minds. Fantasy foods like juices that can ward off autoimmune diseases or cereals that can encourage fat reduction now seem like attainable goals to some scientists. Other researchers think the microbiota could launch an era of personalized nutrition, in which a thorough analysis of your unique gut ecosystem could guide the way to a customized diet, optimized for your health.

The plummeting cost of microbial genome sequencing has triggered an avalanche of research that’s prompting optimism about potential benefits—more than 6,000 studies of the microbiome have been published since 2006. The attendant publicity hasn’t gone unnoticed by consumers: In a recent survey by market-research firm NPD Group, 31% of Americans said they wanted to ingest more probiotics. Seeing an opportunity, foodmakers, from corporate giants like Nestlé and General Mills to fledgling startups, are pouring money into research. “Drug companies, food companies, academics, federal scientists, food scientists, neuroscientists, institutes,” says George Fahey, professor emeritus of nutritional sciences at the University of Illinois. “Everybody wants a piece of it.”

In this crowded race, Danone, the world’s leading dairy producer, has a pronounced headstart. Yogurt, one of the best-known gut-friendly foods, is Danone’s bestselling product globally (including under the Dannon brand in the U.S.). Danone was one of the first food companies to fund microbiota research; it’s currently staging some 100 clinical trials and collaborating with more than 40 academic or commercial partners in the field. Gérard Denariaz, director of strategic R&D partnerships at Danone, describing the company’s scatter-the-seeds philosophy, says: “Who knows who will come up with the next breakthrough?”

If companies do crack the microbiome, the rewards could be huge: Global sales of “fortified/functional foods” reached $275 billion in 2014, according to Euromonitor, and foods that could tout themselves with scientific precision as suppressing obesity or fostering childhood development could be enormously lucrative. But there’s much ground to be covered before today’s research leads to a breakthrough product. During a recent tour of Danone’s operations in Europe, Fortune got to see the promise and the limitations of this food-science frontier where ancient organisms meet high technology.

Danone’s big bet on microbiota was catalyzed in 2006 by a phone call to Denariaz from an old acquaintance—Dusko Ehrlich, then research director of France’s National Institute of Agricultural Research. When the two met, a breathless Ehrlich explained that he wanted to study the gut. He spoke of it wondrously, as if it contained the untapped secrets of an uncharted galaxy. Microbes were the stars in this universe, and like stars, they were almost infinite. And Ehrlich was increasingly certain that this multitude’s interactions with the body had profound consequences for human health.

Nutricia Laboratory
Bacterial strains in cold storage at a Danone lab.Photograph by Michel Porro for Fortune Magazine

Ehrlich sought to recruit Danone to join him in a multiyear investigation to gather detailed data on this microbial community—a full accounting of the bugs’ genetic material. It would involve a crazy amount of work, not to mention intensive analysis of the most accessible microcosm of the gut available to researchers—human feces.

Denariaz, a lanky veteran of 25 years in Danone’s labs, was so skeptical as to be almost speechless, reduced to a single French “Pfft.” Although Danone had worked with probiotics for generations, this project seemed like a stretch for the company, and most of the scientific community saw gut bacteria as a biological afterthought.

Still, after the meeting, he began reading up on Ehrlich’s ideas. One scientist’s work in particular grabbed his attention. In 2006, Jeffrey Gordon, a biologist at Washington University in St. Louis, published his findings from an experiment he had conducted using “germ-free” mice—that is, rodents born and raised without a microbiome. He had transplanted into those mice microbiota from other mouse subjects, some of which were lean and others obese. The mice were given identical diets, but within two weeks, those that had the microbes of obese subjects had gained significantly more body fat than the mice that had received the lean microbes. The microbiota appeared to have the power to remake the bodies of its hosts, for worse or for better.

Denariaz was sold. He called up Ehrlich. “We’re coming with you,” he told his friend.

His original skepticism notwithstanding, the decision seems natural in hindsight: After all, Danone has always been in the bacteria business. Founded in 1919 by Isaac Carasso, a science-minded entrepreneur in Spain, Danone at first sold its yogurts in pharmacies, inspired by research connecting healthful microbes in yogurt with longevity in the Balkans. Since then yogurt has become a mainstream product, and Danone, the world’s largest purveyor of it, has become one of the world’s largest food conglomerates. Now based in Paris, the company has $28 billion in annual sales, and none of its products sells better than Activia, the probiotic-enhanced yogurt introduced in 1987. In 2014 global sales for Activia were $2.7 billion; according to Nielsen, it’s the top-selling dairy brand in the world. Together, Ehrlich’s pitch and Gordon’s research hinted at an enticing prospect for Danone: Breakthroughs in this new field could spawn Activia-like successes across the spectrum of foods Danone sold.

The company first backed Ehrlich in 2008 by committing a handful of company scientists and a sizable financial contribution to the $30 million cost of MetaHIT (Metagenomics of the Human Intestinal Tract), a new European microbial-genome sequencing project. (A similar project began in the U.S. at about the same time.) Danone was the only food company involved with the European project. As a result, the firm’s scientists got an exclusive look at critical advances and fast-emerging technologies in the field. That exposure, in turn, paved the way to deeper commitments: Danone has declared the microbiome one of four “pillars” of its research, devoting a significant share of its $313 million R&D budget to the cause. Since its first MetaHIT investment, Danone has published some 200 papers in the field and picked up a handful of related patents.

Though the finer details of Danone’s microbiome findings are a closely guarded secret, Denariaz can’t quite hide his delight over being an early adopter—he estimates Danone has at least a two-year lead over its competitors in microbiome science. And that lead builds on the company’s long legacy: Danone owns one of the food industry’s largest libraries of bacteria strains, a collection of 4,000 bugs the company has culled from around the world over the decades. Stored in tiny vials and chilled at –80° Celsius in a subzero freezer in one of Danone’s microbiology labs in Paris, the collection is a treasure chest for a superfood future.

The early years of this research push coincided with some findings that helped convince Danone it was on the right track. Wendy Garrett, an immunologist at Harvard who studies the role of the gut microbiome, had been looking into the effects of probiotics on gut inflammation, a condition common in people who suffer from bowel diseases like Crohn’s and ulcerative colitis. In 2007 she found that a fermented milk product containing five bacteria—similar to Activia—reduced the inflammation. Since then her lab, in collaboration with Danone, has continued work on the bacteria and patented some discoveries.

As its broader database of research grew, Danone used it to encourage other outside studies. Emeran Mayer, a gastroenterologist and neuroscientist at UCLA, had long dismissed theories about a brain-microbiome connection as pseudoscience, taking offense at the booths that touted yogurt at conferences he attended. “I’d think, ‘This is ridiculous. We’re doing real science here,’ ” he recalls. So he was resistant when Danone approached him in 2008 about studying the effect of a probiotic on the gut-brain axis. But after seeing intriguing findings in some related rodent studies, he relented. He designed an independent trial, studying the reaction times of a small group of young, healthy women who consumed either probiotic yogurt or a placebo for four weeks. Mayer found a statistically significant difference in the populations: Those who ate the probiotic showed an altered brain response on a functional MRI, consistent with reduced emotional reactivity. While the immediate implications of that finding were unclear, Mayer is now planning a more ambitious experiment to test the effects of manipulating the microbiome in people who suffer from anxiety.

Of course, these encouraging hints fall well short of a breakthrough on a new, health-promoting food. Cataloguing bacteria and prebiotic fibers has proved to be relatively easy; figuring out how they work and converting them into a useful form, less so. Many probiotics reach the gut dead on arrival—they don’t survive shelf life or the harrowing trip down the chemical rapids of the digestive tract. Even when a dose of ingested bacteria does reach the colon, its potency in an ecosystem of 100 trillion is likely to be limited unless repeated frequently, says Dave Mills, a food microbiologist at the University of California at Davis: “It’s like pouring a glass of Kool-Aid in your pool and expecting it to turn red.”

A closeup view of bacteria found in the company’s yogurt.Danone Strain Collection—inra mima2/t meylheuc

Gut bacteria are a particularly confounding subject because each person’s microbiome is a uniquely complex and dynamic ecosystem. The same food may affect two people differently; it could even affect the same person differently on different days, depending on sleep patterns, stress levels, medication use, and countless other factors. Rob Knight, director of the University of California at San Diego’s Microbiome Initiative and one of the field’s preeminent scientists, has monitored his own microbiome every day for six years; he says he still can’t predict or grasp exactly how, why, or when it changes.

That complexity has prompted the industry to focus on a simpler target: the baby gut. Infants pick up their first microbes in the birth canal, and in early life mother’s milk is like a microbiome starter kit—rich with the complex fibers that feed those microbes. Research suggests that these early-acquired bugs are critical for healthy development: Babies born preterm or via cesarean section, and who are in general exposed to antibiotics early on, have been found to be at higher risk for health problems like asthma, allergies, and obesity later in life. A product that could reduce that risk would be a powerful force in the $50 billion global formula market.

Jan Knol, a professor at Wageningen University in the Netherlands who has spent 18 years as an intestinal early-life microbiologist for Danone, remembers a time when he and his colleagues painstakingly analyzed the effects of baby formula by hand, counting bifidobacteria—a highly sought-after bacteria strain, lit by fluorescent tags—in babies’ stools. Today he has more baby-stool data than even the best bioinformatics could make sense of, thanks to cheaper genome sequencing. And his team has used it to create one of the first new healthy-gut products to reach the public. Last year Danone introduced a formula laden with synbiotics—a mix of prebiotic fiber and probiotic bacteria. Launched in Thailand, Australia, and New Zealand, it will soon roll out to other parts of the world.

Still, this doesn’t quite qualify as a victory. Before it can make any bold health claims about the formula’s impact, Danone needs to wait—at least six years to compare asthma rates among users and nonusers, and even longer to study rates of obesity. For rock-solid proof, the company ideally would observe a much larger and more widespread trial population, but Knol suggests that for now it is out of Danone’s reach—it would be a pharma-level financial commitment for a company with more modest, food industry profit margins.

Knol’s observation reflects a reality that makes food executives queasy—the chance that big food won’t be the industry that profits first, or most, from microbiome research. A number of pharmaceutical companies, including Johnson & Johnson JNJ , Pfizer PFE , and Sanofi, have research initiatives focusing on the gut microbiome. The tech sector could also capitalize on the trend: Just about everyone in microbiome research describes a not-far-off future when some kind of diagnostic tool (most have their bets on smart toilets) will analyze your gut bacteria and steer you to the foods you need most—a point of connection between the Internet of things and a new normal of personalized nutrition.

It’s also possible that the first big microbiome breakthroughs won’t even involve the human body. Companies like DuPont DD and Monsanto MON are feverishly studying the microbiomes of soil and plants in hopes of increasing agricultural yields; others are translating the science to pet foods and livestock feed.

For now, it’s hard to tell what products or producers may be close to market. The innovators—Danone included—tend to keep a lid on what they know, for competitive reasons. Still, there are hints to be found among news clippings and patent applications. General Mills, which has an institute engaged in microbiome research, has filed a patent for a fiber mix that ameliorates inflammatory bowel disease. (General Mills GIS declined to comment on pending patents.) Danone has filed for a patent for bacteria that reduced body-fat accumulation. But while such a probiotic may sound like a winning lottery ticket, the effect Danone patented is likely far from human-ready—it was observed in nematodes, better known as roundworms.

With the race to the grocery aisle so secrecy-shrouded and uncertain, Danone is hedging its bets. On the tech side, the company is a co-investor in a venture capital fund that invests in microbiome startups—one such firm is building a diagnostic tool with which Danone sees synergies. Danone has also commissioned a French research institute to build an artificial intestine that could eventually complement TIM, the artificial GI tract, in tracing and predicting biome reactions.

Denariaz remains confident that, no matter which company makes the next major advances, Danone will be in a position to benefit—all the more so if its own research bears fruit (or cereals, or any other good-for-your-gut foods). Weighing the potential rewards, the Frenchman shakes his head, laughs, and reaches for an American catchphrase: “I guess it’s ‘No guts, no glory.’ ” 

This story is from the June 1, 2015 issue of Fortune magazine.