Donald R. Keough, the former president and COO of Coca-Cola who died Tuesday at age 88, was arguably the most powerful non-CEO in recent memory. Guiding Coke’s global expansion and steering the company through management upheaval, Keough outlived a parade of Coke chiefs and probably mattered more than any of them.

Warren Buffett recalls the moment 30 years ago when Coca-Cola (KO) made one of the worst blunders in marketing history—by replacing flagship Coke with a new formula, calling it New Coke, and resisting a firestorm of consumer anger. Roberto Goizueta was CEO back then, and Keough was his president. “Roberto just didn’t want to admit that they were wrong,” recalls Buffett about management’s standoff vs. the public derision.

This was a few years before Buffett started buying stock in Coca-Cola. (Berkshire Hathaway now owns over 400 million shares, or 9.13% of Coke, worth more than $16.8 billion.) But Buffett was a keen observer, and it impressed him that Keough led his own boss, CEO Goizueta, to reverse course, return original Coke to the market, and tell consumers that they were the real boss of the brand.

Thanks to Keough, a “crisis” led to an upturn in sales. “It wouldn’t have happened without Don,” says Buffett, who joined the Coke board and brought Keough onto his, at Berkshire Hathaway. “Don had as much feeling about what makes a great brand as anyone you’ll ever meet.”

Assigned to cover Coke after I arrived at Fortune in 1984, I knew Keough for 30 years and wrote about him and the company frequently. He was everything Buffett says, and more. A charismatic salesman who landed at the world’s largest beverage company in 1960, Keough was the gregarious Mr. Outside who trotted the globe and courted bottlers and customers while Goizueta, a patrician chemist from Cuba, preferred to work his strategies and the numbers back home in Atlanta. “There were two No. 1’s,” says Coke director and Allen & Co. CEO Herbert Allen about the Goizueta-Keough team. “Roberto had the official CEO title, but they were partners. I doubt there was any team ever better.”

During their time at the top of Coca-Cola, Goizueta and Keough aggressively expanded the product lineup (introducing diet Coke in 1982), stormed new territories (beating Pepsi to eastern Europe after the Berlin Wall fell in 1989), bought and sold movie assets (Columbia Pictures), and multiplied Coca-Cola’s shareholder value. When Keough retired in 1993 after 12 years as president, the company’s market value had increased from $4 billion to $56 billion in a decade. Coke was America’s sixth most valuable company as well the world’s best brand .

But in 1997, Goizueta’s death from cancer ushered in a painful decade in which leadership failed. Keough had no official power but was deeply involved and highly influential as he orchestrated from the sidelines. “His constant involvement in Coke’s affairs has too often amounted to meddling that appears vindictive and even vengeful,” Fortune wrote in 2004, just as Keough, who had retired from Coke’s board, had returned to it. Whatever his faults, he provided more help than harm in the grand scheme of Coke’s troubles. He assisted in the ouster of two poor-performing CEOs, Doug Ivester and Doug Daft.

Then Keough played a critical role in recruiting CEO Neville Isdell, a Coke veteran who stabilized the company, and he endorsed and advised Muhtar Kent, Isdell’s successor. Kent became CEO in 2008 and is still at the helm. In a letter to employees on Tuesday, Kent lauded Keough for being “constructively discontent” and for wisely worrying about success. “People, companies and countries can get into trouble when they start to think they’re successful. They get arrogant,” Keough once said.

A source close to Keough’s family says that he was diagnosed with throat cancer five years ago, underwent treatment, and was told by doctors that he was cancer-free. But he contracted pneumonia a few weeks ago, was hospitalized and succumbed to that illness on Tuesday morning, with his family members bedside. Keough and his wife, Mickie, had six children.

The man in full was a global builder, a bodacious marketer and a savvy behind-the-scenes operator. But Don Keough’s greatest talent was counseling leaders who were, at least on paper, more powerful than he. He served on the boards of IAC/InterActive (IACI), McDonald’s (MCD), the Washington Post Company (WPO), H.J. Heinz and Home Depot (HD), as well as Coke and Berkshire Hathaway (BRKA). Moreover, for 22 years after he retired from Coca-Cola, Keough served as non-executive chairman of Allen & Co. Herbert Allen can’t pinpoint what Keough brought to his firm, but it was important, he says. “He brought a spirit and intelligence that’s hard to define. He was, above all, a teacher.”

Buffett told me yesterday that each year after Berkshire Hathaway’s fall board meetings, where CEOs of Berkshire-owned businesses present their businesses to the directors, Keough used to send hand-written letters to each of the executives. “Not business letters—insightful and intimate personal letters, with embedded advice. But you didn’t necessarily know he was giving you advice.”

“They were gems,” says Buffett. The world’s greatest investor and anyone else who knew Don Keough well learned from him: that great relationships constitute the real secret formula of enduring business success.

Here’s Keough in 1985, bringing back original Coca-Cola after new Coke failed.