Indeed, the past 18 months have probably been the most challenging of Chenault’s career. Since the start of 2015, when it was trading near an all-time high at about $95 a share—and a market capitalization of nearly $100 billion—American Express stock has fallen as low as $51, amid a slew of downgrades by Wall Street analysts. The stock has recovered a bit, recently trading at about $65 a share. But AmEx is still the third-worst-performing stock in the Dow Jones industrial average over the past year, even as rivals Visa (V) and MasterCard (MA) have easily outperformed the broader market. And American Express’s market value remains almost $40 billion below its high.

The selloff has tracked a series of setbacks for Chenault’s once-unassailable business. First, in February 2015, the card giant lost its exclusive, highly profitable co-branding deal with mega-retailer Costco (COST) after a 16-year partnership. That same month AmEx also lost an antitrust lawsuit with the Justice Department regarding whether its lock on merchants that offer customers the opportunity to pay for purchases with an American Express card is anticompetitive. For years Chenault had refused to settle the case. As a result, even though the ruling has been stayed pending an appeal, the once seemingly sacrosanct transaction fees that American Express charges merchants—about $2.50 per transaction—have begun to be whittled down closer to the $2-per-transaction fee charged by Visa and MasterCard.

Then things turned from challenging to tragic. In May 2015 came the shocking news that Edward Gilligan, AmEx’s 55-year-old president and Chenault’s heir apparent, had died suddenly, apparently from a blood clot, on a corporate jet that was ferrying him home from Tokyo. It was a devastating loss for Chenault both personally and professionally. Gilligan had already largely taken on oversight of the company’s operations.

Gilligan (on right, with Uber CEO Travis Kalanick) was Chenault’s heir apparent before he died suddenly last year.Photograph by Scott Eells—Bloomberg/Getty Images

Chenault and his executive team didn’t have much time to grieve. Just a few months later, in August, Jeff Ubben, an activist hedge fund manager at the $11 billion ValueAct Capital, acquired a stake in the company and began agitating behind the scenes for change. Suddenly a question began to form on Wall Street: Had American Express lost its way?

For Chenault, 65, this wasn’t how his storied career was supposed to wind down. A graduate of Bowdoin College and Harvard Law School, he joined American Express in 1981. Chenault has served as chairman and CEO since 2001, making him easily the longest-serving top executive at a major U.S. financial services company. (He’s also one of just five African-American CEOs in this year’s Fortune 500.) The plan was for Chenault to spend his final years in the corner office on an extended victory lap until passing the baton officially to Gilligan sometime before the decade was out. Instead he has been forced back into action, retaking the reins of the company on a day-to-day basis at the very moment when the competitive pressures it’s facing have never been greater. (Chenault, who rarely agrees to speak with print journalists, declined numerous requests to be interviewed for this story.)

Not only must American Express confront threats from its huge, well-capitalized credit card rivals, but it’s also starting to feel the burn from the nimbler “fintech” startups that are increasingly targeting particular corners of the credit card oligopoly’s business. AmEx used to be a well-oiled machine that produced high profits without much competition—and was rewarded by Wall Street with a higher earnings multiple than its peer group. But since the onset of the 2008 financial crisis, it has found itself in a highly competitive, highly regulated, ever-changing business in which profit margins are being squeezed at every turn despite the considerable belt-tightening that the company has undergone.

Chenault now has the dual challenge of charting a more growth-oriented path for AmEx while grooming a new successor—that is, if the board lets him stick around to complete the mission. The recent turbulence has spawned rumors that the directors might hasten Chenault’s retirement. But one source close to the board bats down this assertion. “The board has complete confidence in Ken,” he says.

The most important endorsement of Chenault has come from someone who is not on the American Express board but happens to be the company’s largest shareholder: Berkshire Hathaway (BRK-A) CEO Warren Buffett. In Berkshire’s 2016 annual report, Buffett noted that because of American Express’s ongoing stock-repurchase program, the percentage of his stake in the company had increased to 15.6% in 2015, from 14.8%. He wrote that like the other “Big Four” long-term investments in the Berkshire portfolio (Coca-Cola, IBM, and Wells Fargo complete the list), American Express has an “excellent business” run by a manager who is both “talented and shareholder-oriented.” (Buffett declined a request to be interviewed about Chenault.) Of course, Buffett can afford to be charitable, considering he purchased his 151.6 million shares in American Express for $1.287 billion and they are now worth about $10 billion.

Warren Buffett, whose Berkshire Hathaway now owns 15.6% of American Express, has expressed support for Chenault as an executive who is “talented and shareholder-oriented”.Photograph by Daniel Acker— Bloomberg/Getty Images

Certainly the Buffett boost is good news for Chenault in the short term. With the Berkshire CEO’s support, Chenault, who made $22 million in 2015, could theoretically stay in his post forever—AmEx has no mandatory retirement age for executives. But Chenault still must figure out what American Express’s role will be in an increasingly digital economy at the very moment when Wall Street has never been more skeptical that he still has the energy for the struggle. More and more people are wondering: Does he?

An early answer to that question might be found in Chenault’s decision to participate in AmEx’s January 2016 earnings call with Wall Street analysts—something he hadn’t bothered to do for years. Much of the concern swirling around American Express and Chenault in the investor community came to a head during the Jan. 21 call, the purpose of which was to review the company’s performance in the fourth quarter of 2015.

No one was expecting an upbeat report. The last time Chenault had participated in an earnings call, in January 2013, the company had announced that it was restructuring its workforce and dismissing 5,400 of its 63,500 employees. (The company now has about 53,500 employees worldwide.) The message had gone out to Chenault from some of the company’s big investors that if American Express once again had bad news to deliver, he was the one who had to deliver it. Mission accomplished.

Right away Chenault acknowledged that the company had not been performing up to expectations. He said that co-branding opportunities had become less profitable, and that there was “competitive pressure” on merchant fees and “intense competition” for customers. He blamed low gas prices and a strong dollar. He said that the company faced “many challenges,” more than in recent memory, but that he was addressing them with “a strong sense of urgency” and that progress was being made. Chenault also announced that American Express had embarked on a new $1 billion cost-reduction program, which included laying off 4,000 people.

The CEO said that earnings expectations for 2016 and 2017 would be reduced below previous guidance. Craig Maurer, an analyst at Autonomous Research, tried to get a handle on the company’s fading star. “How can you maintain the model long term that has traditionally allowed AmEx to be valued at a premium to competitors?” he wondered.

Without missing a beat, Chenault ticked through the reasons American Express remained worthy of its special valuation. There was still viability to the company’s “spend-centric” model of enticing customers to use their AmEx cards more, and thus gaining the leverage over vendors—vital because much of AmEx’s revenues come from fees paid by those vendors. There was a huge opportunity for American Express among small businesses in the U.S., which, he said, spend about $4.8 trillion annually, but only 10% of that amount is paid with a credit card. Although he did concede to Maurer that the economics of co-­branding were changing in an unfavorable way, he didn’t seem overly concerned about that or much of anything else. While Chenault answered nearly every question asked by the analyst community, he seemed utterly indifferent to its concerns. “I think there is a sea change going on in payments and commerce,” he said. “I think we have the ability to compete. You’ve got to look at the breadth of the portfolio and the number of the levers we have to pull.”

Investors weren’t impressed with Chenault’s performance. The next day the stock dropped 13%.

Despite all the tumult, American Express, No. 85 on this year’s 500, with $34.4 billion in revenues, is hardly a failing business. In fact, it still puts up extraordinary numbers for a large company, with Ebitda (earnings before interest, taxes, and other factors) margins in excess of 30%. Wall Street banks would kill for the 25% return on equity that AmEx had in 2015, given that it’s been years since any big Wall Street firm has achieved more than a 10% ROE. (In the first quarter, Morgan Stanley’s ROE was 6.2%; Goldman Sachs’s was 6.4%.)

The big concern is that the primary factor that once differentiated American Express from its competitors—a stranglehold on the wealthiest consumers, who paid off their balances every month—is no longer the case. For years the company was “on auto­pilot,” says Ben Chittenden, an analyst at Oppenheimer & Co.  “It had nice growth. You were kind of playing in your own sandbox.” AmEx has long plied its high-end ­customers with perks and lower interest rates in exchange for annual fees from those customers and merchants, while the likes of Visa and MasterCard made money off the interest owed on credit balances by its less elite customer base.

Now the business model of AmEx and its competitors is merging. In the wake of the financial crisis, regulators have blocked big banks—many of which are also big credit card issuers—from other business lines that once generated large profits. Their credit card businesses are a way to recoup those lost profits. Visa and MasterCard are going after American Express’s customers. And AmEx, for its part, is increasingly targeting customers who float balances.

JPMorgan Chase (JPM), for one, has taken the fight directly to American Express. For instance, for an annual fee of $595, it now offers a Palladium card, which is made of both palladium and gold supposedly worth a total of $1,000. The Palladium offers the kind of premium hotel benefits and airport club access that AmEx’s exclusive cards have always promised. Chase has also stepped up its rewards program and replicated parts of American Express’s technological infrastructure by agreeing to lease VisaNet, Visa’s transaction-processing network, for 10 years.

Competition is also newly rife in the market for co-branded credit cards—banks and merchants team up to provide retailer-­specific cards that offer rewards when used at the retailer but that can also be used elsewhere. A prime example of that increased competition is the dissolution of AmEx’s deal with Costco. American Express had been the sole card provider in Costco’s nearly 700 stores for 16 years. In 2015, Costco provided American Express with 7 million card members (about 10% of its membership), $76 billion in purchasing volume, and $12 billion in consumer loans.

Starting June 20, Costco will be mailing its new Costco Anywhere Citigroup Visa cards to its customers for immediate use. American Express sold its loan portfolio to Citigroup (C) as part of the deal, netting for AmEx a $1 billion gain, but the annual revenues (and profits) generated by Costco will be hard to replace. Starwood’s recent sale to Marriott (MAR) also raises questions about the future of the Starwood-AmEx co-branded card.

The same day as the January earnings call, Ubben of ValueAct made news by announcing that he had decided to sell his entire stake in American Express and had abandoned his efforts to force change. (He says he got out whole.) Ubben had started buying up American Express stock after the Costco news. In August 2015 he announced that ValueAct had taken a $1 billion stake in AmEx at an average purchase price of about $75.

Unlike other activist investors such as Bill Ackman, Carl Icahn, Starboard Value’s Jeff Smith, and Dan Loeb of Third Point, Ubben does not hit CEOs over the head in public, demanding changes. But working below the radar does not mean working ineffectively; many believe Ubben is the reason Steve Ballmer is no longer the CEO of Microsoft.

Ubben began working behind the scenes with Chenault to persuade him to con­sider ways to unlock more of the value that he believed lay buried inside American Express. They discussed ways to cut costs and increase “data mining,” or using information about customers’ spending habits to offer them real-time discounts, something Ubben believes American Express could do better. (Ubben declined to comment for this story.)

In 1998, president Chenault posed with Amex CEO Harvey Golub; Chenault took over the top job in 2001.Photograph by Jonathan Levine

Among Ubben’s aims was to have American Express get out of the revolving-loan business and sell off its loan portfolio. Or perhaps to buy a company like PayPal, which is a big player in online transactions, unlike American Express. He also wanted more focus on reducing costs. “To restructure the company, that takes tremendous energy,” says a person familiar with Ubben’s thinking. “It takes a really aggressive manager who has the backing of the board that is interested in change.”

Apparently Ubben thought that that energized leader might come from outside. The person familiar with his thinking says that Ubben’s hope was that he could work with AmEx’s board to begin transitioning to a new CEO. Perhaps he would have gotten a wink and a nod that Chenault would set a timetable for departing, giving Ubben an opening to try to attract someone of the caliber of Gordon Smith, the CEO of consumer and community banking at JPMorgan Chase.

The fact that Ubben decided to give up the fight probably means that he was unable to convince Buffett that wholesale changes were needed at the company, including new management.

For more on Ken Chenault, watch this Fortune video:

[fortune-brightcove videoid=3447034032001]

Ubben may have moved on, but the questions he raised about the future of American Express remain unanswered in the minds of much of Wall Street. “The company is going through a very challenging time,” says analyst Ryan Nash of Goldman Sachs. It’s not that AmEx is facing an existential threat, says Nash. It’s more that he and others wish Chenault were moving with greater urgency to reset its strategy.

Nash believes that American Express, like many other global financial companies, is facing a secular change that is affecting its ability to continue to grow at the same rates as it has in the past. “There’s a debate right now,” he says, “as to whether or not AmEx is still the great iconic brand that it once was and that it holds the same cachet as it historically did.”

Chenault could still win that debate—and every high-profile art purchase by Liu helps—but the case is getting harder to make. 

A version of this article appears in the June 15, 2016 issue of Fortune.