Shares of Buffett’s Berkshire Hathaway (BRKA) fell slightly, down about 1%, on Wednesday, a day after the company’s legendary CEO announced he has Stage 1 prostate cancer. Buffett said his condition is “not remotely life-threatening” and would barely slow him down. (He won’t be able to travel, while getting radiation treatment, for two months starting in mid-July.) Surely, part of the small drop has to do with the fact that few think Buffett is going anywhere soon. Nonetheless, the muted market reaction raises the question: What do investors think the 81-year-old CEO is worth?

A decade or so ago, investors used to talk about the Buffett premium. That appears to have mostly eroded. Berkshire’s shares trade for 1.2 times the book value of the company. That’s barely half the 2.3 price-to-book value they fetched back in 1998.

Part of Berkshire’s drop certainly has to do with Buffett’s age. But also weighing on the stock is the fact that it’s not clear who would run Berkshire when Buffett leaves. The best guess from what Buffett has said is that he envisions his job would be split into four, possible five, positions. Berkshire has already brought on investment managers Todd Combs and Ted Weschler. And the company has said it might add another manager to the mix. That would take care of the investing side, though it’s not clear how the portfolio would be divided.

MORE: How Warren Buffett spent the week he found out he has cancer

Buffett has also said that he plans to name another successor who will manage Burlington Northern and the acquisitions the company has made over the years, which includes ice cream chain Dairy Queen, paint company Benjamin Moore and many others. Buffett says he has picked the successor and it’s someone Berkshire’s board of directors knows very well. It’s assumed that the person Buffett has in mind is someone who already runs a company owned by Berkshire. Of those, the most likely candidate is probably Ajit Jain, 59, who has run Berkshire’s largest insurance division for 25 years. If Buffett is going for youth he might go for Greg Abel, 48, who runs Berkshire’s MidAmerican Energy and is one of Berkshire’s youngest top executives. But Matt Rose, 50, who is the head of Burlington Northern, might make the most sense, reflecting Berkshire’s transformation from an insurance business. But oddly, Buffett says the person who he has picked doesn’t know it. So it’s still possible the person could leave the company before Buffett does, or, perhaps, not even want the job.

On top of that Buffett has said that he would like his son, Howard, to become the chairman of Berkshire, a title that Buffett, Warren that is, also now holds. Howard wouldn’t have a day-to-day role at the company. How would all these people work together? Buffett hasn’t said. Would they even get along? Who knows.

But at least some of the erosion of the Buffett premium has to do with changes at Berkshire. A large part of the value of the company always had to do with the cash that was generated by the insurance companies and how Buffett invested that cash. But the company now makes a large part of its money from its operating companies. Buffett still invests that money, but he is relied on less to make that money grow than he used to be.

In fact, a number of Buffett’s ardent fans say that the current market value of the Oracle of Omaha these days is zero. How do they get at that conclusion? According to David Winters, who runs investment firm Wintergreen Advisors, if you were to split up Berkshire and sell off all the businesses, the product would be about $160,000 a share, or roughly 33% more than the $119,650 the shares trade at now. (Berkshire also has a class of cheaper B shares (BRKB) that trade for $79.76. The break-up value for those shares is just over $106.) Says Winters, “You are getting Buffett’s brain for free.”

But that doesn’t mean that Berkshire’s stock is unlikely to drop if Buffett were to suddenly leave the company – feet first or otherwise. Conglomerates, which is what Berkshire is now, often trade at a discount to the sum of their parts, sometimes as large as 50%.

Whitney Tilson, a hedge fund manager and Buffett investor, has said in the past that the market is undervaluing Berkshire. Still, he agrees that if Buffett’s cancer prognosis was that he had four months to live, the stock would at least drop by 10%, or roughly $11 billion. Perhaps, the biggest value that Buffett adds to Berkshire is his ability to pick up businesses and investments at better terms than others because he is Buffett. Tilson says, for example, Buffett was able to buy Israeli metal company Iscar in 2006 at a 50% discount to what others might have had to pay. During the financial crisis, Buffett was able to pick up shares in Goldman Sachs at better terms than the U.S. government. Companies, particularly ones seemingly in trouble, want to be able to say that Buffett believes in them. All that is certainly baked into the value of Berkshire’s shares but how much is hard to tell. “Buffett is a smart guy who does smart things,” says Winters. “No one knows how long he is going to live and how many more smart things he will do.”

But what is clear is that Buffett’s window for smart moves is far more closed than open.