By Shelley DuBois, writer-reporter

With major players like ConocoPhillips (COP) and Marathon (MRO) splitting up, industry leaders and the market are starting to question the model of the huge, integrated oil company that handles every portion of the business, from plumbing crude out of the ground to selling it at the gas tank.

But just because the Big Oil’s big business model is being questioned — and rightfully so — doesn’t mean it’s going anywhere.

A few days ago, Fortune pointed out that many big oil companies are undervalued. Splitting them, in theory, could be worth billions of dollars to shareholders. But Big Oil companies need to consider many more factors than short-term shareholder perks before making such drastic decisions. Some may go for it, deciding to opt out of the integrated, one-stop-oil-shop model. But despite the changes major petroleum companies may experience, oil will likely stay Big with a capital B.

When it makes sense to be big, and when it doesn’t

Within the industry, different parts of oil companies are descried as sections of a river: upstream, midstream and downstream. A typical Big Oil company owns the whole river. Upstream involves looking for new wells, drilling them, and pumping crude out of the ground. Midstream means the transportation of new oil by ship routes and pipelines. Then you hit downstream, which is all about processing the product. Downstream assets include refineries, which distill crude into different chemicals, including gasoline. Gas stations, and other retail operations, are also downstream.

Big profits happen upstream. It pays well and pays quickly to strike oil, but companies have to spend a lot of money up front to explore new sites and then drill them.

When oil prices are low, it makes sense to have a hand in the refining and retail businesses, since both provide a steady stream of money that can fund searching for new wells and drilling. So when oil prices dipped during 1980s, then stayed low through the early 1990s, it spurred a mega-merger trend among major oil companies. Hence, we currently have companies like ExxonMobil, BpAmaco, and ConocoPhillips.

That model failed ConocoPhillips, which announced on July 14 that it will spin off its downstream division by the first half of 2012. The market reacted well to the announcement, and share prices of Conoco jumped 7.5% at the news, but the gap faded by the end of the trading day, closing at 1.6% higher than the opening price. Conoco’s breakup comes on the heels of Marathon’s announcement in January that it would undergo a similar split. The market loved it — the company’s share price rose 30% after the announcement.

The success of these sales generated speculation that other big oil companies may follow suit. The refining business is volatile, and may be more trouble than its worth, some argue. Also, many “integrated” companies aren’t actually integrated in practice, says Phillip Weiss, a senior energy analyst with Argus Research Group. “Generally integrated oil companies do not refine the oil they produce, instead it goes to wherever is closest,” he says.

That was the case with Conoco, and many analysts are starting to look to BP (BP) as the next in line for a split. The company needs to refresh its strategy. Its stock is undervalued, many analysts say. One J.P. Morgan analyst argued that a BP split could unlock $100 billion worth of value for shareholders.

But BP probably won’t split. Not that the integrated model is ideal. It isn’t. But it’s very hard to cut a giant in half. “The integrated model isn’t what it used to be and it may not be necessary,” says Weiss, “ but if you have it, it may not be easy to get rid of it.”

Why Marathon and Conoco are exceptions

Marathon and Conoco were in better positions to split than other companies. Marathon in particular is different: while many oil and gas companies are shucking their refining businesses in America, Gary Heminger, CEO of Marathon’s downstream spinoff Marathon Petroleum, believes the company will have an advantage in the market.

“Our strategy was that diesel sales around the globe are going to outpace gasoline, so we did something different,” Heminger told Fortune.

Instead of cutting off refineries, Marathon is upgrading them to produce a higher percentage of diesel than its competitors, which Heminger hopes will help the company meet growing demand in the global market.

Conoco, on the other hand, will probably focus the bulk of its business on oil exploration and drilling. But Conoco’s also in a slightly different spot than other super majors, Weiss says. “Conoco’s not as large as Chevron (CVX), Exxon (XOM), BP or Shell (RDSA). Sitting where it did, it probably was better positioned to make a run at doing this than larger companies.”

In the wake of these splits, Big Oil companies will be looking at their assets to trim extra flab. But they’ve already been doing this. Chevron announced plans to sell several downstream assets this year, including a major refinery in Ireland. And Shell has trimmed its refining capacity by 40% over the last 12 years.

Since many major oil companies are looking to get rid of assets downstream, it can be tough to find buyers. That’s why Conoco’s spinoff makes sense, says Weiss, because it’s better for shareholders to create a standalone refining company then sell downstream assets too cheaply.

Breaking up is hard to do

As anyone who has ended a relationship will tell you, breaking up can be brutal, even when it’s the right call. “The complexity of de-integrating these companies is massive.” says Alan Thomson, a Houston-based senior partner at The Boston Consulting Group.

You have to untangle shared departments, he says, such as environmental safety and finance. That’s why it does not make sense for a company like Exxon to split. “ExxonMobil has always been the quintessentially integrated company,” Thomson says. “They still very much manage that supply chain from crude to customer.”

Also, profit growth in refining may be slow in the U.S. and Europe, but having that ability can be an advantage in other places. “If you have aspirations in participating in China, being able to offer a full range of technologies and capabilities is still … very attractive,” Thomson says.

As the price of oil fluctuates, as it is wont to do, oil companies will mutate to profit most. Some will shrink and split. But for others, the best course of action will be to stay big. It all depends on whether the price of oil will stay high enough to warrant the headache of breaking apart the giants. Companies that split are banking that the price of petroleum to stay high. But if there’s any guaranteed uncertainty in this business, it’s the price of oil.