Energy analysts have been making gaseous calls since Russia invaded Ukraine. It's time to clear away the smoke of economic groupthink

Today’s news of a surprise cut by Saudi-led OPEC+ of a million barrels a day has caught everyone off guard and sent oil prices momentarily higher–but really there is less here that meets the eye. Russia already announced a 500,000 barrel cut to disguise its own production woes, so really Saudi Arabia is cutting just 500,000 barrels from their quota, against a backdrop of oil prices halving over the past year and uncertain demand. Even more importantly, most OPEC+ countries produce less than their quotas allow, so few real barrels will be lost from the market with just “on paper” cuts–not to mention steadily rising production from the other OPEC+ countries and ugly internal OPEC fractures. The only surefire consequence will be Saudi Arabia and Russia losing even more market share to dramatic production increases in other countries ranging from the U.S. and Canada to Brazil and Guyana.

So once again, doomsday energy analysts are getting it all wrong–but sadly this fits a pattern. Energy magnate J. Paul Getty’s formula for success was “Wake up early, work hard, strike oil!” Energy analysts generally are awake early and work hard but have not exactly struck oil in their unfortunate forecasts this past year as oil and gas prices hit new lows every day despite near-universal predictions of dramatic price spikes over the last few months.

Their recipe for failure is to quickly issue new forecasts claiming to have always been right, reifying the old prognostication strategy that “if you can’t predict accurately, then predict often.”

As we repeatedly warned while developments unfolded over the last year, they have in rapid succession missed:

How oil prices fell by half from last year’s peaks of $140 to around $70 a barrel despite near-universal predictions of higher oil prices, with the average Wall Street analyst forecast exceeding $100 for this year.
How Europe avoided an apocalyptic energy crisis this winter after Putin choked off supplies of natural gas–not only thanks to warm weather but also because of savvy sourcing of additional supply and a helping hand from partner nations. Natural gas prices are now hovering at prices lower than before the war–and even lower than during the dog days of COVID.
How the price cap on Russian oil has effectively choked Putin’s revenue while minimizing disruption to global oil supply by keeping Russian oil export volumes on the market.
And much more, as we listed previously.

So how is it that energy prices are hitting new lows every day when experts uniformly predicted massive price spikes which did not come to fruition?

Given the relevance of energy to the economy, as well as the broader political and societal dimensions ranging from Russia/Ukraine to the renewable energy transition to ballooning consumer costs, we must draw valuable lessons from the dramatic failure of energy analyst forecasts over the past year.

These lessons are applicable to not only energy analysts but also to all industry expert groups regardless of field. One can avoid falling for the hazy obfuscation of expert groupthink by recognizing some classic traps.

Beware of groupthink

Too often, professional experts flock towards the same position–whether due to professional insecurity, a desire for peer affirmation, plain laziness, or risk aversion. Dissenters crashing the party of self-congratulatory agreement can be ostracized as troublemakers or malcontents. When we dared to publish pieces challenging the orthodoxy of energy analysts last summer–and challenging Putin’s propaganda that Russia can make up for lost gas exports–we were attacked as “over-optimistic” despite scant evidence and weak rebuttals.

In rushing to support the presumed conventional wisdom, so-called expert voices are often paradoxically, naively falling for misguided groupthink unsupported by factual evidence. Worse, they derive a level of confidence (if not certainty) that papers over genuine trade-offs, alternative choices, and potential blind spots. Of course, some analysts courageously took unorthodox positions–but on the whole, the uniformity of expert stances was rather striking. Indeed, as we list below, energy analysts tended to overlook the same factors, such as:

Misunderstanding the intention and impact of government policies and the G7 oil price cap

Government actions are becoming increasingly important to energy markets–yet too often, there is a massive gap between the intention of policymakers and how their actions are perceived by industry experts.

For example, even now, some energy industry experts declare that the G7 oil price cap on Russian oil has failed, citing how Russian oil export volumes have remained steady. But that is the entire point of the price cap! The Treasury Department intentionally designed the price cap to achieve a dual set of goals–not only limiting Putin’s profit but also maintaining global oil stability by making sure Russian exports continue to find their way to markets. They have even gone so far as to seemingly turn a blind eye to Indian and Chinese arbitrageurs purchasing Russian oil at deep discounts for resale.

Failing to understand novel government policies or a reflexive dismissal of the influence or efficacy of government policies led many analysts to biased and inaccurate forecasts of doom and gloom.

Underestimating the private sector’s support for Biden’s policies

Similarly, many energy experts initially dismissed the oil price cap’s prospects of success, arguing that private industry would not comply with G7 governments’ enforcement mechanisms.

Energy experts also did not believe that the U.S. government could help redirect private LNG cargos to Europe when the EU needed them, or that the Biden administration could actually forge friendlier relations with energy companies in a successful effort to raise production.

Sure, the administration did not necessarily start off on the best foot with the energy industry, but this kind of cynicism underestimates the latent power of governments over industry–both in terms of carrots and sticks–as well as the fact most businesses, especially large corporations, wish to be good citizens and rarely refuse a call to national service when asked.

When given the choice, private businesses will almost always seek to engage and partner with officials rather than face off as adversaries, even if takes a while to get there. Just last week, Earlier this month, Energy Secretary Jennifer Granholm and oil majors publicly exchanged praise at a major industry conference–a transformation in relations that few would have thought possible last year. Indeed, some ideologues on both sides of the political divide might be surprised to know that the Biden Administration has now approved more permits for oil and gas drilling on federal lands than the Trump Administration! Bet against the force of the public sector combined with the creative ingenuity of private enterprise at your own peril.

Dismissing organic supply chain resilience and the adaptability of global markets

Last year, even when oil prices were at $140 and natural gas prices were at a record high, energy industry experts still somehow predicted prices would continue to soar amidst “structural supply deficits.”

Analysts ruled out new supply for reasons as varied as: the politics of fossil fuels, a lack of access to capital, a preference for share buybacks, an absence of spare capacity, and many more dubious explanations. That’s seriously underestimating the natural resilience and adaptability of markets.

Sure, everyone was caught flat-footed by overnight market disruptions from Putin’s invasion of Ukraine–but all economies are inherently cyclical, and higher prices and the promise of producer windfalls inevitably incentivized greater production coupled with demand destruction until supply and demand were brought more into balance.

These analysts also missed that the response time for new energy projects, particularly U.S. shale, can be a mere month. Even when larger energy companies were dragging their feet on increasing production domestically, smaller operators showed no such compunction and helped lift U.S. energy production to near-record levels last year. Similarly, Europe used to buy 46% of its gas from Russia but quickly found replacement supplies, buying more U.S. LNG than it ever bought Russian gas and bringing Russian reliance down to 7%. Even Putin has learned the hard way that it is always easier for consumers to replace unreliable commodity suppliers than it is for disreputable suppliers to find new markets.

Conflicted sources and sensationalist media narratives

Much as how Vladimir Putin was incentivized to promote a narrative propaganda of energy scarcity to scare the West into surrendering Ukraine, clubby, insular industry expert cabals mindlessly talk their own book and parrot the interests of their clients. When we called out extortionate refinery margins last summer, several industry analysts expressed private agreement but reluctance to speak out publicly for fear of losing clients.

Media sources are biased in their own way, driving attention towards dramatized doom-and-gloom narratives that capture audience attention. Last summer, business media breathlessly played up Europe’s energy challenges, wondering whether Europeans would freeze in their homes or overthrow their governments while missing mitigating factors such as record levels of U.S. LNG shipments. As winter turns to spring with European gas storage levels remaining at record levels, we rarely see any media coverage of how Europe survived the winter far better than anticipated–with media already having moved on to playing up the next “crisis du jour.”

We must not be intimidated by intentional obfuscation by experts. Conflicted analyst predictions that collide with basic market principles have great consequences for society. Business executives, shareholders, government leaders, and journalists need the courage to challenge professional gibberish–even when it’s shrouded in the arrogant righteousness of conventional wisdom.

Jeffrey Sonnenfeld is the Lester Crown Professor in Management Practice and Senior Associate Dean at Yale School of Management.

Steven Tian is the director of research at the Yale Chief Executive Leadership Institute and former investment analyst at Rockefeller Capital.

The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.