AI will change the world. But that doesn’t mean investors will get rich in the process

I’m still buzzing from last week’s electrifying Fortune Brainstorm AI conference in London. So many great insights and discussions. Thank you to the readers who attended. And if you weren’t able to make it, you can catch up here.

One of the key themes that emerged at the conference is that while many businesses have been experimenting with generative AI applications, using all kinds of models and methods, relatively few have put generative AI into full production at scale in a business-critical application. Concerns about reliability and, even more prominently, cost and return on investment, continue to hold back full deployment.

There definitely seem to be some signs that the hype around AI is starting to deflate, and that we are perhaps sliding into the “trough of disillusionment” phase of this technology’s development cycle. Last week’s declines in the share prices of several prominent tech companies may be evidence of this. And later this week, all eyes will be on Microsoft and Alphabet’s quarterly earnings reports.

I, for one, remain convinced that this technology is real and will have a massive impact on how we work—and live—over the coming years. But that is not the same thing as saying that the companies at the forefront of the AI boom, or their investors, will be successful financially.

Last week, Air Street Capital, the London venture capital firm that is run by Nathan Benaich, who has emerged as one of the savviest early-stage investors in AI, published a provocative blog post arguing that the market dynamics for those building AI foundation models at least are looking particularly unpalatable. It’s a sharply reasoned analysis and well-worth reading for anyone interested in whether there is a sustainable business in selling foundational AI technology. Benaich and his colleague Alex Chalmers write “the economics of large AI models don’t currently work.”

The problem? The cost of both training and inference (actually running large AI models on GPUs) is too steep. This means that the operating margins for those offering access to these models through an API (OpenAI, Cohere, Anthropic) are lower than for other software firms, and overall profit margins are likely negative when capital expenditures are considered. (Google and Microsoft are also mostly in this camp, but for them, the models are either underpinning features in other software or serving as loss leaders for cloud computing services—so the business model is slightly different.) Making matters worse, open-source models that are being offered for free are gaining ground on the proprietary models. “We’re slowly entering into a capex intensive arms race to produce progressively bigger models with ever smaller relative performance advantages,” Benaich and Chalmers write.

They also write that the fact that the plethora of LLMs with relatively close capabilities is turning AI into a commodity business, where the AI startups engage in “a competition to raise as much money as possible from deep-pocketed big tech companies and investors to, in turn, incinerate it in pursuit of market and mind share.”

The duo draws an analogy between the companies building large foundation models and another industry that is highly capital intensive, where products are not highly differentiated, and that has also engaged in periodic price wars, destroying value for investors: airlines. It’s an interesting analogy in that the technology of global air travel was very real and definitively reshaped how we work and live. Air travel helped make our modern world. But that didn’t mean anyone could make any money at it. (Another good example is the buildout of the railways in the 19th century; again the tech transformed economies and nations but left a trail of bankrupt railroad companies in its wake.)

Benaich and Chalmers say that in such value-destroying industries, there is usually consolidation but that regulators may not allow that to happen with AI startups given that the most likely agents of consolidation are Big Tech companies that are already under intense antitrust scrutiny.

So where does this leave the AI industry? Well, they argue much smaller and less expensive models, used with fine-tuning and much longer context windows—meaning the model can ingest much longer prompts, including specific documents to analyze or summarize—will turn out to be sufficient for what many companies need to power AI applications. These small models can be run on less capable, older-generation GPUs. They might be served up on devices (on laptops or desktops, perhaps even mobile phones), which means that Nvidia GPUs won’t be in such high demand. (If one wanted another bullet point to add to this part of their argument here, one need only look at Microsoft’s Phi 3 announcement today which I cover further down in the Brain Food section of the newsletter.)

Benaich and Chalmers suggest the market will bifurcate: A few large companies that need the added capabilities of the largest foundation models will be willing and able to pay for them. That may allow a couple of proprietary model purveyors as well as the hyperscalers such as Alphabet, Microsoft, and Amazon, to still earn a modest profit. (But it may be a much smaller business than these cloud giants hope.) The two investors also imply early on in their blog that companies building AI applications that are not general purpose but instead highly tailored to a particular industry and the business needs of that sector will likely turn out to be better investments.

I am not entirely sure things will work out exactly as the Air Street guys layout. For one thing, in many cases, a few extra points of accuracy in a model’s reasoning ability on a benchmark, which may not seem like much, can make a huge difference to what a business can do with a model. It may be that the smaller models look good and are cheap but don’t cross a threshold of usefulness and reliability in deployment that will enable companies to avoid paying for the larger proprietary models. But it is certainly worth considering their bearish case for AI investors (and for Nvidia shareholders).

What do you think? And what’s the best analogy to the current AI industry dynamics? Is it airlines or railroads or something else entirely?

With that, here’s the AI news.

Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn

The news, research, and Fortune on AI sections of today’s newsletter were curated by Fortune’s Sharon Goldman.

AI IN THE NEWS

Japan’s SoftBank makes nearly $1 billion AI push. In yet another example of Japan’s accelerating efforts in the AI space, Nikkei reported that Japanese telecom SoftBank plans to invest $960 million by 2025 to purchase the AI chips—that is, Nvidia GPUs—required to develop and power its own large language model on par with OpenAI’s GPT-4. According to Nikkei, the massive investment in computing infrastructure is believed to be the largest of any Japanese company.

Adobe powers Photoshop with the latest version of Firefly, its generative AI image model. Adobe launched the latest version of its gen AI image model, Firefly 3, with an announcement that it is already available to use in a new Photoshop beta. Adobe’s original Firefly AI image model premiered a year ago to great acclaim, including an integration into Photoshop and a widely hailed Generative Fill feature that lets users add and remove content from images using text prompts. Since then, however, the competitive landscape of AI-generated imagery has become even more fierce. There is OpenAI’s DALL-E 3, the latest versions of Midjourney, and last week’s launch of Meta’s Llama 3, with its Imagine generator that produces real-time images as you type. So it will be interesting to watch whether Adobe can keep pace.

AI biotech startup Profluent brings generative AI to gene editing and CRISPR methods. Could an AI model edit the human genome? The New York Times covered AI biotech startup Profluent and its efforts to bring generative AI to the world of gene editing with CRISPR—a technology that can correct genomic errors and turn genes off or on. Interestingly, Profluent emerged out of a research moonshot at Salesforce called ProGen; one of Salesforce's researchers, Ali Madani, went on to found Profluent with backing from top VCs as well as Google chief scientist Jeff Dean.

Perplexity, an AI search engine startup, raises a new round at $1 billion valuation. As AI startups continue to try to poke at Google’s search dominance, Bloomberg reported that there’s a new unicorn in town: Perplexity AI has raised around $63 million in new funding that values the company at more than $1 billion. Yet, while Perplexity has been hailed by investors and users alike as a search game changer, many have come before them. After all, Microsoft’s Bing copilot was an early generative AI challenge to Google, while You.com has been in the AI search game since 2020—though it has never gotten the funding traction of Perplexity. And Neeva, a search engine that leveraged gen AI, was famously acquired by Snowflake last year, doing a total 180 to focus on enterprise search instead. So it remains to be seen whether Perplexity can make enough of a dent to last for the long haul.

EYE ON AI RESEARCH

Microsoft Research debuts VASA-1, a model that can deliver deepfakes in real time. It is getting easier and faster to generate a deepfake from photos and audio—something we all should be concerned about as the U.S. prepares for a presidential election and deepfake porn has already become a global problem. But Microsoft Research showed off an undeniably impressive example of where this is all going with a new model called VASA-1 that offers “lifelike audio-driven talking faces generated in real time.” With just one photo and a single audio clip, it is capable of a jaw-dropping display of natural head motions and facial expressions. Of course, it’s only a demo of a research version not available to the public—at least for now. 

FORTUNE ON AI

Finland—the world’s No. 1 coffee consumer—is turning to AI and lab-grown beans to energize the industry —Sasha Rogelberg

OpenAI’s Sam Altman is funding a green-energy moonshot as AI’s power demands grow to ‘insatiable’ levels —Dylan Sloan

AI is shaking up how sports like rugby, soccer and cricket are played—and could mint big money for sports clubs —Prarthana Prakash

Dean at top liberal arts university says AI could make Gen Z less skilled, not more: ‘You literally don’t need to know anything to use the technology’ —Ryan Hogg

‘Cesspool of AI crap’ or smash hit? LinkedIn’s AI-powered collaborative articles offer a sobering peek at the future of content —Sharon Goldman

AI CALENDAR

May 7-11: International Conference on Learning Representations (ICLR) in Vienna

May 21-23: Microsoft Build in Seattle

June 5: FedScoop’s FedTalks 2024 in Washington, D.C.

June 25-27: 2024 IEEE Conference on Artificial Intelligence in Singapore

July 15-17: Fortune Brainstorm Tech in Park City, Utah (register here)

July 30-31: Fortune Brainstorm AI Singapore (register here)

Aug. 12-14: Ai4 2024 in Las Vegas

BRAIN FOOD

Are small models better than ultra-large ones? And does data quality matter more than data quantity? Lately there’s been a lot of speculation that the creators of the largest, most powerful LLMs, such as OpenAI, Anthropic, and Google DeepMind, are running out of data to train models. After all, once you’ve already scraped the entire internet, what else is there? Some companies have been turning to synthetic data, using generative AI models to create more data that is in turn then used to train the next generation of generative AI models.

But Microsoft researchers have recently been taking a very different approach. They are looking at much smaller models, ones with just a tiny fraction of the number of parameters that models such as OpenAI's GPT-4 and Google's Gemini Ultra are thought to have. But they are finding ways to get these small models to do some impressive things.

Today, Microsoft debuted its latest family of Phi open-source AI models. Called Phi 3, it is a new family of models that are small by contemporary standards. The smallest, Phi 3-mini has just 3.8 billion parameters but, according to the company’s benchmarking, performs better than the leading 7 billion parameter open source models. (The company is also offering two larger Phi 3 models soon—a 7 billion parameter one and a 14 billion parameter one; all fairly lightweight compared to most LLMs.) How has Microsoft achieved this? Not by finding more data and not by using massive amounts of synthetic data, but by finding better data.

The idea behind the Phi models was to create “textbook quality” small datasets that teach an AI model exactly what researchers want it to learn, and no more. Rather than feeding it the entire internet, they train the models on a much smaller, but carefully curated high-quality dataset. For one of the precursor models to the Phi series, Microsoft created a dataset of just 3,000 words, roughly divided among nouns, verbs, and adjectives, and then used an LLM to create a training set consisting of stories written using just those 3,000 words. This is the same sort of approach, scaled up, that Microsoft has used with Phi.)

It turns out that creating a bespoke curriculum to teach a language model produces far better results than just chucking the entire internet into a statistical blender. Who would have thought?

What's more, it's interesting that it is Microsoft that is doing this work. It could be argued that no other Big Tech player, save maybe Google, has invested as much in ultra-large foundation models. After all, Microsoft has pumped $13 billion into OpenAI to help it build its GPT models. It has integrated GPT-4 into its own products, such as the Copilots for Microsoft 365 as well as Bing Chat. It has used GPT-4 as a big selling point for its Azure cloud computing services. But now it is Microsoft saying, well, maybe not everyone needs an ultra-large LLM solution after all. Maybe small can be beautiful, useful, and much more cost-effective for many customers. It is also saying that maybe open source is sometimes the better way to offer models than through proprietary APIs. It is a curious hedging of bets by the tech giant and its CEO Satya Nadella. And it will be interesting to see which of these bets ultimately pays off the most.