I recently caught up with Ian Hogarth and Nathan Benaich, who each year produce The State of AI Report, a must-read snapshot of how commercial applications of A.I. are evolving. Benaich is the founder of Air Street Capital, a solo venture capital fund that is one of the savviest early-stage investors in A.I.-based startups I know. Hogarth is the former co-founder of concert discovery app Songkick and has since go on to become a prominent angel investor as well one of the founders behind the founder-lead European venture capital platform Plural.
There’s always a lot to digest in their report. But one of the key takeaways from this year’s State of AI is that concerns established tech giants and their affiliated A.I. research labs would monopolize the development of A.I. have been proven, if not exactly wrong, then at least premature. While it is true that Alphabet (which has both Google Brain and Deepmind in its stable), Meta, Microsoft, and OpenAI (which is closely partnered now with Microsoft) are building large “foundational models” for natural language processing and image and video generation, they are hardly the only players in the game. Loosely-organized collectives of A.I. researchers and well-financed, venture-backed startups are challenging these tech giants and their labs with models of their own. AI21Labs, an Israeli startup, has Jurassic, a large language model. So too does Clover, the A.I. research lab of Korean Internet company Naver.
“The traditional dogma in software is about centralization,” Benaich says. “Google, Apple, Facebook will win and build the best products because success begets success and they will just keep sucking up all the talent and having the most compute.” But this has not been the case with A.I. software. “Last year and this year, we see a lot of large scale results out of research collectives,” he says. “Progress is not centralized.”
Some of these newer players are also open-sourcing their models so anyone can build on top of them: Hugging Face has created BLOOM, a very large language model. Eleuther AI, another collective, has built GPT-NeoX, its own open-source riposte to OpenAI’s GPT (it is notable that it did so using Google’s Tensor Processing Units in Google’s datacenters, which Google allowed them to do for free). Stability AI has rolled out the very popular, open-source text-to-image generation system Stable Diffusion, which competes with OpenAI’s DALL-E. Open source versions of DeepMind’s protein-folding A.I. AlphaFold have also been created. (It is worth mentioning that at least a few of the newer research labs—such as Anthropic and Conjecture—were funded by now disgraced cryptocurrency mogul Sam Bankman-Fried. For more on the impact SBF’s downfall has had on A.I. research, check out last week’s issue of Eye on A.I.)
Interest in A.I. software startups targeting business use cases also remains formidable. While the total amount invested in such companies fell 33% last year as the venture capital market in general pulled back on funding in the face of fast-rising interest rates and recession fears, the total was still expected to reach $41.5 billion by the end of 2022, which is higher than 2020 levels, according to Benaich and Hogarth, who cited Dealroom for their data. And the combined enterprise value of public and private software companies using A.I. in their products now totals $2.3 trillion—which is also down about 26% from 2021—but remains higher than 2020 figures.
But while the race to build A.I. software may remain wide open for new entrants, the picture is very different when it comes to the hardware on which these A.I. applications run. Here Nvidia’s graphics processing units completely dominate the field and A.I.-specific chip startups have struggled to make any inroads. The State of AI notes that Nvidia’s annual data center revenue alone—$13 billion—dwarfs the valuation of chip startups such as SambaNova ($5.1 billion), Graphcore ($2.8 billion) and Cerebras ($4 billion). Seventy-eight times more papers used Nvidia hardware than Google’s TPUs. And 98 times more research papers were published in which the hardware used was Nvidia’s than the combined total of all papers citing chips from startups Habana Labs (now owned by Intel), Graphcore, SambaNova, Cerebras, and Cambricon. (Of all those challengers, Graphcore’s chips were used most often.)
Benaich says that the key to Nvidia’s success was not so much its hardware per se, but the popularity of the programming interface allows developers to implement A.I. applications on Nvidia’s GPUs, which is called Cuda. It is Cuda that has enabled Nvidia to “lock in” customers, according to Benaich. “The newer players didn’t focus on software early enough” he says. And Nvidia has continued to evolve Cuda to make it easier to build bigger A.I. models and run them much faster on its chips. “It’s hard to compete with an incumbent that behaves like a startup,” he says.
One portion of the State of AI always deals with politics and polices around A.I. Hogarth was keen to talk about the fact that A.I. seems to be becoming rapidly more capable in areas such as language and image generation, and yet work on how to ensure that A.I. is used safely and responsibly does not seem to be keeping pace. In the past, he says, rates of adoption of some of these systems had been limited by the number of companies that had access to OpenAI’s—and to some extent Google and DeepMind’s—large models. But the growing open source trend was democratizing access and accelerating adoption—which was something of a double-edged sword, according to Hogarth. Open-source models are easier to audit, for example. But they are also much easier for someone to use to generate misinformation or to perpetrate fraud. Hogarth, who invested in A.I. Safety-focused research lab Anthropic, says it is possible the viral popularity of image generation A.I. systems such as Stable Diffusion will wake people up to some of the potential dangers of the technology. He thinks there is “a moral hazard” in the asymmetry between the large amounts of funding going to creating larger and more powerful A.I. models and the relatively paltry resources, especially in terms of actual people focused on the area, devoted to A.I. Safety.
Every year, Hogarth and Benaich end the State of AI with some predictions for the coming year. The ones I found most intriguing in this year’s report were:
•A generative audio A.I. will debut that will attract more than 100,000 developers by September 2023.
•A proposal to regulate research organizations working on artificial general intelligence (this is A.I. that can match or exceed human performance across a wide range of disparate tasks) in the same way biology labs working with potentially dangerous pathogens are regulated will get backing from an elected politician in the U.S., U.K., or European Union.
•That the inability of the A.I.-specific computer chip startups to gain marketshare against Nvidia will result in one of the prominent chip startups being shutdown or acquired for less than 50% of the valuation implied by its last venture capital round.
•That a major user-generated content site, such as Reddit, will reach a commercial licensing deal with one of the major companies building generative models, such as OpenAI, in order to provide license payments for being able to train on their corpus of data. (Right now those building generative models have tended to just scrape this stuff from the Internet without paying anything for it, a controversial practice that has, in the case of Microsoft’s GitHub Copilot, lead to a landmark class action lawsuit.)
There’s plenty more in the State of AI to dive into. You can download the whole report here.
And here’s the rest of this week’s news in A.I.
Jeremy Kahn
@jeremyakahn
jeremy.kahn@ampressman
***
It’s not too late to join us at Brainstorm A.I.
Reid Hoffman is best known as one of the founders of PayPal and LinkedIn. But he’s also been a major investor into A.I. startups as a partner at venture capital firm Greylock. He sits on the board of OpenAI. And now along with DeepMind co-founder and Greylock colleague Mustafa Suleyman he’s co-founded his first company since LinkedIN, Inflection AI. And guess what? Hoffman will be giving the closing keynote at Fortune’s Brainstorm A.I. conference in San Francisco. The conference is taking place on December 5th and 6th and includes an amazing lineup of big thinkers on A.I. and on how A.I. is impacting business. Attendees will hear from luminaries such as Stanford University’s Fei-Fei Li, Landing AI’s Andrew Ng, Meta’s Joelle Pineau, Google’s James Manyika, Microsoft’s Kevin Scott, Covariant co-founder and robotics expert Pieter Abbeel, and Stable Diffusion’s founder Emad Mostaque. We will also hear from Intuit CEO Sasan Goodarzi and top executives from Sam’s Club, Land O Lakes, Capital One, and more. And there’s still a chance to join us. You can apply here to register. (And if you use the code EOAI you’ll get a special discount.) I hope to see you there!
A.I. IN THE NEWS
Amazon played a role in the demise of self-driving company Argo AI. That’s according to a story in Bloomberg News that cited anonymous people familiar with the situation. The news service reported that Amazon had been interested in investing in Argo, which was backed by Ford and Volkswagen. Amazon was considering using Argo’s self-driving software in the Rivian electric trucks it is buying and initially said it was willing to invest several hundred million dollars in Argo. Ford and Volkswagen were eager for a partner to help defray the spiraling cost of developing self-driving cars. But, according to Bloomberg, the companies could not come up with a governance structure for the partnership and Ford and Volkswagen grew concerned that Amazon would divert Argo’s attention from developing technology for cars. Maybe they should have been less concerned though—after Amazon walked away, Argo struggled to attract other investors and did not see a clear path to a public listing. Last month, Ford and VW decided to shutter the startup, which had once been valued at $7 billion.
A.I. analysis of dinosaur tracks suggests scientists wrong about fearsome predator. A machine learning analysis of dinosaur tracks at Lark Quarry Conservation Park in Australia has called into question a long-standing theory that tracks at the site are evidence of a “dinosaur stampede” caused when a large group of smaller herbivore dinosaurs were surprised by a very large predatory dinosaur. The A.I. system, which was trained on images of more than 1,500 dinosaur footprints from various places around the world, classified the larger set of tracks at the Australian site—which scientists had attributed to a meat-eating theropod—as likely made by a plant-loving ornithopod instead. In fact, it classified almost all the footprints at the site as having been made by ornithopods calling into question the entire stampede theory. You can read more in tech publication The Register.
U.K. police using A.I.-enabled cameras to catch people driving without seat belts, using mobile phones. The cameras, which are made by the company Aecom, have been in use since September in the English counties of Devon and Cornwall. While the British police have long used traffic cameras to detect drivers breaking the speed limit, the ability to analyze imagery of what the drivers are actually doing inside the vehicles with the help of machine learning is new. The police have caught almost 590 people so far driving without seat belts and 40 people driving while using their mobile phones, the BBC reported.
EYE ON A.I. TALENT
Dataiku, the New York-based machine learning and data software platform, has hired Ben Taylor to be its first chief A.I. strategist, the company said in a press release. Taylor had been chief A.I. evangelist at A.I. cloud company DataRobot.
EYE ON A.I. RESEARCH
Meta withdraws A.I. language model it said could write scientific papers after user criticism. The company’s A.I. research lab rolled out an ultra-large language model called Galactica that it trained on 48 million examples of scientific articles, websites, textbooks, lecture notes, and encyclopedias. The company said it could “summarize academic papers, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more.”
Meta was so sure it would be a useful tool, it let anyone start playing around with prompting the model and generating text. But what users discovered was that Galactica was mostly good at was making stuff up—it created Wiki-like articles, and even entire scientific papers complete with mathematical equations, that sounded convincing but were either entirely fictional or which confabulated some accurate information with some that was completely wrong. Researchers delighted in getting Galactica to write nonsense academic papers and posting the results on Twitter.
But, as MIT Technology Review reported, many scientists didn’t find it funny—they said Galactica could easily be used to generate misinformation. Several scientists thought Meta releasing it was unethical. After just three days, the company was forced to withdraw the model from public use. Yann LeCun, the well-known A.I. researcher who is Meta’s chief scientist, tweeted “Galactica demo is off line for now. It’s no longer possible to have some fun by casually misusing it. Happy?”
FORTUNE ON A.I.
Barclays and Jaguar looking to scoop up Meta and Twitter cast-offs to ramp up tech divisions—by Christiaan Hetzner
To prevent nurse burnout, a 35-year-old engineer built a time-saving robot that’s now deployed at top U.S. hospitals—by David Meyer
Commentary: Here’s how to survive the VC winter, according to EY’s U.S. venture capital leader—by Jeffrey Grabow
BRAINFOOD
Stable Diffusion doesn’t have a business model yet. But those building on top of its viral text-to-image software do. Generative A.I. is all the rage right now among venture capital investors and startups. Stability AI, the startup behind the text-to-image generation system Stable Diffusion raised $101 million seed round in October at a valuation reportedly north of $1 billion. But Stability AI has not yet figured out a business model. Its main product, the text-to-image generation system Stable Diffusion is open-source: not only is it free to use, others can download and toy with the software’s code. And while Stability hasn’t yet figured out how it is going to make money from this, that hasn’t stopped a crop of startups jumping in to make money on top of what Stability has created.
Last week, I spoke to Alfred Wahlforss, who is a master’s student in data science at Harvard. He and two co-founders—Axel Backlund and Florian Juengermann, both ace programmers (Juengermann, who was a national coding champion in Germany and recently interned on Tesla’s Autopilot team is a fellow Harvard computer science student)—have launched an app that allows users to take a photo of a person’s face and implant it into any fanciful scene they want, with the image created with Stable Diffusion’s text-to-image generator. They call their app BeFake (a play on viral social media app BeReal) and it has already attracted tens of thousands of users. The app has a freemium business model—it is free to download and try out, but then users have to pay to create the images.
Wahlforss says that the key is that BeFake doesn’t just use text prompts to tune a generic Stable Diffusion model. The team actually took the code of the model and tweaked the model’s weights to be able to create near-photorealistic images using people’s faces. Wahlforss says he thinks BeFake’s proprietary tweaks to the model—as well as the way it has figured out how to feed the model text prompts that will produce images that users want—will give the nascent startup enough intellectual property “to build a moat” around it.
How big a moat that could possibly be, definitely remains to be seen. Lots of startups out there are doing similar things. Wahlforss is also well aware of the IP challenges posed by Stable Diffusion and other large text-to-image generation models that have been trained on millions of images taken from the Internet, some of which are themselves copyrighted. He says he has even proposed a solution using blockchain technology and cryptocurrencies that would allow artist to receive small payments if their artwork contributed important elements to the training of an A.I. system that is later used to generate images incorporating elements of their unique style. “You could use A.I. models to figure out which part of the training data are most responsible and assign a small royalty to everyone who contributed that training data,” he says.
Wahlforss also knows there are potential privacy issues. That’s why BeFake doesn’t keep anyone’s facial imagery, he says, deleting those photos after they are used to create the fake images. And he also realizes that this new way of creating what are essentially deepfakes could be misused for fraud or misinformation.
But still, he is sure this is the future. “I think in the future everyone will be staring in their own favorite movies,” he says. “The next blockbuster will created in a dorm room by someone just typing out the movie or the video game.”
