What Google’s ‘woke’ AI image controversy says about AI—and about Google

It was another big week in AI news with lots to discuss, from Nvidia’s soaring valuation following its stellar earnings announcement to indications that Big Tech is not as sure as it once was that proprietary AI models will be the path to cloud success.

But first, let’s talk about the culture war quagmire Alphabet waltzed into with an ill-conceived attempt to overcome AI’s inherent racial biases. The ham-fisted effort at putting some guardrails around the images from its Gemini models blew up in the company’s face, forcing it to temporarily disable Gemini’s image-creation capabilities and issue a public apology. Investors were not impressed, driving Alphabet’s stock down more than 4%. Some vitriolic critics even called for Alphabet CEO Sundar Pichai to step down or be fired.

The controversy highlights a few things. One is the public relations dilemma Big Tech in general, and Alphabet-owned Google in particular, faces on these sorts of issues. In 2022, OpenAI famously wrong-footed Google by releasing ChatGPT well before Google was ready to commercialize the rival LLM-based chatbot Lambda that it had long been incubating inside the company. At the time, Google used the fact that it was trying to be a “responsible” steward of AI technology as a justification for its inertia, and as a way to imply that OpenAI, and OpenAI’s partner Microsoft, were irresponsible in moving to commercialize generative AI so rapidly. Even as it began releasing commercial generative AI products of its own in a bid to catch up, Pichai promised the public that the company would always be “bold and responsible” when it came to AI innovation and gave the employees doing the work marching orders in line with that pledge.

Putting itself on this sort of pedestal when it comes to responsibility means that Google then has to try harder than other companies to put guardrails around the AI products it releases. But this may be a fool’s errand, for reasons we’ll explore in a moment. Other companies may not have tried particularly hard to deal with the well-known issue that AI models trained on historically biased datasets unsurprisingly produce biased results. In fact, there’s lots of evidence that Midjourney and OpenAI’s DALL-E produce racially biased imagery, and it hasn’t much affected investor sentiment around either company.

But Google being Google decided it needed to do something about the problem. The way it seems to have done so was to instruct Gemini behind the scenes to always generate images of an ethnically diverse set of people and to refuse prompts designed to have it generate images of only white people.

Of course, one person’s responsible is another person’s “woke,” and that was one of the big problems here. It also didn’t help that many on the right already see Google and its employees as hopelessly leftwing and were ready to pounce on exactly this kind of over-the-top effort at overcoming LLM’s racial bias. Elon Musk, who has promised that his Grok chatbot is “anti-woke,” happily helped ensure that Gemini’s issues with generating historically accurate depictions of ancient Rome or Vikings received wide airing.

More importantly, Gemini’s problems show the weaknesses of today’s AI models and our ideas about how to put guardrails around them. The idea of using metaprompts—or natural language instructions that are automatically appended to the user’s prompt but hidden from the user—as a way of creating guardrails, which seems to be part of what Google did with Gemini, is fraught. Why? Because LLMs, despite ingesting the entire internet’s worth of data, have extremely weak conceptual understanding and almost no common-sense reasoning.

Ideally, you want to be able to just tell the model “don’t be racist,” and have it understand what you mean and in what contexts it might be okay or not okay to depict non-diverse sets of people. If the model is uncertain, it ought to ask the user for clarification. That is what we would expect a competent human assistant to do if given those kinds of instructions. But the models we have can’t do this. And, in fact, Google seems to have built Gemini’s image generation guardrails partly through metaprompts and partly by fine-tuning the model only on images depicting diversity. But this made it so the model would struggle to generate non-diverse images even in contexts where that was appropriate.

“The technology is not very robust and there is no way to write an AI-based computer program that will make everybody happy all the time,” Meredith Broussard, the New York University journalism professor and author of More Than a Glitch: Confronting Race, Gender, and Ability Bias in Tech, tells me.

That said, the Gemini engineers who designed the guardrail prompts probably should have anticipated that users would ask the model for images of historical settings that were not diverse or try to find and break the model’s guardrails by asking it to generate images of only white or Black people. As much as this is a failure of the AI, it is also a failure of human imagination.  

Broussard traces the problem to the underlying assumption that you can build a “general purpose” conversation agent in the first place. Other AI ethicists have made similar points about the marketing of LLMs as general-purpose tools. In their view, general-purpose technologies are ethically problematic because it is inherently difficult to evaluate them.

They argue it would be better to go back to creating smaller AI models tailored for specific purposes. This would allow the model builders to carefully curate a dataset for the problem the user is trying to solve. If you want to build an AI to generate accurate portrayals of ancient Rome, for instance, why not build a smaller AI model only to do that and train it only on images related to ancient Rome? And if you want it to create an AI model to generate images for marketing contemporary products, then maybe you give it the kind of instructions Google gave Gemini. In theory, it ought to be possible to fine-tune today’s large LLMs on small datasets and with specific prompts, for purposes such as these. But Gemini is not in the business of selling a million small tailored models. There’s more money in selling a single model as the tool for every use case.

The idea of abandoning large models is also, in essence, an abandonment of the quest to create more human-like AI. Some AI ethicists would be fine with that. But a lot of other people would not. It is also, frankly, unrealistic. The genie is out of the bottle. I don’t think we are going to be able to put it back again and revert to simply using small models.

So assuming we are going to keep using large, multipurpose models, then we desperately need to figure out ways of getting the models to understand human intentions. Because it is extremely difficult to create a prompt that will cover every possible scenario, we want a model that has enough common sense understanding to know that when the creator of the model says “don’t be racist” we don’t mean that the model should depict a 9th-century Vikings settlement as if it were a meeting of the Rainbow Coalition.

This is also why I think the schism between researchers working on “responsible AI” and “AI Safety” is unfortunate. Traditionally it is the responsible AI folks who have cared most about AI’s racial bias problem while the AI Safety people, who are concerned about AI potentially killing us all one day, have cared most about making sure AI models can properly understand human instructions and intentions.

As it turns out, a lot of that AI Safety work could also help us build better guardrails that would allow AI models to not be racist, and also not be ridiculously woke. That’s the kind of “bold and responsible” AI a lot of companies would love to have. And it would probably make Alphabet’s shareholders much happier than they are today.

Below, there’s more AI news. But before you go, if you want to learn about the latest developments in AI and how they will impact your business, please join me alongside leading figures from the business world, government, and academia at Fortune’s inaugural London edition of our Brainstorm AI conference. It’s April 15-16 in London. You can apply to attend here.

Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn

Correction, March 1: A news item below wrongly stated that Nat Friedman is GitHub’s CEO. He is the company’s former CEO.

AI IN THE NEWS

Microsoft invests in Mistral. Microsoft has signed a multiyear strategic partnership with the high-flying Paris-based AI startup Mistral that includes an undisclosed investment, the Financial Times reports. The deal is significant because it shows Microsoft is looking to diversify away from being so reliant on OpenAI’s technology for its AI offerings. It's also another indication that Big Tech’s faith in proprietary AI models being the best way to serve customers may be wavering. Mistral is known for its highly capable open-source LLMs. And Microsoft’s move follows Google’s decision to launch its own line of Gemma open models, too. Taken together, the moves may indicate that cloud customers are balking at the expense and inflexibility of the proprietary models and opting for open-source options.

Microsoft president Brad Smith also framed the partnership with Mistral as a way to build AI models specifically for European governments, which have been wary of being dependent on U.S.-based technology providers, and which also want models that have been trained to perform well in European languages. Mistral, which is considered a darling of the Paris startup scene and a national champion in France, has touted its models' performance in French, German, and other European languages. Europe’s competition watchdog said it would look into Microsoft’s Mistral investment, according to Bloomberg. The agency is already investigating Microsoft’s relationship with OpenAI.

Tyler Perry suspends studio expansion after seeing OpenAI’s Sora videos. The actor and director had been planning a $800 million expansion of the studio he runs in Atlanta, but he told Hollywood Reporter that he was putting the project on hold after seeing OpenAI’s Sora videos. Perry said he has been closely tracking AI’s development but was blown away by the Sora demo. He said AI would soon mean “I no longer have to travel to locations. If I wanted to be in the snow in Colorado, it’s text. If I wanted to write a scene on the moon, it’s text, and this AI can generate it like nothing,” Perry said. He said he feared there would be massive job losses in the entertainment industry as a result.

Alibaba leads $1 billion funding round for Moonshot AI. The Chinese tech giant is putting the money into hot, one-year-old AI startup, alongside existing investor Monolith Management in a deal that values Moonshot at $2.5 billion. That is a stunning eight times its last valuation when the company was launched. The startup, which is known for its Kimi chatbot, is the vanguard of Chinese tech companies working on generative AI models. Its previous backers include food delivery company Meituan’s investment arm Long-Z and Hongshan, which was formerly Sequoia China, according to Bloomberg.

Coding AI startup Magic secures $100 million from former GitHub CEO. GitHub’s former CEO Nat Friedman and investing partner Daniel Gross put that figure into Magic, which is building a better AI coding co-pilot. The Information reports that the reason they invested is that Magic has achieved two breakthroughs. One is a 3.5 million token context window, which is three times what Google is offering with its new Gemini 1.5 Pro model. The other is a breakthrough in logical planning that produces better code but also might point the way towards AI models that can perform lots of other reasoning tasks better than existing models.

Figure 8 raises $675 million from Jeff Bezos, Nvidia, OpenAI, and others to pursue humanoid robots. That is according to a story in Bloomberg. The funding round values the startup at $2 billion. It is among several companies, including Elon Musk’s Tesla, pursuing humanoid robots to take on dangerous and repetitive industrial tasks.

Stability AI debuts Stable Diffusion 3. The controversial London-based AI startup unveiled an updated version of its popular open-source text-to-image generator. The new version uses a new AI architecture that is more similar to the one OpenAI has said it used for Sora. The resulting model is better at multi-subject prompts and typography, Venture Beat reports.

EYE ON AI RESEARCH

You can plan on it. Everyone thinks getting transformer-based models—which are the kind of AI design that underpins the entire generative AI revolution—to be better at planning is the next step in AI’s development. It could help turn today’s content-generating AI models into true personal agents, that can carry out tasks for us across the web. It could also make them much better at helping us with strategic planning or solving complex problems that the models have not encountered before in their training. This is why there was so much excitement about Magic AI (see news item above) and why rumors of OpenAI’s alleged Q* model, which could reportedly do the kind of basic mathematical reasoning that stumps a lot of today's AI chatbots because they don't know how to plan ahead, created such a stir.

Now researchers from Meta’s FAIR AI lab have taken a step in this direction by showing how a transformer-based model can be coupled with one of the best existing planning AI models, known as A*. (It is also thought that Q* probably builds on A*, which was a model developed by researchers at Google DeepMind.) Called “Searchformer” the new technique uses a transformer to predict the best search steps for the A* model to take. This reduces the searches A* must perform to find the best plan by about a third, the researchers reported. The method may solve a bottleneck to using A*, which is that the method requires a lot of computing power. You can read more about Meta’s work in this paper published on the non-peer-reviewed research repository arxiv.org here.

FORTUNE ON AI

Will AI soon kill off the humble app? Don’t bet on it —by David Meyer

Forget beating Amazon and Google, now Nvidia is part of the $2 trillion club —by Chris Morris

As Nvidia hits $2 trillion, billionaire Marc Rowan’s asset manager Apollo calls AI a ‘bubble’ worse than even the dotcom era —by Christiaan Hetzner

Retail investors have been clamoring to invest in AI. With Fundrise, there’s now a path into private AI juggernaut Anthropic —by Allie Garfinkle

The AI side bet: The less obvious contenders that stand to benefit from the halo effect —by Chris Morris

AI CALENDAR

March 18-21: Nvidia GTC AI conference in San Jose, Calif.

March 11-15: SXSW artificial intelligence track in Austin

April 15-16: Fortune Brainstorm AI London (Register here.)

May 7-11: International Conference on Learning Representations (ICLR) in Vienna

June 25-27: 2024 IEEE Conference on Artificial Intelligence in Singapore

EYE ON AI NUMBERS

500 million

Okay, this isn’t so much an AI number as a number showing you how much gets wasted by not using AI. It’s the amount, in dollars, the Federal Aviation Administration (FAA) is earmarking to spend building new air traffic control towers at U.S. airports. But, as a fascinating video report from the Wall Street Journal shows, that spending may be completely unnecessary.

That’s because increasingly high-definition cameras and systems that can fuse those camera streams with data from radar and transponders and lots of other sensors and then use AI to analyze all that data can let human air traffic controllers effectively monitor ground traffic at airports from locations below ground or even thousands of miles from the airport. These remote virtual air traffic control towers are already being used in many locations in Europe. But the FAA has so far refused to authorize any of these systems in the U.S., which means that is instead spending money to build new tower buildings. It seems like a missed opportunity. Anyway, check out the video for a glimpse into the future. (Of course, the only problem with remote towers is that they are more vulnerable to power outages and cyberattacks.)