AI watermarks aren’t just easy to defeat—they could make disinformation worse

With all the legal drama surrounding Elon Musk and OpenAI these past few days, an important AI headline has fallen through the cracks. A pair of researchers said it took them just two seconds to remove watermarks intended to indicate content was generated by AI. It’s not the first piece of evidence to undermine the efficacy of AI watermarks and labels—my Eye on AI colleague Jeremy covered some of these issues in this newsletter in late January— but it certainly is another blow to the approach. And it comes just as platforms are increasingly throwing their weight behind watermarking amid mounting pressure to ensure users can determine real from fake in the lead-up to elections around the globe. 

The new findings were published in IEEE Spectrum, where the researchers specifically took aim at Meta’s recently announced intention to use the C2PA and IPTC technical standards for watermarking AI content to fight disinformation on its platforms, calling the plan “flimsy, at best.”

“Given that it takes about 2 seconds to remove a watermark from an image produced using the current C2PA watermarking standard that these companies have implemented, Meta’s promise to label AI-generated images falls flat,” they wrote. “We know this because we were able to easily remove the watermarks Meta claims it will detect—and neither of us is an engineer. Nor did we have to write a single line of code or install any software.”

The most obvious weakness, they said, is that Meta’s approach will work only if the bad actors creating deepfakes use tools that already put watermarks into their images, which isn’t the case for most open-source generative AI tools. They’re also concerned bad actors can easily circumvent Meta’s labeling even if creating content the company says will be detectable, including content made using AI tools from OpenAI, Google, Microsoft, Adobe, Midjourney, and Shutterstock. They did exactly that, almost instantly, and using nothing but the age-old screenshot.

Aside from watermarks—which involve embedding a unique signal into the actual output of an AI model, whether it be text, a photo, or video—other platforms are pursuing a labeling approach. YouTube and TikTok both last year announced new policies requiring users to disclose if a video they upload is created using AI. In addition to giving users a way to self-label their videos as AI-generated, TikTok has also been experimenting with using AI to detect and automatically label content made using AI. 

In many cases, however, these labels create more confusion than clarity. For example, the other day I came across a video labeled by TikTok as “AI generated” that I knew was not. It showed a laptop screen playing a clip from a podcast episode that I had coincidentally just listened to, and it was true to the original. 

When asked how TikTok applies its automated labels, the company told me it currently applies the “AI-generated” label to content that uses TikTok effects or tools powered by AI. The TikTok effects part of this may explain what was going on here, as the video began with a few seconds of voiceover introducing the clip in what’s commonly referred to as the TikTok “robot voice.” Yes, the voice is powered by AI text-to-speech technology. But it’s also existed unlabeled and as a staple on TikTok for years and…quite literally sounds like a robot. It was also a very minor component of the full video.

The industry is holding up watermarks and labels as a way to fight disinformation and prevent us from being deceived by AI-generated content, but in this case, I was more deceived by the label. If the brief use of text-to-speech was indeed the trigger for it, that would mean the video was considered “AI-generated” when its visuals were 100% human-generated, its audio was over 93% human-generated, and the AI part of the audio was still 100% true to the video’s content and just voiced by AI. It seems like a misapplication of the policy outlined by the company when you click on the label, which says “TikTok may automatically apply the ‘AI-generated’ label to content that we detect was completely AI-generated or significantly edited by AI.” And I’m not the only one who’s confused—TikTok users have been posting to the platform and other sites such as Reddit saying their videos are being labeled as AI-generated and they don’t understand why.

All this goes to show it’s not as easy as slicing and dicing what content is and is not AI-generated. It’s about making sure we can tell reality from content designed to deceive us, and it’s not going to be as easy as slapping on a label or watermark. In fact, these markers can even cause more confusion. 

Recently in MIT Technology Review, policy analyst Daniel Leufer made an important point that the mere existence of these labels and watermarks—no matter how accurate or inaccurate they are—bolster the perceived legitimacy of unlabeled content.

“Enforcing watermarking on all the content that you can enforce it on would actually lend credibility to the most harmful stuff that’s coming from the systems that we can’t intervene in,” he said.

I don’t think all hope is lost for watermarks and labels. There’s a lot of work going into them, and they’re worth pursuing. But they’re very much still a work in progress and also unproven at stemming disinformation.

“Unfortunately, social media companies will not solve the problem of deepfakes on social media this year with this approach,” wrote the researchers in IEEE Spectrum. “Indeed, this new effort will do very little to tackle the problem of AI-generated material polluting the election environment.”

And with that, here’s more AI news.  

Sage Lazzaro
sage.lazzaro@consultant.fortune.com
sagelazzaro.com

AI IN THE NEWS

DOJ charges a former Google engineer with stealing AI tech from the company for Chinese firms. The former Google employee, a Chinese national, had been secretly working with two Chinese companies while stealing AI-related trade secrets from Google, the Associated Press reported. He was arrested in California and has been charged with four counts of federal trade secret theft, each punishable by up to 10 years in prison. Lawmakers and justice department officials increasingly view AI as a significant national security concern and are sounding the alarm over how foreign adversaries could weaponize the technology against the U.S. 

Inflection says it has 1 million daily users, unveils more powerful Inflection 2.5 model. Inflection, the well-funded AI startup cofounded by former DeepMind cofounder Mustafa Suleyman, says its chatbot Pi, which it says is optimized for "empathy" and emotional intelligence, now has 1 million daily active users and 6 million monthly active users, Axios reported. The company also unveiled that Pi is now being powered by a new, more powerful LLM called Inflection 2.5 that comes close to matching the performance of OpenAI's GPT-4, including on tough topics such as mathematics and coding questions, while having been trained using what the company says is 40% fewer computing resources. This probably means the model is smaller and less expensive to run than GPT-4 or Google's Gemini 1.0 Ultra model. 

OpenAI hits back at Elon Musk's lawsuit, publishing his own emails with company leadership to show he was in on its for-profit plans. The blog post, published Tuesday by OpenAI, details how Musk and the company discussed for-profit plans in 2017 and how Musk wanted to personally control the venture. He sought majority equity, initial board control, to be CEO, and even suggested merging OpenAI into Tesla. He also withheld funding when the discussions didn’t go his way before ultimately stepping away, according to the post. “We're sad that it's come to this with someone whom we’ve deeply admired—someone who inspired us to aim higher, then told us we would fail, started a competitor, and then sued us when we started making meaningful progress towards OpenAI’s mission without him,” it reads. 

More than 250 AI researchers and academics sign an open letter calling on AI companies to allow access to their systems for “good faith research.” That’s according to the Washington Post. The letter was sent to companies including Anthropic, Google, OpenAI, and Midjourney and calls for “a safe harbor for independent AI evaluation.” “While companies’ terms of service deter malicious use, they also offer no exemption for independent good faith research, leaving researchers at risk of account suspension or even legal reprisal,” the letter reads. Signatories include Mozilla president Mark Surman, HuggingFace cofounder and CEO Clem Delangue, and notable computer science researchers from MIT, Princeton, Stanford, and more research institutions.

Mozilla is seeking nominations for its Rise 25 Awards. The organization is gearing up to recognize 25 everyday people from around the world who are driving ethical advancements in AI and shaping the future of the internet. It will award five honorees in each of the five categories: advocates, artists, builders, entrepreneurs, and change agents. Mozilla is accepting entries through March 29 and will honor the recipients in Dublin, Ireland later this year.

FORTUNE ON AI

Inside Mastercard’s multibillion-dollar AI arms race against fraudsters —by John Kell

Nvidia board members cash in stock in the $2 trillion AI company following blockbuster 27% run-up in price —by Amanda Gerut

AI could be critical to feeding a growing global population—and Big Food is taking notice —by John Kell

AI CALENDAR

March 9: MIT Sloan AI & ML conference in Cambridge, Mass.

March 18-21: Nvidia GTC AI conference in San Jose, Calif.

March 11-15: SXSW artificial intelligence track in Austin

April 15-16: Fortune Brainstorm AI London (Register here.)

May 7-11: International Conference on Learning Representations (ICLR) in Vienna

June 25-27: 2024 IEEE Conference on Artificial Intelligence in Singapore

BRAIN FOOD

Whose AI outrage matters? The recent controversy around Google Gemini’s image generation capabilities—wherein some accused the model of being “woke” after it included people of color in images where it didn’t make sense historically, and in some cases, refused to generate images of white people—has mostly calmed down. It culminated in Google CEO Sundar Pichai admitting, “We got it wrong,” and temporarily disabling the model’s image-creation capabilities. But I think there’s someone else who should get the last word.

“My only comment on Gemini is that y'all take everything seriously when it offends white people when we've been yelling about issues for how long and when has a product ever been pulled and when has there ever been this type of shock and media attention?” wrote Timnit Gebru in a LinkedIn post this week. 

Gebru is, of course, the AI researcher and former co-lead of Google’s ethical AI lab who was fired by the company in 2020 after she refused to rescind a research paper about the risks of large language models. And she has a great point. Gebru and countless other researchers have continuously surfaced evidence of AI models generating content that depicts non-white people in stereotypical, inaccurate, and offensive ways that would cause them direct harm, but the industry has largely accepted this as a cost of doing business in AI and something they’ll hopefully, eventually, work out a solution to. It hasn’t sparked this kind of immediate direct action, and for the most part, has been perceived as activism rather than technical due diligence around how well the products being developed and sold actually work. 

I spoke with Gebru and Margaret Mitchell, her former Google counterpart who now works as a researcher and chief ethics scientist at Hugging Face, about this exact topic and their experience a few years ago. They both shared strong feelings about how their work was perceived which speaks volumes against the backdrop of the Gemini controversy and the current state of AI overall. 

“If being against discrimination makes you an activist in someone’s mind, then chances are they have a very discriminatory view,” Mitchell said during that conversation, remarking on why she’s often been grouped with activists and why raising questions around how a technology adversely impacts different groups is often reflexively perceived by the industry as activism rather than making sure the technology works.

We’re now deeper into the AI revolution than ever before, with more capable (yet still deeply flawed) models being made available for public use nearly every day. So it is ironic and frustrating to be contending with the Gemini controversy now after years of proven concerns about racial discrimination and inaccuracies in AI models were largely ignored—dismissed as “activism” or relegated to  “responsible AI” talking points. Gebru and Mitchell were two of the company’s top AI researchers, paid by the company to help steer its AI technologies, and were fired. The Gemini controversy was promoted largely by right-wing posters on X and got an immediate response.