TikTok announced today that it will begin automatically labeling AI-generated content, such as photos, videos and audio that users post to its platform.
The popular video-sharing app has become the first social media company to agree to label AI-generated content using Content Credentials—a “nutrition label” technical standard for transparency, led Coalition for Content Provenance and Authenticity (C2PA), which Adobe co-founded. Content Credentials attaches metadata to digital files, including details on where and how the content, including AI-generated content, was made or edited.
Meta, which owns Facebook, Instagram and Threads recently announced it will start labeling AI-generated photos over the coming months using its own watermarking tools. Last month, YouTube announced a tool to allow creators to self-label their AI-generated videos.
TikTok’s announcement comes as AI-generated imagery has become increasingly sophisticated and difficult to tell apart from real photos and videos. For example, an AI photo of pop star Katy Perry at the Met Gala that went viral on social media was good enough to fool her own mother. And TikTok has begun clamping down on Taylor Swift AI voice clones that have generated hundreds of thousands of views.
According to TikTok, for the past year the platform had already required creators to label ‘realistic’ AI-generated content using TikTok’s own tool (cartoon-style AI does not fall under those rules, the company said, as long as it doesn’t violate other policies such as impersonation and misinformation). But it is unclear how well this requirement was enforced.
‘Realistic’ AI-generated content will automatically be labeled
Now, all realistic AI-generated content created using TikTok Tools will automatically be labeled using Content Credentials. The system uses cryptographic asset hashing—that is, they are tamper-evident – meaning you can tell if someone makes changes to the content or the attached data. The creator of the content can either include attribution or remain anonymous, while viewers can access historical information about the content with the metadata through the Verify site.
In addition, TikTok will expand the auto-labeling to AI-generated content uploaded to TikTok but created on other platforms that also use Content Credentials. Besides Adobe, these include OpenAI, which attaches Content Credentials to images created by DALL-E, and will apply them to Sora videos once the model is released to the public. Microsoft has brought Content Credentials to all AI-generated images created with Bing Image Creator, Designer and Copilot. Respeecher, a voice-cloning startup, has added Content Credentials to its output to indicate that an audio file was generated with its tech, while Leica recently launched the first camera with built-in Content Credentials.
In the coming months, TikTok will be building on this effort, said Adam Presser, TikTok’s head of operations & trust and safety. “Content that is created on TikTok and then is downloaded off of Tik Tok will also have Content Credentials embedded,” he told Fortune.
Labeling effort will not solve all issues with AI-generated content
Both TikTok and Adobe emphasized that Content Credentials will not identify all AI-generated content on the platform, since some may be uploaded from other tools and platforms not affiliated with the standard. In addition, Dana Rao, general counsel and chief trust officer at Adobe, said that the point of Content Credentials is about the ability to prove an image is authentic or has attribution rather than to identify all AI-generated content. “The thesis is that it is for somebody who says I want to be authenticated in a world where all digital content can be manipulated,” told Fortune.
Then there are security concerns: Recent research, for example, found it was easy to remove watermarks (an embedded marker not detectable by the human eye and ear) from Content Credentials. And, in fact, Adobe just expanded a bug bounty program for ethical hackers and security researchers to hunt down security flaws in the product. But Rao pointed out that Content Credentials combines secure metadata, watermarking and digital fingerprinting. “You have these three levels of security built into the standard,” he said.
But there are other issues as well, such as confusion about “AI-generated” labels. Was something completely generated by AI? Or simply edited? Of course, that is something that the historical information in Content Credentials is supposed to address—but, of course, users would have to be aware of what to look for and how the Content Credentials system works.
TikTok, for its part, says it is working with experts to develop media literacy campaigns, including videos that raise awareness around AI labeling and potentially misleading AI-generated content.
“These are building blocks, a series of tools that we want our viewers to have when they’re looking at content,” said Presser, adding that they are in addition to current policies in place to remove harmful AI-generated content and misinformation. “So that users will have the sensibility to be able to distinguish what is fact and what is fiction.”