In late August, an appalling trend spread on the viral video site TikTok: Users donned concentration camp garb, and pretended to be deceased victims of Nazi-era horrors. The trend even gave rise to a hashtag, #holocaustchallenge.
Tasteless or hateful memes rippling across social media are hardly new. But the TikTok incident is notable because of the platform’s incredible growth—it recently boasted of more than 100 million U.S. users—and its role at the center of a power struggle between the U.S. and China.
The controversy also raises the question of whether TikTok is fated to become the next Facebook—a massive social platform that profits from toxic content, and that cannot or will not do anything to stop it.
#holocaustchallenge leads to outrage, advertisers stay silent
The “Holocaust challenge” videos first began appearing in mid- to late August. It’s unclear if participants posted the videos as provocation or as a bid for attention or in a misguided attempt to educate followers about the horrors of Nazism.
The TikTok videos soon gained attention outside TikTok in late August as users on Twitter posted images like the one below to denounce them.
The trend gained the attention of media outlets, including the BBC, and also led to condemnation from Jewish groups.
“The digitization of anti-Semitism online and its corresponding reach on social media has contributed to rising levels of global anti-Semitism. The #holocaustchallenge is shameful,” said Holly Huffnagle of the American Jewish Committee in an email to Fortune. “Many of the insulting videos have been viewed thousands, if not hundreds of thousands, of times.”
TikTok responded to the outcry with a statement saying the company had blocked the hashtag and that it was redirecting those who searched for it to “Community Guidelines” so as to promote the “supportive, inclusive community we are working to foster.”
Unlike recent controversies over offensive material on Facebook, the #holocaustchallenge has not attracted any public backlash from TikTok advertisers, which include Kellogg’s and Procter & Gamble.
This may be because TikTok is still a blip in the digital advertising market with revenue of $177 million in 2019, according to one estimate, versus $70 billion for Facebook and $160 billion for Google.
Claire Atkin of Check My Ads, a site that helps big companies avoid placing ads next to offensive content, says clients have yet to raise TikTok as a concern and remain focused instead on other sites.
Kellogg’s declined to comment on the TikTok controversy, but a person familiar with the company said it had only purchased ads on the app in a pilot project that has since ended. A spokesperson for Procter & Gamble said the company constantly reviews media to ensure it does not advertise “near content that we determine is hateful, denigrating, or discriminatory.”
Meanwhile, the “Holocaust challenge” is just one face of what some call “trauma porn” on TikTok. Other examples include users role-playing 9/11 casualties or victims of domestic abuse.
Repeating Facebook’s mistakes?
While TikTok has yet to obtain the economic clout of Facebook or Google, it is rapidly acquiring an outsize influence on culture and politics. Research firm Statista reports that TikTok is used by more than six out of 10 teens, while the app is also being embraced by a growing number of celebrities and sports stars.
TikTok is also rapidly expanding its ad offerings. Last week, the company announced a series of partnerships with ad tech companies that will make it easier for brands to run marketing campaigns across thousands or millions of TikTok videos.
This growth suggests TikTok could soon rival Facebook—and give rise to the same torrent of hate, misinformation, and political mischief spawned by the earlier social network.
“While all of the social networks are investing a lot of resources to govern the content on their platforms…the reality is that such content will always find its way into the digital world in which we live,” says Yuval Ben-Itzhak, the CEO of Socialbakers, a firm that provides social media marketing for brands like McDonald’s and L’Oréal.
Critics of social media companies say Facebook and others fail to properly police their platforms for hate or misinformation because it is not in their interest to do so. They allege these companies turn a blind eye to false or incendiary material, since such content drives user engagement, which in turn produces ad revenue.
Those critics include Atkin of Check My Ads, which she founded with Nandini Jammi, who helped build Sleeping Giants, a group that pressures advertisers to stop supporting far-right media.
According to Atkin, TikTok has failed to build in safeguards to stop the viral spread of toxic content and is “just as dangerous as Facebook.”
She adds that TikTok may even pose a greater threat given that it’s a Chinese company and subject to censorship demands from Beijing. Atkin claims TikTok has scrubbed videos that call attention to China’s genocidal policies toward its Uighur population and has also programmed its algorithms to effectively ban people with disabilities.
If such allegations are true—reports from the Guardian and other media outlets suggest they are—then TikTok may have the ability to do more to suppress “trauma porn” videos in the same way it suppresses other content. (TikTok did not respond to a request for comment about the censorship allegations).
All of this raises the stakes in the current geopolitical fight over TikTok as the White House pressures its Chinese owner, ByteDance, to spin off its American operations to a U.S. company like Microsoft or Walmart. China has recently said it could intervene in any such deal.
Whatever the outcome of the TikTok negotiations, it appears for now that the company is unprepared or unwilling to prevent future unsavory trends like the “Holocaust challenge” from rippling across its platform.
More must-read finance coverage from Fortune:
- Investors are pouring record amounts into Wall Street’s new favorite “safe haven”
- First he took energy trading and the NYSE electronic. Now Jeff Sprecher of ICE shares his plans to digitize your mortgage
- The bizarre reason Amazon drivers are hanging phones in trees near Whole Foods
- The humbling of Europe’s most-hyped startup incubator: Rocket Internet
- Fortune’s 2020 40 Under 40
