As Americans search for answers over Saturday’s shooting spree in Buffalo that left ten black New Yorkers dead, online chat forums including Discord and Twitch have come under intense regulatory scrutiny.
Right in the middle of a highly-charged debate over Elon Musk’s $44-billion planned acquisition of Twitter to rectify perceived left-wing censorship, Payton Gendron’s racially-motivated executions further polarized the issue of policing social media.
The 18-year-old killer posted repeatedly on Discord hate-filled diatribes after being radicalized online, and attempted to livestream his act on Twitch.
Both platforms boast tens of millions of users and arose mainly as means for video game players to share their virtual exploits, exchange tips and chat about their favorite lore.
On Wednesday, New York State attorney general Letitia James launched a formal inquiry into both Twitch and Discord, deemed a top candidate for the stock market with a $15 billion valuation.
“We are taking serious action to investigate these companies for their roles in this attack. Time and time again we have seen the real-world devastation that is borne of these dangerous and hateful platforms,” James said in a statement.
“The fact that an individual can post detailed plans to commit such an act of hate without consequence, and then stream it for the world to see is bone-chilling and unfathomable.”
Social media under the spotlight
While social media platforms Twitter, Snapchat and Meta’s Facebook were not mentioned by name, the state attorney general’s office said the inquiry is open ended.
The probe follows Sunday’s criticism from the state governor, Kathy Hochul, that these social media platforms had been abused as instruments that allow “hatred to ferment and spread like a virus.”
This is not the first time social media has been investigated following a mass shooting.
Authorities had examined the role played by Gab, a social media network that “champions free speech” in the lead up to the Tree of Life synagogue massacre in Pittsburg.
Some 12 months after the killer’s rampage in October 2018, the inquiry was abandoned however with no discernible results.
What is Discord?
Discord was founded in 2015 by Jason Citron and Stanislav Vishnevskiy to create a platform for gamers to chat, but has since expanded to encompass all sorts of communities meeting virtually for debate and discussion.
It has been likened to gathering around an “online water cooler”.
By comparison, Twitch was already launched back in 2011 and co-founded by Justin Kan and Kyle Vogt, CEO of General Motors’ robotaxi subsidiary Cruise.
With its focus on streaming video gameplay complemented by a live chat function, the service took off so quickly that it was scooped up by Amazon for nearly $970 million only three years later.
Synonymous with gaming tournaments like League of Legends, Twitch can draw audiences rivaling major sporting event viewership. According to the company at any given time 2.5 million viewers are tuned in and over an entire day some 31 million will access the platform.
How is Discord responding?
According to the Department of Homeland Security, racially or ethnically motivated extremism poses one of the most significant terrorism-related threats to the country.
Discord has been monitoring violent extremist groups ever since it learned how organizers of the 2017 Charlottesville white nationalist march used its platform to plan their activities.
At the time, its Trust & Safety team consisted of one employee, but by May of last year, it had grown to 15% of the company’s near 400-strong staff.
In its latest transparency report, it said only about 2.7% of the complaints it received during the second half of last year were linked to violent extremism, versus 21% for harassment and bullying.
Yet a review by Bloomberg of Gendron’s posts showed he had spoken explicitly about his plans to commit a terrorist act on Discord since at least last December, in part in an attempt to garner interest for his Twitch livestream of the killings.
In a statement published on Monday, Discord told the publication it extended its deepest sympathies to the victims and their families. “Hate and violence have no place on Discord,” it said, adding it is cooperating with law enforcement fully.
Free speech vs hate speech
Responding to the Buffalo killings, a “devastated” Twitch said it worked swiftly to take down the feed within minutes and emphasized its zero-tolerance policy against violence of any kind.
“We are taking all appropriate action, including monitoring for any accounts rebroadcasting this content,” it said in a statement to the media.
The greater scrutiny of Twitch, Discord and others raises ethical questions over Musk’s bid for Twitter.
The naturalized American citizen said a core rationale was to attract conservatives deeply suspicious of Big Tech censorship back to the platform.
Many had departed following the company’s high-profile ban of its most famous user, Donald Trump, for inciting violence that led to the Capitol riots last January.
Conservative financier Robert Mercer attempted to build an no-holes-barred free-for-all rival to Twitter called Parler, but it has struggled ever since cloud computing provider Amazon Web Services chose to no longer host it.
In addition to the voluntary efforts by platform providers themselves, regulators have also sought ways to enforce better policing of these sites.
After social media helped birth the QAnon movement and other conspiracy theories during the long Covid lockdowns, the Biden administration decided to recently create a Disinformation Governance Board in an attempt to monitor and report potential threats to national security.
On Wednesdy it was shelved, however, after conservatives lambasted it as an Orwellian “Ministry of Truth” only interested in policing free speech.
Sign up for the Fortune Features email list so you don’t miss our biggest features, exclusive interviews, and investigations.
