• Home
  • Latest
  • Fortune 500
  • Finance
  • Tech
  • Leadership
  • Lifestyle
  • Rankings
  • Multimedia

Trendingnow

1

After forcing workers back to the office, Goldman Sachs and JPMorgan Chase are now letting their staff work remotely—but only for the World Cup

2

Markets tumble worldwide as Fed resets expectations: $400 billion wiped off SpaceX stock

3

Current price of oil as of June 23, 2026

1

After forcing workers back to the office, Goldman Sachs and JPMorgan Chase are now letting their staff work remotely—but only for the World Cup

2

Markets tumble worldwide as Fed resets expectations: $400 billion wiped off SpaceX stock

3

Current price of oil as of June 23, 2026
TechAI

The Artificial Intelligence Race Is On to Help Take Down Violent Videos

By
Reuters
Reuters
and
Michelle Toh
Michelle Toh
Down Arrow Button Icon
By
Reuters
Reuters
and
Michelle Toh
Michelle Toh
Down Arrow Button Icon
April 28, 2017, 3:07 AM ET
Graymatics employees pretend to fight as they record footage to be used to "train" their software to watch and filter internet videos for violence, at their office in Singapore
Graymatics employees pretend to fight as they record footage to be used to "train" their software to watch and filter internet videos for violence, at their office in Singapore April 27, 2017. REUTERS/Edgar SuEdgar Su— Reuters
Add Fortune on Google for similar content.

Companies from Singapore to Finland are racing to improve artificial intelligence so software can automatically spot and block videos of grisly murders and mayhem before they go viral on social media.

None, so far, claim to have cracked the problem completely.

A Thai man who broadcast himself killing his 11-month-old daughter in a live video on Facebook this week, was the latest in a string of violent crimes shown live on the social media company. The incidents have prompted questions about how Facebook’s reporting system works and how violent content can be flagged faster.

A dozen or more companies are wrestling with the problem, those in the industry say. Google (GOOGL) – which faces similar problems with its YouTube service – and Facebook (FB) are working on their own solutions.

Most are focusing on deep learning: a type of artificial intelligence that makes use of computerized neural networks. It is an approach that David Lissmyr, founder of Paris-based image and video analysis company Sightengine, says goes back to efforts in the 1950s to mimic the way neurons work and interact in the brain.

Teaching computers to learn with deep layers of artificial neurons has really only taken off in the past few years, said Matt Zeiler, founder and CEO of New York-based Clarifai, another video analysis company.

It’s only been relatively recently that there has been enough computing power and data available for teaching these systems, enabling “exponential leaps in the accuracy and efficacy of machine learning,” Zeiler said.

FEEDING IMAGES

The teaching system begins with images fed through the computer’s neural layers, which then “learn” to identify a street sign, say, or a violent scene in a video.

Violent acts might include hacking actions, or blood, says Abhijit Shanbhag, CEO of Singapore-based Graymatics. If his engineers can’t find a suitable scene, they film it themselves in the office.

Zeiler says Clarifai’s algorithms can also recognize objects in a video that could be precursors to violence — a knife or gun, for instance.

But there are limits.

One is the software is only as good as the examples it is trained on. When someone decides to hang a child from a building, it’s not necessarily something the software has been programmed to watch for.

“As people get more innovative about such gruesome activity, the system needs to be trained on that,” said Shanbhag, whose company filters video and image content on behalf of several social media clients in Asia and elsewhere.

Another limitation is that violence can be subjective. A fast-moving scene with lots of gore should be easy enough to spot, says Junle Wang, head of R&D at France-based PicPurify. But the company is still working on identifying violent scenes that don’t involve blood or weapons. Psychological torture, too, is hard to spot, says his colleague, CEO Yann Mareschal.

And then there’s content that could be deemed offensive without being intrinsically violent — an ISIS flag, for example — says Graymatics’s Shanbhag. That could require the system to be tweaked depending on the client.

STILL NEED HUMANS

Yet another limitation is that while automation may help, humans must still be involved to verify the authenticity of content that has been flagged as offensive or dangerous, said Mika Rautiainen, founder and CEO of Valossa, a Finnish company which finds undesirable content for media, entertainment and advertising companies.

Indeed, likely solutions would involve looking beyond the images themselves to incorporate other cues. PicPurify’s Wang says using algorithms to monitor the reaction of viewers — a sharp increase in reposts of a video, for example — might be an indicator.

Michael Pogrebnyak, CEO of Kuznech, said his Russian-U.S. company has added to its arsenal of pornographic image-spotting algorithms – which mostly focus on skin detection and camera motion — to include others that detect the logos of studios and warning text screens.

Facebook says it is using similar techniques to spot nudity, violence, or other topics that don’t comply with its policies. A spokesperson didn’t respond to questions about whether the software was used in the Thai and other recent cases.

Some of the companies said industry adoption was slower than it could be, in part because of the added expense. That, they say, will change. Companies that manage user-generated content could increasingly come under regulatory pressure, says Valossa’s Rautiainen.

“Even without tightening regulation, not being able to deliver proper curation will increasingly lead to negative effects in online brand identity,” Rautiainen says.

About the Authors
By Reuters
See full bioRight Arrow Button Icon
By Michelle Toh
See full bioRight Arrow Button Icon
Add Fortune on Google for similar content.

Latest in Tech

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025

Most Popular

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Fortune Secondary Logo
Rankings
  • 100 Best Companies
  • Fortune 500
  • Global 500
  • Fortune 500 Europe
  • Most Powerful Women
  • World's Most Admired Companies
  • See All Rankings
  • Lists Calendar
Sections
  • Finance
  • Fortune Crypto
  • Features
  • Leadership
  • Health
  • Commentary
  • Success
  • Retail
  • Mpw
  • Tech
  • Lifestyle
  • CEO Initiative
  • Asia
  • Politics
  • Conferences
  • Europe
  • Newsletters
  • Personal Finance
  • Environment
  • Magazine
  • Education
Customer Support
  • Frequently Asked Questions
  • Customer Service Portal
  • Privacy Policy
  • Terms Of Use
  • Single Issues For Purchase
  • International Print
Commercial Services
  • Advertising
  • Fortune Brand Studio
  • Fortune Analytics
  • Fortune Conferences
  • Business Development
  • Group Subscriptions
About Us
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • Facebook icon
  • Twitter icon
  • LinkedIn icon
  • Instagram icon
  • Pinterest icon

Latest in Tech

Tesla cofounder JB Straubel’s first pitch to Elon Musk failed. Then he turned his ‘hobby’ into a $1.3 trillion success
SuccessBrainstorm Tech
Tesla cofounder JB Straubel’s first pitch to Elon Musk failed. Then he turned his ‘hobby’ into a $1.3 trillion success
By Rachel VentrescaJune 24, 2026
33 minutes ago
Amazon Prime Day isn’t a midsummer shopping event anymore. Here’s what changed in 2026
RetailAmazon
Amazon Prime Day isn’t a midsummer shopping event anymore. Here’s what changed in 2026
By Vidhi Choudhary and Retail BrewJune 23, 2026
10 hours ago
The hidden cost of your AI rollout: burning out the high performers running it
Workplace Cultureburnout
The hidden cost of your AI rollout: burning out the high performers running it
By Mikaela Cohen and HR BrewJune 23, 2026
11 hours ago
Quantum computing stocks surge after Trump signed executive orders backing the sector
Investingquantum computing
Quantum computing stocks surge after Trump signed executive orders backing the sector
By Marco Quiroz-GutierrezJune 23, 2026
11 hours ago
Alan Greenspan testifying before the Senate Banking Committee.
BankingFederal Reserve
The man who invented the Fed’s magic trick just died. His successor is about to try it again
By Eva RoytburgJune 23, 2026
13 hours ago
Google DeepMind CEO Demis Hassabis (left) stands on a spiral staircase next to Google DeepMind researcher John Jumper.
NewslettersEye on AI
Defections from Google DeepMind prompt questions about Alphabet’s efforts to stay at the forefront of AI
By Jeremy KahnJune 23, 2026
13 hours ago

Most Popular

After forcing workers back to the office, Goldman Sachs and JPMorgan Chase are now letting their staff work remotely—but only for the World Cup
Success
After forcing workers back to the office, Goldman Sachs and JPMorgan Chase are now letting their staff work remotely—but only for the World Cup
By Orianna Rosa RoyleJune 23, 2026
19 hours ago
Markets tumble worldwide as Fed resets expectations: $400 billion wiped off SpaceX stock
Banking
Markets tumble worldwide as Fed resets expectations: $400 billion wiped off SpaceX stock
By Jim EdwardsJune 23, 2026
21 hours ago
Current price of oil as of June 23, 2026
Personal Finance
Current price of oil as of June 23, 2026
By Joseph HostetlerJune 23, 2026
19 hours ago
Meet the 2 men putting New York's $300 billion pension fund in play for the first time in 20 years
Investing
Meet the 2 men putting New York's $300 billion pension fund in play for the first time in 20 years
By Nick LichtenbergJune 22, 2026
2 days ago
Former U.S. Secret Service agent says bringing your authentic self to work stifles teamwork: 'You don’t get high performers, you get sloppiness'
Success
Former U.S. Secret Service agent says bringing your authentic self to work stifles teamwork: 'You don’t get high performers, you get sloppiness'
By Sydney LakeJune 21, 2026
3 days ago
Texas and Charlotte used to build huge McMansions—now they're copying the California design tricks they once mocked
Real Estate
Texas and Charlotte used to build huge McMansions—now they're copying the California design tricks they once mocked
By Sydney LakeJune 22, 2026
2 days ago

© 2026 Fortune Media IP Limited. All Rights Reserved. Use of this site constitutes acceptance of our Terms of Use and Privacy Policy | CA Notice at Collection and Privacy Notice | Do Not Sell/Share My Personal Information
FORTUNE is a trademark of Fortune Media IP Limited, registered in the U.S. and other countries. FORTUNE may receive compensation for some links to products and services on this website. Offers may be subject to change without notice.