A research team tied to Google unveiled a new tool on Thursday that could have a profound effect on how we talk to each other online. It’s called “Perspective,” and it provides a way for news websites and blogs to moderate online discussions with the help of artificial intelligence.
The researchers believe it could turn the tide against trolls on the Internet, and reestablish online comment forums—which many view as cesspools of hatred and stupidity—as a place for honest debate about current events.
The Perspective tool was hatched by artificial intelligence experts at Jigsaw, a subsidiary of Google-holding company Alphabet (GOOGL) that is devoted to policy and ideas. The significance of the tool, pictured below, is that it can decide if an online comment is “toxic” without the aid of human moderators. This means websites—many of which have given up on hosting comments altogether—could now have an affordable way to let their readers debate contentious topics of the day in a civil and respectful forum.
“Imagine trying to have a conversation with your friends about the news you read this morning, but every time you said something, someone shouted in your face, called you a nasty name or accused you of some awful crime,” Jigsaw founder and president Jared Cohen said in a blog post. “We think technology can help.”
The tool works through a rating system that reports how “toxic” a given comment is. It lets the website publisher, or even readers themselves, choose a “toxicity” threshold for comments that won’t be displayed publicly. This screenshot, for example, shows a climate change discussion that include a high level of negative comments:
And this screenshot shows more comments on the same thread, this time with the “toxicity” meter turned down to the left side of the spectrum. One could argue that the comments, while still opinionated, are less caustic:
In an interview at the company’s New York office, Jigsaw researchers told Fortune that they hope online comments will start to feel more like a “letters to the editor” section of a newspaper. Ideally, they will be a place to find what all sorts of people are thinking about a given issue—but without the rambling, inflammatory remarks that have become a hallmark of online commentary.
A tool for “Trump’s America”
Right now, the Perspective tool is designed only to tell if a given comment is “toxic” or not. But in the coming year, Jigsaw intends to use its machine learning tools to add other capacities. In particular, it will start screening for off-topic comments or ones that lack any real substance, meaning that website publishers, or their readers, could configure the comment settings so only the sharpest, most relevant stuff shows up.
Two publications, The New York Times (NYT) and Wikipedia, have already been working with the tool for months. This allowed Jigsaw to collect hundreds of thousands of comments, which served as training material for the company’s artificial intelligence technology. More recently, The Economist and The Guardian also signed on to try out the automated moderation tool.
“In Trump’s America, there’s this bifurcation. The Economist has long been a place for debate, but the comments are not at the level we’ve hoped them to be,” said Denise Law, the community editor at the Economist, adding that the “really good” comments often get lost amidst the ranting and rambling.
Law said that The Economist, which currently uses human moderators in an attempt to provide some level of comment curation, will begin by trying Jigsaw on certain blogs, and possibly expand from there.
She said the initiative is also driven by The Economist’s desire to capture conversations on its articles that take place on forums like Reddit and Quora—and subsequently, encourage those commenters to subscribe to its publication.
For more on online commentary, watch Fortune’s video:
While only these four websites have had a chance to try Perspective, this will change following Thursday’s announcement. As of today, anyone will now be able to ask Jigsaw for access to the API, a software tool that will let websites draw on the company’s repository of information and build moderation tools that are specially customized for their websites.
For now, website owners will have to rely on their IT staff to integrate the Jigsaw technology onto their sites. But soon, the company predicts third-party developers will build features allowing anyone, even if they lack any coding knowledge, to install the comment moderation tools.
A work in progress
Jigsaw’s plan to reclaim civility on the Internet is a noble one, but will it actually work? Fortunately, the company lets you test it out for yourself, with a section on the Perspective website where you can type in phrases to see how “toxic” they are.
I tried it out, and unsurprisingly, familiar insults or slurs score high on the “toxicity” meter—while truly offensive terms score near the 100% mark. So far, so good.
But I also tried a lesser-known insult (“libtard”) that’s become a nasty slang term and fixture on certain political sites. It appeared to pass muster.
The sarcastic phrase, “nice work, libtard” only obtained a 4% “toxicity” store, raising the possibility that would-be trolls will start reaching for newer or unusual slurs to avoid detection.
Meanwhile, I also typed the off-color but benign expression “life’s a bitch,” and found that the phrase got a very high “toxicity” score. The Jigsaw system, in other words, appears to be a work in progress so far.
This is fine with Lucas Dixon, one of Jigsaw’s research scientists, who pointed out that the system is built to improve the more it is used. He argues that the goal isn’t to achieve 100% accuracy, but rather, to make it good enough so that people with decent commentary are no longer reluctant to join an Internet discussion.
Dixon adds that website owners can set up Perspective to show how toxic a comment is even as it’s being typed—and that is proving to have a considerable effect on how people write. Jigsaw’s research shows that indicating to a writer that his or her comment goes too far will typically prompt them to alter it, and write something more civil.
