AT A CROSSING IN SHENZHEN, a giant screen displays the faces of pedestrians who dared to jaywalk, unaware they were being watched. But it wasn’t police who caught them—it was street-side surveillance cameras equipped with the latest in facial recognition technology. The miscreants’ mug shots are broadcast on the screen to shame others into compliance.
The technology behind the stunt was provided by a local company called Intellifusion, one of the smaller players in China’s advanced surveillance market. At the forefront of this booming industry are two other Chinese startups, SenseTime and Megvii, whose A.I.-powered facial recognition systems are likely the most powerful in the world.
SenseTime began as a research project at the Chinese University of Hong Kong. The company was established in 2014 by professor Tang Xiao’ou, an expert in the field of computer vision technology, and his protégé, Xu Li, who now serves as CEO. Today Sense- Time is the world’s most valuable A.I. startup, with a valuation of at least $4.5 billion.
At an estimated $2 billion, rival Megvii is less valuable. But the company is the originator of the world’s largest open-source facial recognition platform, Face++. More than 300,000 developers use Face++ to build their own face detection programs. Founded in 2011 by three graduates of Tsinghua University—China’s MIT equivalent—Megvii claims it wants to “build the eyes and the brain” of Chinese cities and extend police powers to a point “beyond what is humanly possible.”
For China’s government, that means not only being able to identify any of its 1.4 billion citizens within a matter of seconds but also having the ability to record an individual’s behavior to predict who might become a threat—a real-world version of the “precrime” in Philip K. Dick’s Minority Report.
To get there, the government is both benefactor to and beneficiary of companies like Megvii and SenseTime. More than 40 municipal police forces, known locally as Public Security Bureaus, have purchased surveillance systems from SenseTime. The company says its tech has helped Guangzhou police identify more than 2,000 suspects, arrest more than 800 people, and solve close to 100 cases since it was deployed in the city last year.
“The intention is to weave a tighter net of social control that makes it harder for people to plan action or push the government to reform.”
Megvii, meanwhile, supports the state’s nationwide surveillance program, which China, with troubling inferences, calls Skynet. Launched in 2005, Skynet aims to create a nationwide panopticon by blanketing the country with CCTV. Thanks to Face++, it now incorporates millions of A.I.-enhanced cameras that have been used to apprehend some 2,000 suspects since 2016, according to a Workers’ Daily report.
China claims such tech will be used to track fugitives and locate missing people, but the potential for misuse is obvious. Both Sense-Time and Megvii have absolved themselves of any responsibility for how their technology is used, claiming to simply be suppliers, not rulemakers. (Both companies declined to comment.)
For Luciano Floridi, director of Oxford University’s Digital Ethics Lab, this shirking of responsibility doesn’t fly. “The justification of saying ‘Oh, we’re just producing a tool’ has not worked since we started sharpening stones a long time ago,” he says.
In the U.S., some companies are actively seeking oversight for A.I. applications. In July, Microsoft, which both researches and commercializes A.I., called for “a government initiative to regulate the proper use of facial recognition technology.” Meanwhile, Axon, formerly Taser, has established an external advisory board to guide its development of A.I.-fitted police gear.
Jeffrey Ding, an Oxford University researcher focused on Chinese A.I., believes there is more pushback in the West against deploying facial recognition technology for security purposes. “There’s more willingness in China to adopt it,” he says, “or at least to trial it.”
But there’s also less freedom to oppose the onslaught. “The intention of these systems is to weave a tighter net of social control that makes it harder for people to plan action or push the government to reform,” explains Maya Wang, senior China researcher at Human Rights Watch. With ever more intelligent cameras, who will watch the watchers?
A version of this article appears in the November 1, 2018 issue of Fortune with the headline “Every Step You Take, They’ll Be Watching You.”
