Why Did Google Offer Black People $5 to Harvest Their Faces?: Eye on A.I.

Google provides the latest case in point. It came under fire after The New York Daily News reported last week that a contractor working for the tech giant had offered black people $5 gift cards in exchange for completing a demographic survey and agreeing to play “a selfie game” that had them performing tasks, such as following a dot on the screen of a mobile phone screen that the contract workers had brought along. What the contractor was not clear about was that while people were playing, the phone was capturing their images, which would later be used to train a facial recognition algorithm.

Workers for the contractor, the recruitment firm Randstad, wound up approaching homeless people, students, and people attending the BET Awards in Atlanta, many of whom later said they were unaware that buried in legal disclaimers they signed was the right to use their faces in this way. Some Randstad workers told The Daily News that Google managers instructed them to target homeless people specifically because they’d be unlikely to understand what they were agreeing to and were “the least likely to say anything to the media.”

The premise of the project was, at least in Google’s telling, actually pretty noble: past facial recognition systems have been found to perform much less accurately with darker-toned faces, partly because black faces were underrepresented in the large datasets used to train the systems. Google said it wanted to build a better dataset so the facial recognition—which it says will power the unlock feature on its new Pixel 4 phone—will be as fair as possible. But the lack of transparency used in collecting the new dataset is appalling. (Google said it had temporarily suspended the project and was investigating Randstad for violating its policies on transparency and consent for research.)

Google is not the only company to stumble in this way. Earlier this year, IBM sought to compile a more diverse dataset of one million faces and then make it freely available to academic researchers. But the company created the database by scraping images from the photo-sharing site Flickr and other public Internet sites, without seeking consent from any of those pictured.

These companies should know better. The fact that they keep getting it wrong makes me wonder if they actually want to get it right. Which is not to say this is an easy problem. Today’s A.I. systems require huge datasets to work well, and obtaining enough data, especially personal or biometric information, with proper consent, is a challenge. Ultimately, synthetic data—which is artificial data created by researchers to mimic the characteristics of real-world data—or simulations may offer a solution. But, in the meantime, businesses need to try harder.

Jeremy Kahn 
@jeremyakahn
jeremy.kahn@fortune.com

A.I. IN THE NEWS

U.S. Congress Investigates Military's Use of Facial Recognition. The House Committee on Oversight and Reform has launched a wide-ranging look at how the U.S. military is using facial recognition technology, according to a report in the technology publication One Zero. The publication had obtained a letter outlining the probe that the committee sent in July to the various branches of the armed services.

U.S. Regulators Probing Tesla Smart Summon Crashes. The U.S. National Highway Traffic Safety Administration (NHTSA) said last week that it is looking into a series of parking lot crashes that involved Tesla vehicles using a new feature called Smart Summon. The feature is supposed to enable a Tesla to autonomously leave its parking space and drive to its owner, so long as the owner is within 200 feet of the car and within a line of sight. But, according to Reuters, several owners have posted videos online that appear to show collisions or near-misses while using the feature. 

France Plans to Use Facial Recognition For New National I.D. France is set to become the first European country to use facial image data to create a supposedly secure digital identity for every citizen. President Emmanuel Macron’s government wants to roll out the new ID program, called Alicem, as soon as November. But according to an article in Bloomberg, the country’s own data regulator has said the program breaches European rules on consent and data privacy. 

Tensorflow 2.0 Is Here. Google has released the second full version of its popular deep learning language, Tensorflow. The new version, which had been in testing since the spring but which is now available for wider public use, allows for faster training times on Nvidia graphic processing chips as well as tighter integration with Python-based neural network toolset Keras. 

BE CAREFUL OF OVER-HYPING A.I.'S ABILITIES IN MEDICINE

A.I. Is About as Good as Doctors at Making Diagnoses from Medical Imagery. That was the conclusion of a review of studies where computer vision systems were pitted against human doctors, published in the medical journal Lancet Digital Health. Deep learning systems "correctly detected a disease state 87% of the time – compared with 86% for healthcare professionals – and correctly gave the all-clear 93% of the time, compared with 91% for human experts." But the review also highlighted the poor quality of a lot of A.I. medical research—the evaluators found only 14 papers, out of 20,000 published since 2012, had robust enough data and sufficient testing practices to use for the Lancet study. "This excellent review demonstrates that the massive hype over AI in medicine obscures the lamentable quality of almost all evaluation studies," David Spiegelhalter, a professor of statistics at the University of Cambridge, told The Guardian. 

EYE ON A.I. TALENT

Cazoo, a startup online car retailer, has hired Piers Stobbs as its chief data officer. Cazoo is hiring up as it prepares to enter the U.K. market. Stobbs was previously the chief data officer at MoneySuperMarket.

Neil Lawrence, a prominent A.I. researcher who formerly held two top roles overseeing machine learning at Amazon.com, has left the Everything Company to become the first DeepMind Professor of Machine Learning at the University of Cambridge. His chair was endowed by DeepMind, the London-based A.I. research firm owned by Google-parent Alphabet.

EYE ON A.I. RESEARCH

Using A.I. to Generate Comments for News Articles. Researchers from Beihang University in Beijing and Microsoft China have used two neural networks to extract information from news articles and then generate comments relevant to those articles, according to a paper published on the Arxiv research repository at the end of last month. The authors say their system, which they call DeepCom, may be useful to help sites generate more reader engagement. What they don't mention is that the same system could be used by troll farms or to further polarize political debate. 

Using Machine Learning to Better Track Political Trolls. In research that might provide an antidote to problems spawned by the DeepCom software mentioned above, computer scientists at Sofia University in Bulgaria, the ISI Foundation in Italy, and the Qatar Computing Research Institute successfully used a machine learning technique called graph embeddings to automatically analyze the behavior of political trolls, according to a paper published on Arxiv last week. The scientists trained and tested their software using more than 2.9 million tweets associated with the Russian-affiliated Internet Research Agency during the 2016 U.S. Presidential Election and later released by the U.S. Congress as part of its investigation into Russian political interference.

FORTUNE ON A.I.

Hong Kong’s Mask Ban Pits Anonymity Against the Surveillance State – By Naomi Xu Elegant and Grady McGregor

Why Tesla Quietly Acquired DeepScale, a Machine Learning Startup That’s ‘Squeezing’ A.I.– by Don Reisinger

This Startup Raised $3 Million to Staff Hospitals With Robots: Brainstorm Health – By Sy Mukherjee

BRAIN FOOD

Contested A.I. Competitions. A fascinating post by Luke Oakden-Raynor, an Australian radiologist, on The Health Care Blog has questioned the usefulness of A.I. competitions on benchmark datasets. He argues that the structure of these competitions, in which each competitor tests algorithms on the same unseen test set of data, don't do enough to control for dumb luck. In any competition, one of the competing algorithms will likely perform better than the others just by chance. Oakden-Raynor calls this problem "the over-fitting of crowds" and says it is the same as the problem of "multiple hypothesis testing" in epidemiology,  The only way overcome it, he says, is test the competing algorithms on much, much larger test sets that can produce statistically significant results. "You talk to competition organizers … and they mostly say that competitions are for publicity," he writes. "And that is enough, I guess. AI competitions are fun, community building, talent scouting, brand promoting, and attention grabbing. But AI competitions are not to develop useful models."