A driver faces a terrible ethical dilemma: hit a pedestrian in the middle of the road, or swerve out of the way and drive into a brick wall?
Humans can make a decision in that split second that is reflective of their personal code of ethics. But what happens when they’re riding in an autonomous vehicle and are not in control of the car?
This question was raised at Fortune‘s Brainstorm Tech in Aspen, Colo., on Tuesday during a session on finding humanity in artificial intelligence, which addressed the ethical quandaries of the technology.
Omoju Miller, a machine learning engineer at software development platform GitHub, says that the decision will ultimately still lie with the person who owns the vehicle. “This is software,” she explained. A customer is delivered an autonomous vehicle, “and you choose your setting,” just like you would choose the language setting on your iPhone. “You’re going to pick what morality mode you subscribe to,” she says. “The car will make a decision based on your morality.”
Companies must also understand the nature of culture and cultures in different parts of the world when they do machine learning on human-curated data, Miller said. “You literally have to go around and talk to your friends, who are ethnographers and anthropologists,” she added.
Miller gave the example of diversity in the workforce and the challenges that come with trying to apply a global scale algorithm. “One of the biggest things about diversity in the U.S. might be race based,” she said, but if you then go to Nigeria, “the whole thing falls apart because it is not valued in that scenario. You have to build the systems in a way that adheres to the cultural norms of the environment in which they will be used.”
More must-read stories from Fortune Brainstorm Tech 2019:
—A.I.’s hidden biases continue to bedevil businesses. Can they be stopped?
—Land O’Lakes CEO: Big data is helping farmers deal with climate swings
—How Spotify “playlisting” turned an unknown artist into a star
—U.S. risks falling behind in crypto, warns ‘Crypto Mom’ SEC commissioner
—Verizon executive calls for federal privacy rules on 5G
Get Fortune’s Eye on A.I. newsletter, where artificial intelligence meets industry