In comments published last week by Car and Driver, Mercedes-Benz executive Christoph von Hugo said that the carmaker’s future autonomous cars will save the car’s driver and passengers, even if that means sacrificing the lives of pedestrians, in a situation where those are the only two options.
“If you know you can save at least one person, at least save that one,” von Hugo said at the Paris Motor Show. “Save the one in the car. If all you know for sure is that one death can be prevented, then that’s your first priority.”
Get Data Sheet, Fortune’s technology newsletter.
This doesn’t mean Mercedes’ robotic cars will neglect the safety of bystanders. Von Hugo, who is the carmaker’s manager of driver assistance and safety systems, is addressing the so-called “Trolley Problem”—an ethical thought experiment that applies to human drivers just as much as artificial intelligences.
In the original formulation, the trolley problem imagined someone standing at a railway switch, watching a train barrel towards a group of innocents on the track. The observer can choose to reroute the train to another track— where another person or smaller group would still be killed. Should someone making such a decision be praised for reducing the overall number of deaths—or blamed for the deaths of those sacrificed?
An autonomous vehicle will someday find itself in a parallel situation—having to choose, for instance, between crashing full-speed into a concrete barrier, or striking pedestrians while keeping its passengers safe. Earlier this year, a paper in Science (paywall) explored how we think about this dilemma in the case of driverless cars. It found that while people believed in the abstract that cars should choose to save as many lives as possible, most also said they would not buy a car that could choose to save others’ lives while putting their own at risk.
For more on self-driving cars, watch:
So, Mercedes’ decision to weight passenger lives more heavily than those of bystanders will seemingly make the cars more attractive to buyers. But von Hugo’s rationale is more broadly compelling—crash situations, he says, are so chaotic that an A.I. should prioritize the lives it has most direct control of.
“You could sacrifice the car. You could, but then the people you’ve saved initially, you don’t know what happens to them after that,” he told Car and Driver, “So you save the ones you know you can save.”
Update 10/20/16 1:25 p.m. EDT: In a comment provided later to Fortune, Mercedes-Benz insists that von Hugo was misquoted, and says the automaker’s official position is that “neither programmers nor automated systems are entitled to weigh the value of human lives,” and that the company is not legally allowed to favor one life over another in Germany and other nations.
