Tesla officially touts its partial self-driving Autopilot feature as a backup to human drivers, but some of the carmaker’s comments —and some high-profile videos—have led many drivers to think of it as a replacement.
The questions of whether drivers are being lulled into a false sense of security by such technology, and who is legally to blame for an accident, have come into focus after it was made public on Thursday that a driver using Autopilot died when he drove under a truck in May.
An investigation of the crash has not yet established if the driver was distracted, or if his hands were on the wheel.
Tesla Motors (TSLA) Chief Executive Elon Musk told reporters last year that its partial self-driving technology was “hands on.” When Autopilot is activated, the driver hears a chime and sees an indication on the dashboard telling him or her to “please keep your hands on the wheel. Be prepared to take over at any time.”
Last October, Musk told reporters: “We’re being especially cautious at this stage, so we’re advising drivers to keep their hands on the wheel just in case.”
Get Data Sheet, Fortune’s technology newsletter.
But by April, he told a conference that Autopilot was “almost twice as good as a person,” even in its first version.
It is the second message that appears to have registered with drivers. Spurred by confidence in the system’s ability to self-steer and stay in its lane on highways, Tesla drivers have posted YouTube videos of themselves driving hands-free—even from the back seat.
One of them was Musk’s ex-wife, actress Talulah Riley, who in April was filmed by a friend covering her eyes and dancing behind the wheel of her Tesla Model X, while the car’s Autopilot feature drove the pair on a crowded highway. The video was posted on YouTube and has been viewed nearly 30,000 times. Riley was not available for comment.
Musk himself has retweeted news reports showing drivers using Autopilot with no hands on the wheel.
Mixed Messages
The potential for discrepancy in messages is a concern in the auto industry and beyond.
Brad Stertz, a spokesman for Germany’s Audi, told Reuters in November that carmakers must be “absolutely clear” what level their technology was capable of handling, after the launch of Autopilot.
“Kudos to Tesla for bringing out the system but you also need to be responsible and clear about what the technology is capable of doing,” Stertz said.
The self-driving feature appears to have played a part in the May fatality, in which a Model S driver using Autopilot hit a truck on a Florida highway. A video disc player was found in the car, police said.
In a statement, Tesla said that Autopilot “is by far the most advanced driver assistance system on the road, but it does not turn a Tesla into an autonomous vehicle and does not allow the driver to abdicate responsibility.”
Legally, Tesla may have a solid defense under Florida law, with safeguards telling drivers they cannot cede control every time they engage Autopilot.
Tesla itself rolled back some features in January, including reducing the speed at which Autopilot can operate on residential streets.
If the system senses a driver is not paying attention, it makes visual and auditory warnings to place their hands on the wheel or take over immediately. The car can sense whether a driver has hands on the wheel.
“It sounds like (Tesla) they did a fairly good job of designing into the mechanism prompts and reminders about what the deal was, and for whatever reason, this guy was not paying attention,” said Lars Noah, a University of Florida law professor.
Still, several Tesla owners said Autopilot could drive for several minutes before the system beeped to order hands onto the steering wheel. “It only beeps if it gets confused,” said Bob Preger of Incline Village, Nev.
Alphabet’s Google unit, which is also working on self-driving technology, is taking a different approach, saying that asking drivers to pay attention while the car drives itself is wrong. Google (GOOG) is focusing its efforts on fully autonomous vehicles where humans take no part in driving, whereas Tesla believes it should not hold back features to wait for the day full autonomy is perfected.
“Human drivers can’t always be trusted to dip in and out of the task of driving when the car is encouraging them to sit back and relax,” Chris Urmson, director of Google’s self-driving car program, told the United States Senate in March.
