As Boeing Co. was developing its latest version of the 737 airliner, it discovered the design was slightly more prone to a loss of control. So the company added a computer-driven safety feature — one that is now a focus of the investigation into a fatal crash last month near Indonesia.

If preliminary findings are borne out, the Oct. 29 crash of the Lion Air 737 Max 8 may end up being one of a number of cases in which the cockpit automation that’s made flying safer also had the unintended consequence of confusing pilots and contributing to tragedy.

For decades, planemakers have been adding automated systems to help pilots set engine thrust, navigate with higher precision and even override the humans in the cockpit if they make mistakes. Airline disasters have become increasingly rare as a result, but automation-related crashes have become a growing share of the few that continue to occur, according to government studies and accident reports.

“There’s no question that automation has been a tremendous boon to safety in commercial aviation,” said Steve Wallace, who served as the chief accident investigator for the U.S. Federal Aviation Administration. “At the same time, there have been many accidents where automation was cited as a factor.”

A 2013 report by the FAA found more than 60 percent of 26 accidents over a decade involved pilots making errors after automated systems abruptly shut down or behaved in unexpected ways.

For example, pilots on Air France Flight 447 inexplicably made abrupt movements and lost control of their Airbus SE A330 over the Atlantic Ocean in 2009 after they lost their airspeed readings and the plane’s automated flight protections disconnected. All 228 people on board died.

The U.S. National Transportation Safety Board concluded that pilots of an Asiana Airlines Boeing 777-200ER that struck a seawall in San Francisco in 2013 while trying to land, killing three, didn’t realize they’d shut off their automatic speed control system in part because it wasn’t properly documented.

Pilots on Lion Air Flight 610 were battling multiple failures in the minutes after they took off from Jakarta on the early morning flight, according to Indonesia’s National Transportation Safety Committee. The pilots had asked to return to land as they dealt with the issues, but plunged into the Java Sea at high speed before they could get back, according to investigators. All 189 people aboard were killed.

Flight Recorder

Data from the recovered flight recorder shows that the Max’s new safety feature, known as Maneuvering Characteristics Augmentation System, was triggered. An errant sensor signaled that the plane was in danger of stalling and prompted the MCAS to compensate by repeatedly sending the plane into a dive.

The pilots counteracted it repeatedly by flipping a switch to raise the nose manually, which temporarily disabled MCAS. The cycle repeated itself more than two dozen times before the plane entered it’s final dive, according to flight data.

This occurred as multiple other systems were malfunctioning or issuing cockpit warnings. Most notably, the cockpit was permeated by the loud thumping sound of a device on the captain’s side of the cockpit known as a stick shaker, which is designed to warn the pilots they are in danger of losing lift on their wings. The stick shaker was erroneous too, prompted by the same false readings from the sensor.

Boeing has stressed that a procedure pilots train for should have overcome the malfunction. “Automation is an aid to flight crews on our airplanes and is a significant contributor to improving safety, ” the Chicago-based planemaker said in an emailed statement.

Airline accidents almost never occur from a single cause and preliminary information from the investigation suggests multiple factors were at work in the fatal Lion Air flight.

While maintenance and pilot training may be found to be more significant, the underlying issue with an automation system behaving in unexpected ways puts the accident in a now-common category.

Man Versus Machine

Planemakers have been adding more automation to help pilots avoid errors as aviation technology has become increasingly sophisticated.

At Airbus, flight computers oversee pilots’ control inputs on models built since the late 1980s and won’t allow steep dives or turns deemed unsafe. Boeing’s philosophy has been to leave more authority in the hands of pilots, but newer designs include some computerized limits and, like Airbus, its aircraft are equipped with sophisticated autopilots and systems to set speed during landings, among other functions.

The new feature on the 737 Max family of aircraft was designed to address one of the most common remaining killers in commercial aviation. By nudging the plane nose down in certain situations, the MCAS software lowers the chances of an aerodynamic stall and a loss of control. Loss-of-control accidents killed 1,131 people from 2008 through 2017, by far the biggest category, according to Boeing statistics.

This type of automation is credited with helping create the unprecedented safety improvements of recent decades, yet it hasn’t been perfect.

“A lot of the experts have commented that human beings are not very good at monitoring machines,” said Roger Cox, a former NTSB investigator who specialized in pilot actions. “The reverse is better. Machines are pretty good at monitoring human beings.”

Devices that offer relatively simple warnings of an impending mid-air collision, for example, have proven nearly fool proof. On the other hand, more complex systems that aid pilots but require human oversight have on rare occasions confused crews and led to crashes.

It’s also important to keep in mind that issues with automation can be exacerbated by pilot actions, Cox said.

“Often times, what we call an automation error is really a proficiency error or a lack-of-attention error, and not fundamentally a fault of the automation,” he said.

NASA Research

Indeed, the pilots in the Lion Air crash didn’t follow an emergency procedure that could have deactivated MCAS and allowed them to fly normally, according to investigators. A different pilot crew the night before the accident had effectively shut off MCAS during an identical emergency and landed routinely.

At least one reason that these type of accidents occur may have to do with how pilots’ manual flying skills atrophy as cockpits become more automated, according to a 2014 study by NASA research psychologist Stephen Casner.

While basic tasks like monitoring instruments and manually controlling a plane tend to stay intact in the automated modern cockpit, the study found “more frequent and significant problems” with navigation and recognizing instrument system failures.

A different study by Casner and others in 2013 found a similar issue: flying has gotten so safe that pilots don’t experience emergencies much during regular operations, if at all. That is good news in the main, but it means that crews also aren’t as prepared.

The study suggested that airlines devise more realistic and complex training scenarios, and that they give pilots more practice reacting to emergencies that occur while automation is off.

“Where novices are derailed, discombobulated or taken by surprise when problems are presented under novel circumstances, experts characteristically perform as if they have ‘been there and done that,’ ” the authors said.