A Life and Death Question for Regulators: Is Tesla’s Autopilot Safe?

Robin Geoulla had doubts about the automated driving technology equipped on his Tesla Model S when he bought the electric car in 2017.

“It was a little scary to, you know, rely on it and to just, you know, sit back and let it drive,” he told a US investigator about Tesla’s Autopilot system, describing his initial feelings about the technology.

Geoulla made the comments to the investigator in January 2018, days after his Tesla, with Autopilot engaged, slammed into the back of an unoccupied fire truck parked on a California interstate highway. Reuters could not reach him for additional comment.

Over time, Geoulla’s initial doubts about Autopilot softened, and he found it reliable when tracking a vehicle in front of him. But he noticed the system sometimes seemed confused when faced with direct sunlight or a vehicle in front of him changing lanes, according to a transcript of his interview with a National Transportation Safety Board (NTSB) investigator.

He was driving into the sun before he rear-ended the fire truck, he told the investigator.

Autopilot’s design allowed Geoulla to disengage from driving during his trip, and his hands were off the wheel for almost the entire period of roughly 30 minutes when the technology was activated, the NTSB found.

The US agency, which makes recommendations but lacks enforcement powers, has previously urged regulators at the National Highway Traffic Safety Administration (NHTSA) to investigate Autopilot’s limitations, potential for driver misuse and possible safety risks following a series of crashes involving the technology, some of them fatal.

“The past has shown the focus has been on innovation over safety and I’m hoping we’re at a point where that tide is turning,” the NTSB’s new chair, Jennifer Homendy, told Reuters in an interview. She said there is no comparison between Tesla’s Autopilot and the more rigorous autopilot systems used in aviation that involve trained pilots, rules addressing fatigue and testing for drugs and alcohol.

Tesla did not respond to written questions for this story.

Autopilot is an advanced driver-assistance feature whose current version does not render vehicles autonomous, the company says on its website. Tesla says that drivers must agree to keep hands on the wheel and maintain control of their vehicles before enabling the system.

Limited visibility

Geoulla’s 2018 crash is one of 12 accidents involving Autopilot that NHTSA officials are scrutinizing as part of the agency’s farthest-reaching investigation since Tesla introduced the semi-autonomous driving system in 2015.

Most of the crashes under investigation occurred after dark or in conditions creating limited visibility such as glaring sunlight, according to a NHTSA statement, NTSB documents and police reports reviewed by Reuters. That raises questions about Autopilot’s capabilities during challenging driving conditions, according to autonomous driving experts.

“NHTSA’s enforcement and defect authority is broad, and we will act when we detect an unreasonable risk to public safety,” a NHTSA spokesperson said in a statement to Reuters.

Since 2016, US auto safety regulators have separately sent 33 special crash investigation teams to review Tesla crashes involving 11 deaths in which advanced driver assistance systems were suspected of being in use. NHTSA has ruled out Autopilot use in three of those nonfatal crashes.

The current NHTSA investigation of Autopilot in effect reopens the question of whether the technology is safe. It represents the latest significant challenge for Elon Musk, the Tesla chief executive whose advocacy of driverless cars has helped his company become the world’s most valuable automaker.

Tesla charges customers up to $10,000 (roughly Rs. 7.3 crores) for advanced driver assistance features such as lane changing, with a promise to eventually deliver autonomous driving capability to their cars using only cameras and advanced software. Other carmakers and self-driving firms use not only cameras but more expensive hardware including radar and LiDAR in their current and upcoming vehicles.

Musk has said a Tesla with eight cameras will be far safer than human drivers. But the camera technology is affected by darkness and sun glare as well as inclement weather conditions such as heavy rain, snow and fog, experts and industry executives say.

“Today’s computer vision is far from perfect and will be for the foreseeable future,” said Raj Rajkumar, a professor of electrical and computer engineering at Carnegie Mellon University.

In the first known fatal US crash involving Tesla’s semi-autonomous driving technology, which occurred in 2016 west of Williston, Florida, the company said both the driver and Autopilot failed to see the white side of a tractor trailer against a brightly lit sky. Instead of braking, the Tesla collided with the 18-wheel truck.

Driver misuse, failed braking

NHTSA in January 2017 closed an investigation of Autopilot stemming from that fatal crash, finding no defect in the Autopilot performance after some contentious exchanges with Tesla officials, according to documents reviewed by Reuters.

In December 2016, as part of that probe, the agency asked Tesla to provide details on the company’s response to any internal safety concerns raised about Autopilot, including the potential for driver misuse or abuse, according to a special order sent by regulators to the automaker.

After a NHTSA lawyer found Tesla’s initial response lacking, Tesla’s then-general counsel, Todd Maron, tried again. He told regulators the request was “grossly overbroad” and that it would be impossible to catalog all concerns raised during Autopilot’s development, according to correspondence reviewed by Reuters.

Nevertheless, Tesla wanted to co-operate, Maron told regulators. During Autopilot’s development, company employees or contractors had raised concerns that Tesla addressed regarding the potential for unintended or failed braking and acceleration; undesired or failed steering; and certain kinds of misuse and abuse by drivers, Maron said, without providing further details.

Maron did not respond to messages seeking comment.

It is not clear how regulators responded. One former US official said Tesla generally co-operated with the probe and produced requested materials promptly. Regulators closed the investigation just before former US president Donald Trump’s inauguration, finding Autopilot performed as designed and that Tesla took steps to prevent it from being misused.

Leadership vacuum in NHTSA

NHTSA has been without a Senate-confirmed chief for nearly five years. President Joe Biden has yet to nominate anyone to run the agency.

NHTSA documents show that regulators want to know how Tesla vehicles attempt to see flashing lights on emergency vehicles, or detect the presence of fire trucks, ambulances and police cars in their path. The agency has sought similar information from 12 rival automakers as well.

“Tesla has been asked to produce and validate data as well as their interpretation of that data. NHTSA will conduct our own independent validation and analysis of all information,” NHTSA told Reuters.

Musk, the electric-car pioneer, has fought hard to defend Autopilot from critics and regulators. Tesla has used Autopilot’s ability to update vehicle software over the air to outpace and sidestep the traditional vehicle-recall process.

Musk has repeatedly promoted Autopilot’s capabilities, sometimes in ways that critics say mislead customers into believing Teslas can drive themselves – despite warnings to the contrary in owner’s manuals that tell drivers to remain engaged and outline the technology’s limitations.

Musk has also continued to launch what Tesla calls beta – or unfinished – versions of a “Full Self-Driving” system via over-the-air software upgrades.

“Some manufacturers are going to do what they want to do to sell a car and it’s up the government to rein that in,” the NTSB’s Homendy said.

© Thomson Reuters 2021