Ethics on autopilot: The safety dilemma of self-driving cars

Self-driving cars, or autonomous vehicles, are revolutionizing how we move from one place to another. They're also raising major safety and responsibility issues.

In an article in the Journal of Information Technology, Business School Professor Michael Myers and co-authors discuss trustworthiness and the allocation of responsibility for autonomous driving, focusing on ethical and legal safety challenges.

"Our research indicates that there are contradictions in how responsibility is assigned for supposedly safe autonomous systems. These contradictions are linked and reveal ongoing confusion and lack of clarity about how responsibility is shared among different parties involved," the researchers state.

In an early case that demonstrates uncertainty around responsibility in the event of an accident, the United States National Transportation Safety Board found that human error was to blame in a 2016 Tesla crash. However, the safety body later revised its decision and criticized Tesla for allowing the autopilot feature to be activated on roads for which it had not been designed.

Autonomous driving systems mix a high level of socio-technical complexity with significant risks, says Myers, highlighting the massive recall of more than two million Teslas this week.

"It's ironic that one of the problems automation intends to solve, such as allowing individuals to relax instead of driving, still requires the driver to actively monitor the system if it's not fully autonomous. It's clear that drivers aren't always doing that, and this creates significant safety concerns."

Monitoring of autonomous driving systems requires a human to understand system operations, says Professor Myers, but the National Transportation Safety Board (US) says humans are "notoriously inefficient" at doing so.

"We're also finding that this kind of technology often leads to deskilling, and if an issue arises, a person may not have the skills needed to react when required," says Myers.

Many vehicles come preloaded with different software that's regularly updated, and the researchers say that although manufacturers continue to promote automation, people often have no choice about the level of automation installed in a vehicle and little knowledge of how it operates. Then, if there's an accident, people tend to be blamed.

The driver, however, might not know if they're totally in control in an emergency, say the authors, and as self-driving technology develops, the question of who is liable in the event of a crash needs far greater attention.

Because automated driving systems are connected to the external environment, they can't be tested in every situation, says Myers. As a result, they can be unpredictable, due to extreme weather, wildlife or road conditions that the vehicle is unfamiliar with.

"We are rushing headlong into automation without understanding all the consequences," says Myers. "Our project demonstrates the need for research that critically examines the social, political and technical aspects of autonomous driving systems, especially in relation to safety, responsibility and trust."