The Hazard Identification under Autonomous Drive. Why It Matters, and How We Learn to Turn Over the Control.

Dr. Rolf Johansson has more than 20 years of cross-industrial experience of complex critical embedded systems before he joined the SP Technical Research Institute of Sweden as a researcher in safety-critical systems. He holds a Master degree in engineering physics from Lund University and a doctoral degree in computer engineering from Chalmers University. Since 2010 he is with SP as a researcher and accredited safety assessor. He has been involved in the Swedish working group for ISO 26262 and the development of the standard from the very beginning.

Dr. Johansson , there is a controversy, or at least a discussion, about the ‘if’ and ‘how’ safety standards like ISO 26262, can be adapted to autonomous driving. Certainly, there are some areas which would be suitable, but others which are not so. Where do you see possibilities of adopting parts of the ISO standard?

The general discussion which is really political at the moment – is about the consequences of the international conventions we have today, and how they impact on the laws and regulations in different places. This very much goes back to the Vienna convention from the 60s, which stated in general that every vehicle should be controlled by its driver which leads to the question how we should interpret that. This is done very differently within different countries, and in the US within different states.

What is the Swedish approach?

In Sweden, we do things step by step, involving every actor, including the authorities – both the regional authorities and the traffic authorities from the central government that set the regulations. We are neither for non-autonomous nor fully autonomous vehicles. Instead, we are introducing this step by step to gain acceptance from society and to get the confidence that what we are introducing is safe enough.

Let me give you an example, when you start to ask the vehicle for ABS – that is, you press the brake pedal to the floor – things begin to happen that we can in a sense call autonomous. The car decides that it understands you want to stop and therefore takes over the control of how much to brake. It understands your intention, but you cannot request the maximum braking power at all times. You just ask the ABS to make the stop and the pressure of the brake is decided by the car, whatever you do or want. You cannot make an override, even if you think that in the particular special case you should apply more braking power. We have accepted that for a long time: controlling the car does not mean controlling the details. You can control your intention, knowing what the ability of the car is. Now, you can raise that level more and more.

Currently many cars offer Advanced Driving Assistance Systems (ADAS). Where do you draw the line between assistance features and autonomy? Is there even a line?

ADAS today are designed to control drivers. We assume that drivers are not perfect, and they are the ones that make the mistakes. ADAS should compensate for that. Then at the same time, to ask the driver to "compensate for the functionality that is there to compensate for the driver" is a very complicated documentation. The more responsibility you put on the ADAS system to really compensate for driving mistakes, the less you can put the responsibility on the driver to compensate for the mistakes of compensating for the mistakes. Today in an ADAS system, those kinds of failures where the ADAS functionality is non-working are generally considered non-safety-critical. Whenever we will develop a higher level of autonomy we have this little change where we say, "This is not assisting anymore, this is the responsible party." We ask the driver to do something else, not to try to make an override of the system. This has an impact on all parts of standard hazard and risk assessment.

One of the paradoxes here is that when you go fully automatic in the end, we all agree that the driver has nothing to do with controlling the vehicle. That’s the easy one.

[inlinead]

If we have a strategy of going step by step – leaving the responsibility in certain cases but not in others – there are tricky questions of who is currently in control of the vehicle, because as a manual driver I don’t hand over the control of the autonomous vehicle to the autopilot for all situations, I have the final say in some situations. This is our mitigation strategy, giving more and more to the automatic driver as long as we have the confidence.

How can these questions about the mandate of ‘override’ be addressed within the new edition of ISO 26262?

In ISO we need to take into account what it means when you do a handover from one driver to the other. How can you do that and how does it apply if neither of the drivers think that they are the one currently in control? There are potential failure cases, where both the manual and the automatic driver think that they will make the last decision because the other one is wrong, or both think "Oops! I made a mistake so the other one is in charge right now." This kind of conflict between two pilots – a manual pilot and an autopilot – are a class of problems that you have to consider when going more automatic than today. People working on interpreting the standards are not really confident of how to do this, which is a testament to the complexity of the problems. People are very keen on getting on the road very fast: they want to have fully autonomous vehicles. But, if they think the safety schemes prescribed today are too hard because of the complexity, that might be a sign that we are perhaps a little too early in trying to put these complex things on the road.

If we look at airplanes where pilots are used to being assisted by the autopilot most of the time, of course they know how to manage and operate their plane, but if suddenly some critical hazardous situation occurs, they might not be fast enough to respond. Is that a concern for the automotive industry?

There is a lesson we should learn here. There was an accident over the Atlantic not so long ago. The autopilot told the manual pilots that it was a situation where the autopilot could not take full responsibility because "it was not 100% sure" (inconsistent values from some sensors). It requested the pilots to take over. The pilots most likely thought, with no possibility to convince the autopilot that the situation was so complicated and confusing that it would have been preferable to make the autopilot continue. However, because of how the autopilot was designed its protocol said "Now you are in control. I’m not one hundred percent sure, so you have to take the necessary decisions."

So it transpired that the manual pilots had to assume manual control of the aircraft in a situation when they were confused and unsure about their capability. Eventually, the plane crashed. The question is, if the autopilot had continued without being one hundred percent sure, thus giving the manual crew the possibility not to take over, could the crash have been prevented?"

What is the learning from this episode?

Once you have given the control to an autopilot, you cannot then force it back to the manual driver. The autopilot should always, once it has got the control, have a strategy for all cases and know that the manual pilot can refuse to reassume control. There could be hundreds of reasons – maybe they are sleeping, maybe they are not paying attention, or maybe they say that the situation is too complicated for them. All of this is very much in contradiction to the way the ISO 26262 standard is written today, where it says that we should evaluate the situation from the perspective of the driver being able to take over.

The ISO standard has been differently received in different markets – North America, Europe, Asia, etc. The discussion here, in Germany and Scandinavia, seems to be much deeper than in other countries. How come?

If you have a number of business relations, adopting the language of the Safety Standard might make your business relations much more efficient if a significant number of them are talking using this language. It’s a unifying way to have the sort of dialogue that you want to have anyhow. I think all over the world and in different companies, it’s not as much a question of if, but rather when, and how much at a time, ISO 26262 is introduced. Being the leader is often costly, so many companies agree upon ISO 26262, but there is no reason for them to be the first adopters. Their rule is, "Let’s see what the others do, let’s listen to what problems others have, and when it’s the right time, we can do our mitigation when it’s efficient for us."