HMI Technology Report: The Holistic Approach

With infotainment and other electronics systems now proliferating system by system, it’s time they improve communication with each other to provide a seamless user experience and minimize driver distraction. Manufacturers also need to operate more intuitively for consumers, who increasingly start to consider HMI performance a key quality metric.

Connectivity in vehicles has made leaps and bounds over the last few years, and in a similar vein to the technological explosion we experienced when smart phones were first introduced, we are now seeing the same scramble to implement new features into our cars. Customers now demand to be connected in their vehicles, and manufacturers are keen to provide better, more exciting services to differentiate their brand and stay ahead of the competition.

However, as we have seen in the mobile phone market, just because you can develop something doesn’t necessarily mean you should. There have been a multitude of technologies and applications which have swiftly been consigned to the scrapheap when it became clear they were of no real use to the end user. The rapidly growing development of connected features in vehicles is putting a huge emphasis on the development of HMI systems capable of effectively managing them.

[eventPDF]

Unlike the mobile market, the automotive market has one very specific and important factor to consider – the safety of the driver. For consumers to be able to stay connected and take advantage of new features and applications, it is essential that the HMI system becomes a sophisticated user interface, able to interact with the driver, provide detailed information from a number of sources, and actively improve safety, reducing the chances of an accident.

This is a tall order, and most manufacturers are now coming round to the idea that one type of user interface is not necessarily the best way forward; but that the HMI systems of the future will be multi-modal and capable of interacting with the driver in a number of different ways depending on a number of different circumstances.

Multi-modal HMI to reduce driver distraction According to AAA research, in the U.S. driver distraction contributes to approximately 80% of crashes and 16% of fatalities. Especially teenagers are more likely to be in a crash than any other driver demographic: 6 out of 10 teen crashes involve driver distraction e.g. interacting with one or more passengers (15%) and using a cellphone (12%).

The general consensus throughout the industry until recently was that advanced infotainment and HMI systems were promoting safety. It was assumed that by encouraging drivers to keep their hands on the wheel and eyes on the road, voice activated and hands-free systems would provide a safe means of user interaction.

These systems are obviously preferential to a driver using a hand-held device as a means of communication, but research by the AAA Foundation for Traffic Safety last year questioned whether voice activated and hands-free systems are as safe as initially thought.

A comprehensive study found that as mental workload and distractions increase reaction time slows, brain function is compromised, and drivers scan the road less and miss visual clues such as stop signs and even pedestrians. This study caused a huge wave of publicity and criticism throughout the industry, but it remains an issue that will certainly have an effect on HMI design in the coming years.

The AAA predicts a five-fold increase in infotainment systems in new vehicles by 2018, and multi-modal systems which can provide information to the driver through a variety of sources will be a crucial when it comes to incorporating new connected features, and delivering greater functionality for consumers. Intelligent multi-modal systems will also be an integral feature of semi-autonomous vehicles, as drivers will require different information and will be able to use different interfaces as they switch between driving and monitoring the vehicle as it takes over the controls.

The technologies coming to the fore It is an exciting time in automotive HMI, and there are a variety of new technologies promising to add to the overall user interface. Here is a snapshot of just a few of the new developments which are likely to become part of multi-modal HMI systems in the near future:

Smart rearview mirror In the U.S. drivers have gotten used to rearview cameras, which make driving in reverse easier and safer and will be required on all new cars by May 2018. Now, Nissan is taking the next step forward, adding the benefits of a camera to the rearview mirror.

The "smart rearview mirror" looks like a standard mirror, but when the driver flips a switch, a much clearer view of what’s behind the car is shown, piped in from a camera in the rear windshield. This is a nifty piece of technology which utilises a high resolution camera mounted on the back of the vehicle, to provide the driver with a clear unobstructed view of the rear flanks, allowing the ability to check blind spots and other traffic conditions.

Image Source: Nissan

The design of some modern vehicles has led to a more restricted view through the mirror, while other obstructions such as C-pillars, passengers and luggage can all affect rearward visibility. The high quality camera and image processing system gives a wider view and is designed to provide a clear image in a variety of conditions which might otherwise diminish a driver’s view, such as bright sunlight, snow, or glare from the headlights of vehicles behind.

This is an excellent example of a piece of technology that can safely provide extra information to the driver in a seamless and non-distracting way as part of an overall multi-modal HMI system.

3D Gesture control Using gestures to control certain aspects of HMIs is an exciting concept. This is primarily because it presents an opportunity to bring back the direct control and feedback which existed in early cars, although it is not without challenges. The sensing of 3D gestural data is getting progressively easier, not only because of low cost sensors and processors, but also as better algorithms become available.

Sensors can detect not just macro changes in physical characteristics, like nodding, facial position and hand gestures, but also micro changes like eye movements. However, the new interaction patterns emerging from low cost computer vision have not been fully catalogued and understood, which poses a challenge when mapping and learning a gestural interface.

There are literally hundreds of 3D gestures possible and it takes time to learn and understand a set pattern and thus in its present state cannot be relied on as a pure interaction?—?especially with regards to safety.

Rather than fiddling with manual controls or concentrating on a touch screen, this technology will enable drivers to navigate in-car systems via a series of gestures, including swipe, scroll and zoom actions. Again, the key to this technology is that it reduces cognitive workload on the driver, and enables access to information in a much easier and safer way.

Volvo & Apple CarPlay In March 2014 Volvo Car Group announced that it is joining forces with Apple to bring the technology giant’s CarPlay system to its vehicles. The first model to incorporate the system will be the new XC90 SUV, due to be introduced later this year.

The new CarPlay system will bring all the services and features familiar to iPad, iPhone and iPod users directly into the car via Volvo’s large centre console touchscreen display. These functions will be enhanced as they will be linked to a specially-developed multi-modal interface which will allow drivers to use voice and steering wheel controls to access the different features and services.

Apple is set to make inroads into the car infotainment market, and similar systems can be expected to be introduced by a number of other manufacturers in the next few years. The integration of the recognisable user interface and the sophisticated voice control system, Siri, with Volvo’s touch screen and steering wheel controls creates a multi-modal HMI concept that is sure to develop further in the coming years.

Image Source: Volvo

Multi-Modal Displays lead the way The key to the multi-modal displays of the future is not to incorporate as much technology and functionality as possible, but to develop systems which actually benefit the driving experience and make it safer. Enhanced heads-up displays will allow consumers to begin to trust vehicle safety systems such as lane departure warning and automated braking, paving the way for wider acceptance of more advanced autonomous-vehicle technology.

Advanced HMI systems of the future will find ways of conveying essential information to the driver without causing a distraction from the road. They will incorporate vehicle controls which make it easy for drivers to navigate in-car functions without causing them to take their eyes off the road and without causing excessive cognitive workload. And they will use intelligent algorithms to predict the driver’s behaviour and most likely choices to present those particular features directly to the driver, making them quicker and easier to access.

The ultimate goals of multi-modal HMI systems will be to integrate all of the connected features and services that consumers demand, but to do so in a manner which does not distract the driver but actively improves safety.

Sources:

https://www.aaafoundation.org/measuring-cognitive-distractions

https://www.aaafoundation.org/using-naturalistic-data-assess-teen-driver-crashes

http://www.melexis.com/Optical-Sensors/Optical-Sensing/MLX75031-815.aspx?stab=video&utm_source=Print&utm_medium=Ad&utm_campaign=Gesture%20Recognition

http://www.intertraffic.com/news/Pages/Multi-modal-HMI-approach-will-combat-driver-distraction.aspx

http://www.mitsubishielectric.com/news/2014/0210.html

https://www.media.volvocars.com/global/en-gb/media/pressreleases/140633/volvo-cars-brings-apple-carplay-to-the-all-new-volvo-xc90

http://nissannews.com/en-US/nissan/usa/releases/nissan-motor-develops-the-smart-rearview-mirror-which-helps-provide-clear-rearward-visibility-in-various-conditions