Making Connection: Upcoming Speech and Gesture Control in HMI

New technology is leading to new ideas and new designs in the world of automotive User Interaction. There has been a proliferation of new innovations entering the market over the last few years, and OEMs are showing an increasing desire to introduce these designs as a way of setting their brand apart. Speech recognition and gesture control are amongst the most fascinating innovations to guide our way into the future of mobility. During this year’s re:publica 2015 conference, Automotive IQ met Philipp Wex, Senior Manager for Speech Interaction & User Sensing at Daimler. He is currently responsible for all speech interaction programs at Daimler / Mercedes-Benz and leads an engineering group in the Mercedes-Benz Technology Center. Mr. Wex, In-vehicle HMI is certainly one of the most fascinating areas and a growing field within OEMs and suppliers. Looking at the field of speech recognition over the last couple of years, where do you see the growing demand? We first introduced automotive speech recognition systems back in the 90’s, but then we only took very small steps – because the industry for embedded speech recognition products was only maturing and embedded technology capabilities were still quite limited. As soon as Apple Corp. launched Siri, and all the other players like Google came in, the market completely kicked off again. Today we are seeing small companies and start-ups appearing that are really amping up the whole industry. As customers get exposed to such new speech recognition systems and get used to them, there is a big push for us to make such features also much more accessible than we have done in the past. How would you describe the approach of tackling and embracing speech recognition within Mercedes-Benz? Are you trying to find solutions in-house, or are you going straight to co-operation with start-ups, tech companies and other specialists outside your own turf? One thing that we definitely will never let out of hands is the user experience, i.e. what the user really sees within the car. One example for this is the dialogue design: how you interact with the car via speech, how the car talks back, how you actually solve a problem and are guided to solve a problem. So, for example, if you want to navigate to a city like Stuttgart, all the necessary and optimized dialogue steps are crafted by us, and that’s a user experience we don’t want to let out of our hands. On the other hand we are talking about underlying technologies necessary for speech recognition, e.g. recognition algorithms, the signal shaping or signal enhancement. These are areas where we are partnering with technology companies. You mentioned in your presentation that the two main drivers for speech recognition would be safety and comfort within the car. This is true for most of the HMI technologies, but what about the safety aspect of it? When you look at infotainment solutions today – no matter which OEM – you see that using the system is becoming increasingly complex because of the increasing number of functions that need to be made accessible via such systems. Somehow each new automotive innovation will request a way to interact with one of these systems. For us, speech recognition is one of the easiest ways to activate those functions if it’s done well. I’m explicitly saying that because we still have to learn a lot: there is room for improvement in speech [inlinead] recognition, but we are on the right path. Once a customer gets used to using speech recognition and knows that wherever they are in the entertainment system, they can say, as an example "Dial Philipp Wex," and it works without having to scroll through a list in your address book, that’s a great benefit also for safety: it allows the user to keep focused on the road and on the business of driving. When I am able to tell my car via speech to make a U-turn next opportunity, for example, I definitely don’t want to have a malfunction within the system. I know that functional safety is always a huge barrier to climb for everyone. Are you already working on specific standards between OEMs? In terms of functional safety, to be clear, we’re not talking about using speech recognition. Just because, as you said, there is a risk of misunderstanding. We are not yet at a stage where we want to go through a certification process of speech recognition systems from a functional safety perspective. We’re thinking about it, but today the challenges and the hurdles to that are very high. Of course we would have use cases for it, but I think that really requires a long way to go. For you as an engineer, do you still feel that you are on top of the technology in terms of understanding and detecting all the complexity of these systems? It’s getting very complex, and that’s why we can’t do it all by ourselves. We need to partner with technology companies. There are companies out there that do a very good job of developing technology that can be used for speech recognition. Nevertheless, we definitely bring a unique perspective to that whole field. As an example: the noise environment of an A-Class is totally different from that of an S-class and again different from a transporter. Nevertheless, the user would expect a good performance no matter what the interior of such a car looks like. Also, the user interaction most likely needs to be different from a smartphone just because in our scenario the focus of the user needs to be on driving whereas the focus of a smartphone user can be on using the smartphone Let’s talk about the challenges you face within this process of getting ever more detailed and precise and complex. How about the challenge of reducing noise from outside the car or from within? It is always easiest when you have a microphone right in front of your mouth. Then you have a perfect speech signal. Within the car, the microphones are about 60-80cm away from you, either in the ceiling or in the rear-view mirror. So, the signal level of your input suddenly is quite different. A second dimension is road noise, which is significant when you are driving, and things like bumps in the road that disturb the signal. Then of course there is noise that you produce within the car, such as for example the windscreen wipers when it’s raining. Those are sources that we need to at least cope with before we bring a system to market. If we know them and we tune our systems towards them, our speech recognition accuracy will be improved. As soon as you have fixed the noise issue, the intelligently designed dialogue must be a huge factor. People want to talk to their car system properly. Where are you at this point? This is another area that I think is unique to us. Let me give you an example: When you use a system like Siri or Google, you have many requests that result in a web search. Within the car, a web search is mostly meaningless while driving – from a driver’s distraction perspective even something we would not be able to do. You can’t just bring up a browser on a head unit and have a textual representation which the driver would read through. As a result, we have to be very intelligent when designing such dialogues. We either have to talk about error recovery strategies – if you don’t understand you need to prompt back – but at some point the system status needs to be very determinable by the customer. Another topic is disambiguation. I had one example in my presentation: navigating to Neustadt. If you type Neustadt into Wikipedia, there is a list of about 50 different Neustadt results. You need to have a very good way for the customer to disambiguate those results. The customer has a certain Neustadt in the back of his mind, because he knows where he wants to go to. How do you get the customers to where they want to be? This is also intelligent dialogue design. And last but not least, there is natural language understanding. For example, if at some point you say, "I’m hungry," why shouldn’t the system understand that you are looking for a restaurant close by? This is something we are working on. This is a very interesting point, because at some point it touches base with the whole area of artificial intelligence. That’s why we are very closely following all those technology fields like artificial intelligence and machine learning, because those will be enablers for us to improve our whole user experience. We call it predictive user experience, and we showed some concepts at CES 2014 about this. We try to learn about the behaviour of the customer. One example: it's winter. Today, whenever you go out in your car to drive to work and it’s cold outside, you always have to turn on the seat heater manually. If the vehicle knew its environment, its context, and if would be able to learn about your behaviour, why wouldn’t it automatically turn on those heating systems? The whole topic of machine learning will open endless opportunities for us, and we will benefit from that. What are your expectations in terms of the relevance of HMI systems, especially speech recognition, in the next years? For example, one could argue that speech recognition and gesture control will be the big thing. Is there some sort of competition between different HMI techniques? Right now, we have the strategy to offer redundant interaction methods. For the future, I don’t really picture a vehicle without any displays at all. You need to have a location with all aggregated functions available to you. I also don’t picture a high-end system that doesn’t allow voice interfaces. We think that voice interaction is a good way to deal with complexity and offers many new possibilities. Looking at the whole topic of gestures: We have gestures today – we have a touchpad – and we have to think about what kind of gestures make sense. We want to make it safe and comfortable. Mr. Wex, thank you very much.

Making Connection: Upcoming Speech and Gesture Control in HMI

Upcoming Events

Vehicle Interiors Technology Summit 2026

Automotive Chassis Systems USA 2026

SDV & AV Technology Summit 2026

Automotive Cyber Security Europe

SDV & AV Technology Europe

Automotive Chassis Systems Europe

Follow Us

Latest Webinars

Automotive Autonomy: Comfort & Safety Redefined

Scaling Virtual ECUs for System-Level V&V Scenarios

Turbocharge your Automotive Software Development Life Cycle (SDLC) with Virtual ECUs

Recommended

Discover Infineon's PROFET™ family

Top 5 EV Platforms in 2025

Reducing Rare Earth Materials in E-Motor Design

Practical Strategies the E-Motor Industry is Taking to Reduce Dependency on Rare...

Optimize EV battery design and manufacturing for quality

Reducing dependency on rare earth materials: Practical strategies by the e-motor...

Interview | Taking a System-Level Approach to E-Motor Efficiency Optimisation

Achieving Balance Between Performance, Cost, Efficiency, and Sustainability in E...

PHEVs: The US Automotive Industry Shifts Gears

Practical Strategies to Reduce Dependency on Rare Earth Materials

The Rise of Silicon Carbide (SiC) in Electric Vehicle Power Electronics

Enhance EV Technology and Reduce Cycle Times with Emulation

FIND CONTENT BY TYPE

Automotive IQ COMMUNITY

ADVERTISE WITH US

JOIN THE Automotive IQ COMMUNITY

Making Connection: Upcoming Speech and Gesture Control in HMI

Upcoming Events

Follow Us

Subscribe to our Free Newsletter

Latest Webinars

Recommended

FIND CONTENT BY TYPE

Automotive IQ COMMUNITY

ADVERTISE WITH US

JOIN THE Automotive IQ COMMUNITY