This is the eighth article in an 8-part series featuring articles on Innovations in Electric and Autonomous Vehicles. The series presents an insight into the developments and challenges in the automotive sector as it undergoes massive transformations with the arrival of electric and self-driving vehicles. This series is sponsored by Mouser Electronics. Through the sponsorship, Mouser Electronics shares its passion for technologies that enable smarter and connected applications.

Look at any modern car’s dashboard today, and you’ll see very little in the way of an instrument cluster of dials, gauges, and switches. It wasn’t that long ago that analog gauges and dials presented necessary information such as speed, engine revs, and coolant temperature. Today, you’ll probably be presented with a sleek glass panel in a prominent position within the cockpit, some of which look more like a tablet computer than a vehicle dashboard.

In this article, we take a look at innovations in human-machine interface (HMI) design for electric and autonomous vehicles. We analyze the critical factors to be considered by automakers for designing an intuitive and safe HMI system. In the later sections of the article, we explain the developments in displays, voice recognition, and touch controls. To conclude, we list out the common challenges in HMI design and discuss the current state of the industry.

Introduction

HMI is a combination of hardware and software systems that enable users to interact with a machine. Also known as man-machine interfaces, these systems must ensure smooth communication between users and machines. With advancements in sensing, display, electronics, and communication technologies, the way we interact with machines is changing.

Car dashboards with HMIs are sleek and stylish. Touchscreens are now the norm. They are used to navigate to a new destination, set the vehicle’s advanced driver assistance system (ADAS) features, or change the color of the interior trim lighting. Smartphones are tightly integrated into the HMI via USB or Bluetooth® through apps such as Android Auto and Apple CarPlay. For motor manufacturers, touchscreen HMIs herald a new era of replacing mechanical dials, buttons, and switches and advancing the aesthetics of a vehicle’s interior design.

Critical Factors of Automotive Human Machine Interface Design

Safety is a critical design factor for any automotive HMI. Its operation should be intuitive, not distract the driver,  and, if possible, disable access to certain features while the vehicle is moving. Selecting a different music album to listen to or reading a text message are all non-essential actions that result in the driver taking their eyes off the road ahead. With so many functions now available from an automotive HMI, the opportunity for distraction increases significantly, promoting legislation and international standards.

Although still in their infancy, standards such as ISO 15005:2017, which stipulates HMI actions should take 1.5 seconds or less, are beginning to evolve.^[1] For example, Volkswagen’s HMI monitors the driver’s usage and warns the driver if they spend too much time accessing different functions, such as searching for a contact in the address book or attempting to read an item of news.

The use of touchscreen controls in automotive HMIs is increasingly being questioned on safety grounds. Voice control is emerging as a safer, more viable candidate for HMI input and control, although voice recognition does present some technical challenges.

Apart from improving cockpit aesthetics, another driver of automotive HMIs is to present a simple-to-use, intuitive interface to access a host of comprehensive functions, particularly in the case of semi- and fully autonomous vehicles. Electric and hybrid vehicles require many different displays than a conventional internal combustion-powered vehicle, for which a software-driven touchscreen HMI is ideal. In the above examples, an increasing amount of user interface (UI) and user experience (UX) design effort is required to create an HMI that is informative and easy to use.

Over the years, vehicle manufacturers have learned that consumers are particularly attracted to HMIs and that user interface and user experience (UI/UX) is a significant factor in purchasing decisions. Developing custom icons, symbols, and fonts might be a chore for the embedded developer, but such attention to detail pays dividends.

For the driver and their passengers, the HMI permits the personalization of vehicle functions to suit their mood, from comfort features and interior lighting to balancing engine performance with fuel economy. Let’s review some of the technologies used and the challenges that extremes of temperature and humidity present to the design team.

Popular Automotive Human Machine Interface Technologies

HMIs need two essential technologies; a display, typically an LED/LCD screen, and a method of input or control. Touchscreens are a proven HMI method, using projected capacitive touch (PCAP) sensing in a thin stack up above the screen. 

Voice recognition, using machine-learning neural networks, is another input method that is rapidly gaining popularity based on driver safety. Other technologies used with a touchscreen include haptics, gestures, proximity, and force detection. As application processors become more powerful, the use of eye-tracking algorithms to provide input controls can become attractive.

Voice Recognition

According to automotive research company Tractica, the use of in-vehicle artificial intelligence (AI)-based voice recognition HMI assistants will grow significantly in the next few years, creating a global market worth $4.6 billion (USD) by 2025.^[2] An estimated 80% of automotive HMIs will integrate a voice-recognition system, excluding smartphone assistant applications such as Apple’s Siri and Google Assistant.

Voice commands are already regularly used to control the media functions and to find the quickest route to a destination. Speech recognition relies on the use of machine-learning neural networks, the recurrent network model being the most suitable for this purpose. 

We might already be familiar with our smartphone voice assistants, but they rely on the almost infinite computing resources in the cloud to turn speech into command actions. Introducing voice recognition into an automotive environment where reliability is paramount and cellular connectivity is not guaranteed, requires local inference. Considerable processor and inference engine innovations are already underway to enable high-performance, low-power computing at the edge, and these developments will significantly advance in-vehicle capabilities.

More advanced neural networks lead to more comprehensive natural language algorithms that more reliably understand regional dialects and are more adept at removing background noise. In addition to AI, digital signal processing is needed to remove background road, tire, and wind noise. 

Speech recognition is a prudent approach to avoid driver distraction and improve road safety.

Touch Controls

PCAP touchscreens are a mature technology with widespread adoption for automotive applications. Multi-touch touchscreen controller integrated circuits (ICs) are available to manage large screen sizes up to 25 inches; for example, Tesla models use at least a 15-inch portrait touchscreen. 

Fig. 1: Dashboard of a Tesla Model X with a large touchscreen display

LED/LCD screens need to meet the demands of rapidly changing ambient light conditions, from bright sunshine to darkness in seconds. High-brilliance, high-contrast screens are crucial to developing a vehicle’s HMI. Larger screen sizes are preferred over smaller panels, the latter more likely to distract the driver for longer when using. 

The UI/UX design is also critical, since too much information presented is also distracting for the driver. A UX/UI that is structured with frequently used controls always in the same location keeps distraction to a minimum, increasing usability and safety.

Multi-touch controllers are now the norm, with most controller ICs incorporating gesture detection capabilities such as pinch and stretch. Conventional mechanical buttons are also replaced with buttons, wheels, and slider features implemented in software. Audible tones provide user feedback during operation.

The convenience of the touchscreen approach is further enhanced with the use of some innovative features. Haptic sensors, a vibrating or rotating micromechanical element, provide user feedback. Proximity detection is based on capacitive sensing techniques but indicates a finger approaching the LCD panel. It can be used to switch on the screen or to open up a selection menu. A relatively new touchscreen enhancement is the use of a layer of piezo film in the display stack-up to detect the applied pressure or force of touch. Force detection enables a 3D-user experience that can simplify the UI for some applications, or together with haptics, provide a more positive feedback mechanism to emulate the pressing of a mechanical button. 

Haptic sensor development for automotive applications continues to evolve, too, with in-air haptics promising a completely different and potentially safer user experience. In-air haptics integrate hand-tracking machine-learning algorithms with a matrix of ultrasonic speakers. The speakers emit an ultrasonic waveform that can be shaped by the speaker array to create an ultrasonic pattern a few inches above a control surface that the human hand can feel. For example, a circular control knob can be projected, which the fingertips can feel and grasp. The hand-tracking algorithm can then detect specific gestures or movements, such as turning the knob. This innovative virtual touch UI offers excellent potential for a wide variety of touchless applications and is ideal for automotive applications.

Automotive Human Machine Interface Design Challenges

The operating environment of an automotive HMI brings several challenges and technical considerations. Firstly, environmental factors such as extremes of temperature and humidity and rapid changes of both might result in condensation or static. Touchscreens are particularly prone to water droplets forming on the active surface or moisture passed from damp fingers. 

Typically, this causes erratic control actions or the controller IC to stop completely. Dust and dirt can also result in erroneous touch controller activity, but the surface is easily wiped down to remove unwanted particles. Static never mixes well with sensitive electronics, so all aspects of the touch controller and display need to conform to automotive electrostatic discharge (ESD) standards.

All components also need to be automotive qualified in terms of operating temperature up to 85°C and AEC-Q200/Q100 stress resistance.^[3] With ambient light conditions varying greatly, particularly in soft tops, and from moving to bright sunshine to a dark tunnel, the display brightness needs to compensate quickly.

For voice and speech recognition, extraneous noise from other vehicle occupants, high-frequency tire noise, and low-frequency mechanical rumbles need to be filtered out. Wind noise from open windows or rushing sounds emanating from air conditioning or heating vents also needs management. The use of multiple microphones placed in the cockpit area, coupled with digital signal processing, will aid in reducing unwanted audible noises to a minimum.

Recommended reading: Active Noise Cancellation in Electric and Autonomous Vehicles

Ensuring an HMI conforms to relevant technical standards is a key priority throughout the design and  development process. The use of automotive-grade components is essential, and testing against EMI/EMC standards is equally important. Wireless communication is ubiquitous within today’s vehicles, with Wi-Fi, cellular, and Bluetooth protocols being the most popular. The likelihood of interference with the media player, touchscreen, and ADAS functions is high. Also, electric vehicle drive chains generate significant high-frequency switching signals and high dV/dt transients (voltage spikes generated during the rapid switching of power semiconductor devices), the latter potentially causing permanent damage to sensitive electronics if not managed correctly.

A final point on functional safety applies if the HMI is involved in the operation of ADAS functions. The automotive functional safety standard ISO 26262 examines the potential risks of any software-based system that controls a vehicle’s operation.^[4] Risk is assessed based on the potential for harm, the probability that it might occur, and how the system might avoid the risk.

Conclusion

Automotive HMIs have become a crucial part of modern automobiles. The functionality they control and give access to is staggering, everything from in-car media entertainment to wireless connectivity, communication, and navigation. Most utilize a touchscreen as the primary input method. Still, natural language AI-based voice recognition is advancing quickly as high-performance, next-generation application processors and inference engines become available.

This article is based on an e-book by Mouser Electronics. It has been substantially edited by the Wevolver team and Electrical Engineer Ravi Y Rao. It's the eighth article from the Innovations in Electric and Autonomous Vehicles Series. Future articles will introduce readers to some more trends and technologies transforming the automotive sector.

The introductory article presented the different topics covered in the Innovations in Electric and Autonomous Vehicles Series.

The first article discussed the present state of autonomous vehicles.

The second article explains MaaS and how improvements in automotive technologies are speeding up its adoption.

The third article takes a look at how the 3GPP intends to use 5G in V2X applications with significant advantages over current dedicated short-range communication or other Cellular-V2X proposals.

The fourth article explains Driver Monitoring Systems (DMS) and their role in reducing the possibility of mishaps due to human errors

The fifth article examines some of the ways EV technologies will evolve, and some of the obstacles they need to overcome before the automotive industry can transition to fully electric

The sixth article discusses solid-state batteries, a promising alternative to conventional Li-ion batteries

The seventh article explains active noise cancellation and why it’s a very effective solution for eliminating noises in automobiles

The final article focuses on Human Machine Interfaces, the sleek and stylish dashboards that are changing the way we interact with cars

About the sponsor: Mouser Electronics

Mouser Electronics is a worldwide leading authorized distributor of semiconductors and electronic components for over 1,200 manufacturer brands. They specialize in the rapid introduction of new products and technologies for design engineers and buyers. Their extensive product offering includes semiconductors, interconnects, passives, and electromechanical components.

References

[1] ISO 15005:2017, Road vehicles — Ergonomic aspects of transportation and control systems — Dialogue management principles and compliance procedures, ISO, [Online], Available from: https://www.iso.org/standard/69238.html

[2] Stacy Wu, ‘Automotive Display Design & Technology Tracker – 1H22 Analysis’, 06 Sep. 2022, Omdia, [Online], Available from: https://omdia.tech.informa.com/OM024295/Automotive-Display-Design--Technology-Tracker--1H22-Analysis

[3] AEC documents, AEC Council, [Online], Available from: http://www.aecouncil.com/AECDocuments.html

[4] ISO 26262-1:2018, Road vehicles - Functional safety, ISO, [Online], Available from: https://www.iso.org/standard/68383.html