The MIH Consortium believes that in the future, cars are not just a means of transportation but a second living space that connects drivers and passengers to food, clothing, housing, transportation, and entertainment. In the era of traditional gas-powered vehicles, people care about the power. Since the global shift toward electric power, the automotive industry has emphasized technology, particularly car cockpit has been transformed by introducing features like multiple touch display screens, gesture control, voice interaction, fatigue detection, and more. In August MIH Insights, Ted Lien, MIH technical consultant, dives into the smart cockpit trend and provides updates on the working group's progress.  There are four major trends of smart cockpit development: vision, hearing, intelligent interaction, and rear seat control system.  

• Vision: you can see various applications, such as large display screens (with high resolution, high computing power, and high bandwidth), AR navigation, naked-eye 3D instruments, transparent window displays (for advertising), etc. The automotive industry has invested more in developing camera-based Driver Monitor System (DMS) and Interior Monitoring System (IMS) to provide more advanced features to consumers. 

• Hearing: the auditory modality represents an obvious alternative to the vision for interacting with in-vehicle technologies. The common features from voice control to Artificial Intelligence (AI) voice interaction.  

• Intelligent Interaction: we have since progressed to "multi-modal interaction" which combines multiple senses—for instance, hearing and vision. A driver can ask a car to provide a detailed description of the external roadside environment. Intelligent Interaction will "listen" to the driver's question (hearing) and "see" the driver's point of view (vision) to judge and provide a tailored driving experience. 

• Rear Seat Control System: presently, rear seat entertainment system is one-way communication, such as display advertisements in a taxi or movies for kids on a road trip. In the future, rear-seat passengers may control the operation of related applications. 

Develop MIH Standards and Realize Multimodal Interaction

People expect electric vehicles to be the bridge between life and technology. To improve user experience, automotive OEMs are racing for the implementation of innovative smart cockpit technologies to pursue personalized customer experiences. However, this business model will lead to the increase of development costs for the supply chain and Tier 1. This is a drawback of the popularization and evolution of electric vehicles. 

MIH hopes to solve the challenges mentioned above. The goal of the Smart Cabin Working Group is to standardize the interface. Traditionally incompatible components, e.g., computing units or the test chassis due to manufacturer differences, will become compatible through the standard interface developed by MIH and MIH members. The hardware interface means cables and connectors, and the software interface refers to protocols and data format. The interfaces related to the cockpit need to be standardized too. 

The process of interaction will inevitably involve the interface. For example, Driver Fatigue Detection System monitors drivers via cameras and sensors. When a driver dozes off, in addition to issuing a warning, DMS can transmit the information to the In-Vehicle Infotainment (IVI) System. Then, the IVI will play music to relieve passenger fatigue.   At present, each car brand has its proprietary system for interaction between DMS and IVI. MIH's mission is to cooperate with partners to standardize the interface for manufacturing efficiency; if the DMS of manufacturer A is out of stock or there is a chip shortage, OEM and Tier 1 can quickly switch to the DMS of manufacturer B. 

Support Mainstream In-vehicle OS and Chip Solutions  

MIH's goal is to help developers reduce development time and cost through three aspects:  

1. To establish standardized interfaces between the five senses' sensors and systems. 

2. Develop agnostic hardware and software. Achieve interoperability with mainstream in-vehicle operating systems: Android, AGL, and QNX. And the agnostic hardware can be compatible with leading automotive chip vendors, such as NXP, Renesas, TI, and more. 

3. Ensure investment for software developers in anticipation of a "software-defined automobile" future. Many in the auto industry have developed software using various programming languages: Java, C++, Python, etc. MIH realizes compatibility through APIs and supports all programming languages. Software companies can reuse their codes. This is very different from other automotive architectures that limit the use of specific programming languages. MIH can help software developers quickly realize their ideas while ensuring their previous software investment, and accelerating industrial innovation.  

More than twenty companies have joined the MIH Smart Cabin Working Group. We have Clientron as the Working Group Chair and have been working on several projects with TomTom, Cerence, etc. We will announce our development results on Demo Day in November. 

The Smart Cabin Working Group still has a lot of work. Besides vision, we hope to work with more partners who develop other sensor technologies. Another goal is to develop communication interfaces between the digital cockpit and the vehicle's Electronic Electric Architecture (EEA). MIH plans to launch an EEA working group in the future. The collaboration will define and synchronize the specifications of the system interface.  

Ted Lien, MIH Autonomous Driving Consultant A veteran in open source maker community with enthusiasm in entrepreneurship. He serves as the Senior Manager of MIH CTO Office and oversees the development and application of Robotics and Autonomous Driving System. He wrote more than 10 technical books and also serves as an Adjunct Assistant Professor at CGU Taiwan.