How Mercedes-Benz is Bringing ChatGPT to In-Cabin Voice Assistants - Alexander Schmitt Mercedes-Benz's Head of Speech Technology

The MBUX (Mercedes-Benz User Experience) Voice Assistant is an infotainment and voice command system designed for Mercedes-Benz vehicles. The car company recently announced it's bringing ChatGPT to MBUX equipped vehicles for its US customers 

"Our software team integrated the ChatGPT-3.5 large language model via a Microsoft API and the beta is available in all 900,000 MBUX-equipped vehicles in the US as of June 16, giving our tech-savvy U.S.customers an opportunity to test it," Alexander Schmitt, Head of Speech Technology at Mercedes-Benz, tells Auto Futures.

"By integrating the ChatGPT large language model, the MBUX Voice Assistant can interact more naturally and adaptively with our customers. When ChatGPT transmits an answer, it is also marked accordingly in our user interface," he adds.

Drivers experience a voice assistant that not only accepts natural voice commands but can also conduct conversations - while, most importantly, keeping their hands on the wheel and eyes on the road.

"Existing functionality of the MBUX Voice Assistant remains as is - e.g. opening windows, starting massages, navigation to certain destinations or asking about the weather. With the voice assistant being backed by ChatGPT, users can enjoy more complex interactions -  e.g. when asking more sophisticated knowledge questions, chat about destinations and travel-related topics."

Mercedes-Benz has received mainly positive feedback from users of the service.

"Mainly in the sense that there is now the capability that now a larger range of questions are answered from the Assistant. Moreover, we are observing a significant increase in usage of the voice assistant in general, also for traditional requests such as navigation or weather queries.So,the ability to interact in a more natural way seems to stimulate users to engage more."

"At Mercedes-Benz, we believe in in house software development with strong partners. Large language model technology will help us to render the MBUX voice assistant more naturally, user-friendly and 'broadly'. Our users may expect to have this technology in their cars soon across all main markets and languages that are supported by our MBUX Voice Assistant," explains Schmitt. 

The Power of The Voice

In recent years,voice assistant technology has become increasingly integral to the automotive industry. One of its most important contributions is to driver safety.

"By enabling hands-free operation of various in-car functionalities, voice-activated systems ensure that drivers can remain focused on the road. The convenience factor is equally compelling: voice commands streamline tasks such as GPS navigation, phone calls, and media selection, eliminating the need for manual interaction and intricate menu navigation," says Schmitt.

Voice assistants are also quickly emerging as a key differentiator for consumers choosing between automotive brands, Schmitt tells us.

"As autonomous driving technologies continue to evolve, we envision voice assistants playing a crucial role in facilitating seamless handover scenarios betweenthe driver and the vehicle's autonomous systems."

Getting Personal

Finally, we took the opportunity to asked Schmitt what in-cabin experiences will be like by 2030, and what role voice activations will play.

"Voice assistants in the automotive industry are set to become an even more integral part of all vehicular systems, extending beyond their current roles in climate, entertainment, and navigation to include advanced safety mechanisms. Leveraging complex AI algorithms, these next-gen voice assistants will have the capability to discern context and mood while also proactively providing recommendations based on real-time information and historical user data," he predicts.

"Microphones throughout the cabin, coupled with profile recognition technology, will enable the voice assistant to identify individual speakers and offer a personalized in-car experience." .

"Benefiting from advancements in large language models, these systems will achieve a more natural, conversational tone, thereby minimizing intrusion and enhancing the user experience. Moreover, tight integration with Advanced Driver-Assistance Systems (ADAS) will enable the voice assistant to issue immediate safety alerts and recommendations," concludes Schmitt.