Initially announced in 2021, Meta CEO Mark Zuckerberg shared an update on the company’s research partnership with BMW, which focuses on integrating AR and VR into vehicles to make people more productive, social, and entertained while traveling.
The ultimate aim in the BMW/Meta partnership is to accurately anchor virtual objects relative to the car’s motion by hooking into the tracking system of both the car and a Meta headset, which researchers say includes the Meta Quest Pro standalone mixed reality headset and the company’s in-development AR headset, Project Aria.
Without such a system in place, the headset’s rotational tracking would noticeably drift as the car makes turns and other adjustments, making it essentially unusable for anything but perfectly straight sections of road.
Check out the video detailing the research below:
Still considered a proof-of-concept prototype, Meta says the partnership has already overcome some key technical challenges, such as fusing the headset and car’s sensors to understand their relative position. That said, the companies don’t think it’s ready for the public just yet.
“It is too early to tell exactly how or when this technology will make it into customers’ hands, but we envision a number of potential use cases for XR devices in vehicles—from assisting the driver in locating their car in a crowded parking lot to alerting them to hazards on the road and surfacing important information about the vehicle’s condition,” said Claus Dorrer, Head of BMW Group Technology Office in the US. “The implications of future AR glasses and VR devices—for passengers as well as drivers—are promising. The research partnership with Meta will allow us to discover what immersive, in-vehicle XR experiences could look like in the future and spearhead the seamless integration of such devices into cars.”
AR and VR integration in cars isn’t an entirely new area of research. It’s been the sole focus of Audi-backed startup Holoride, which recently partnered with HTC to deliver in-car VR entertainment via HTC Vive Flow. Still, Holoride has been mostly grabbed headlines as a tradeshow mainstay; it hasn’t seen mass adoption yet despite only requiring a $200 retrofit pack, which enables Vive Flow owners to play VR in cars.
In the end, it seems car companies are now seeing the writing on the wall that riders will maybe very soon—but not right now—want to bring their own XR devices and actually use them in the car, just like you might a smartphone, albeit with more utility than any infotainment screen on offer.