Imagine being able to use Google Maps Street View in a more interactive way, simulating the environment and adjusting conditions like weather or even imagining a disaster scenario. This is the goal of Google’s latest integration. Google DeepMind is now connecting Street View to Project Genie, a platform that creates diverse, interactive environments. This feature was announced at the Google I/O developer conference.
Jack Parker-Holder, a research scientist on DeepMind’s team, explained its potential, such as helping a robot in London adjust to rare sunny conditions. You can also preview different weather settings for a trip, like seeing a snowy New York City block.
Google has spent two decades collecting data for Street View, encompassing over 280 billion images from 110 countries and all continents. Jack highlighted that this integration of real-world data with world simulation is powerful.
Genie 3, released last August and available to Google AI Ultra subscribers, allows users to create interactive worlds, useful for games, education, and robotics. It’s also used by Waymo to train self-driving cars in rare scenarios. Integrating Street View data further enhances preparation for Waymo’s expansion.
Street View in Genie is launching for some Ultra users in the U.S., with broader access planned. Diego Rivas from DeepMind noted that this feature is still experimental, with room for improvement in accuracy.
Current simulations resemble video game quality but lack some realistic physical interactions. Parker-Holder suggests it might take six to twelve months to catch up with video models in terms of precision. Jonathan Herbert from Google Maps noted that while Genie can’t yet fully reconstruct streets, it offers critical spatial continuity, enabling new environment simulations based on Street View data.