Google DeepMind Unveils Genie 4 World Model With Audio and Live Steering

Google DeepMind released Genie 4, its newest world model, in research preview on Tuesday. The model generates photorealistic, interactive worlds and lets users type prompts mid-simulation to change the scene. It also produces audio matched to the simulated world, including music, ambient noise and interactive dialogue.

Genie 4 follows Genie 3, released as a research preview a little over a year ago, and arrives a couple of months after AI video startup Runway launched its second world model, GWM Worlds 2, a playable model that responds to user inputs and includes audio generation.

Beyond sharper visuals and sound, Genie 4 supports free-form edits such as changing the weather or camera angle live, or adding people and objects. It holds a consistent scene for up to two minutes, double its predecessor's limit, with better physics and more humanlike characters. Gemini is more tightly integrated too, planning steps and checking frames while Genie creates the visuals. In a research experiment tied to Gemini Robotics, a Gemini 4 Argon model broke tasks into steps that Genie 4 simulated live, suggesting a way to test robot scenarios in simulation before the real world.

Tejas Kulkarni, a research director at Google, said during a briefing that the future will bring "hybrid world models, where you have a combination of Gemini writing code and text, and Genie ingesting that in real time and simulating reality at different levels of abstraction."

Pitched commercial uses include gaming, interactive learning and robotics. Trusted testers are pointing to another: prototyping films. "The real time aspect becomes most valuable in the areas where keeping a state of flow in the creativity space becomes incredibly useful," engineer Diego Rivas said, citing ideation and storyboarding. Google also offers Veo, its AI video-generation model for high-definition clips with synchronized audio, which has been gaining traction among filmmakers. Rivas said users could eventually turn prototypes into creative assets with Gemini Omni, a multimodal model that generates video from images, audio and text, though he said those capabilities are still a ways off.

Rivas stressed it is early days for Genie 4 and the end-to-end product experience is not yet clear. The team acknowledged gaps: details and objects can drift as simulations run longer, and ensuring accurate audio and legible in-world text remains unresolved.

Genie 4 is available now as a research preview for trusted testers. In coming weeks it will roll out to Google AI Ultra subscribers in Project Genie, Google's tool for generating interactive worlds from text or image prompts, and to early access partners of the Gemini API and Google AI Studio.

Source: TechCrunch