World Models Move From Research Labs Toward Everyday Robots and Cars

World models are drawing serious attention as the next big direction in artificial intelligence, and they may soon affect daily life even for people who never use them directly. Unlike language models that predict the next word in a sentence, world models take in visual information and predict the next visual state of the world, generating 3D environments that simulate physics.

Brian McClendon, chief technology officer at Niantic Spatial, said the company is focused on reconstructing the real world and hopes to use its store of world data to build replicas. Oliver Cameron, CEO of the AI lab Odyssey, described how these 3D generations work as digital training grounds that look and feel real.

The technology has clear applications for robotics and driverless cars. McClendon said someone could photograph a space where a robot is meant to operate, generate a 3D model, and then run millions of simulations to train the robot on that space without physical movement. Jamie Shotton, chief scientist at Wayve, said a world model must understand physics, cause and effect, and how other objects and people respond to our actions. Wayve's generative world model is powered in part by dashcam footage and is used to recreate hazardous roads and unsafe conditions that would be impractical to test in real life.

Video games are another area of interest, with major players like Tencent investing in their own world models. Building these systems also requires real-world data, creating gig work for people who record everyday activities for AI labs.

Source: Wired gear