On September 1st, Fei-Fei Li's World Labs released Atlas.
The release page featured a set of impressive visuals. Several phones were mounted on tripods and fixtures, capturing the same scene from three to five angles. Time froze, the camera started to move, circling around the characters, reaching positions never occupied by a real camera before.
Bullet time, this technique has not been uncommon in the past two years. The difference lies in the additional three things Atlas simultaneously delivers.
The first is depth. Assigning to each pixel in the image its distance from the camera, a regular photo transforms into a color-coded topographic map.
The second is point cloud. Sprinkling millions of points in space, each point remembers its three-dimensional coordinates. Put together, they form the shape of this room.
The third is 3D Gaussian Splat. Instead of constructing models with triangles, it piles up a large number of small spots with color, transparency, orientation, and size variance, allowing the entire scene to be re-rendered from any angle.
The common feature of these three things is measurability. The diameter of a cup, the distance from a table to the wall, all can be directly read from them. The generated results are no longer just visual images for viewing but can be input into game engines, design software, or a control code for a robotic arm.
World Labs boldly refers to Atlas as a world simulator.
They do not use the word "simulator" lightly. Three months ago, they wrote an article themselves, specifically defining what kind of system deserves this name.
Visible and Touchable
Published in June of this year, the article discussed what constitutes a world model, concluding by categorizing existing systems into three levels: renderer, simulator, planner.
The origin of this classification method is quite old. In 1943, British psychologist Kenneth Craik made a point that humans can anticipate how things will develop because there is a miniature version of the real world in their minds. This notion was later taken over by cybernetics, neural networks, and reinforcement learning, evolving into three terms: state, observation, action.
Take a kitchen, for example.
The placement of a cup on the table, the distance from the edge of the table, how far the table is from the wall, whether the door is open or closed—all these together make up the state of the kitchen at this moment.
A person standing at the door can only see a part of the state. He cannot see the part of the floor blocked by the refrigerator, nor does he know how much water is left in the cup. The information his eyes take in is called an observation.
He walks over and pushes the cup—that's an action.
State, observation, action—how many aspects of a system are under control determines its level.
A renderer handles visuals. When a person takes a step, it calculates what should be seen in that step. If it looks right, matches the previous frame, the job is done. It doesn't need to know how tall the cup is or how far the table is from the wall because all it produces is an image.
A simulator handles the state. The cup's position, the table's dimensions, how wide the door can open before hitting the wall—these numbers need to be constantly stored and readily available. It is because these numbers are stored that other programs can come in, and a robotic arm can reach into this kitchen.
A planner handles actions. It not only knows that the cup is on the table and what will happen if it falls but also needs to decide whether to reach now, which one to take first, and from which angle to approach for the most stability.
By this standard, Google DeepMind's Genie 3 is classified at the lowest level, just a renderer. Also included in this level is RTFM, released by World Labs in October 2025.
Before this, the term "world model" could encompass almost anything. Generating a scene that one can step into from a sentence is a world model. Changing a company's slogan to using AI to simulate the world is also a world model. This article draws a clear line for the first time: visuals are visuals, the world is the world.
Most of the demonstrations over the past year have mostly focused on the first level. They have indeed made significant progress compared to simply generating videos from text. Press a directional key, and the scene changes accordingly.
However, what the renderer produces is enough for humans but not usable by machines.
Atlas needs to meet the level of a simulator. According to their own definition, the simulator must maintain the entire state of this world, including physics.
The publicly disclosed test sets focus on camera condition generation and 3D reconstruction. Challenges like complex materials, fluids, and cloth interactions have not been resolved, neither by the whole industry.
The technical name of Atlas is multimodal autoregressive diffusion transformer, with each of the four words not being particularly new.
Transformer and autoregressive belong to the realm of large language models, where the generated content continues unfolding one after another. Diffusion pertains to the domain of image and video generation, responsible for rendering the visual content. Multimodal indicates that it simultaneously takes in several types of input.
What's novel about the multimodal aspect is that it specifically encompasses four types: text, images, camera poses, and depth maps.
Camera pose refers to where the camera was positioned in three-dimensional space and which direction it was pointing when the photo was taken, described by six numbers. When combined with a depth map, each photo is no longer just a standalone image but one that is annotated with the shooting location.
All photos are annotated on the same coordinate system, which is the shared spatial context of Atlas.
For instance, taking a photo of a sofa from the living room doorway and then moving to the window to take another photo. The two photos show the sofa at different sizes, angles, and partially obstructed by a coffee table. A conventional model would see two distinct images. A person would see the same sofa because they know how many steps they took and how much they turned.
What Atlas receives is precisely this "how many steps taken and how much turned" data.
This data serves another purpose, as demonstrated by the previous generation product RTFM from World Labs. Its aim was to dynamically generate a world that one could keep exploring in real-time on a single H100 board. As one moves forward, the model generates further scenes, adjusting the viewing angle accordingly. The challenge lies in memory.
Consider someone wandering around this virtual house for half an hour. If all the images generated in that half-hour are crammed into the context, the model indeed remembers everything, but as the context grows longer, computational resources will be the first to give in. If only the most recent few frames are retained, computational resources are conserved. However, if one walks from the bedroom to the kitchen, the model no longer remembers where the table was placed twenty minutes ago, causing the same kitchen to be rendered differently.
World Labs referred to the approach it gave to RTFM as posed frames as spatial memory. Besides the visual content, each frame also retains the position and orientation of the camera in space at the time it was captured. When the model reaches a specific location, the system doesn't need to revisit all memories; it merely retrieves a few frames based on proximity to reconstruct the necessary context for that particular generation.
The key here is location-based indexing. If you want to know what the kitchen looks like, just find frames taken near the kitchen; when the frames were taken is not important.
Human memory works in a similar way. Trying to recall what you did last Wednesday afternoon is usually challenging, but walking into a meeting room triggers memories—remembering who sat across from you and what was written on the whiteboard happens quickly. Searching by time requires counting from the beginning, while searching by location gets you there instantly.
Reconstruction and Generation
Atlas' release page also features an experiment. Initially, only a photo of a garden is provided, and the model is tasked with generating a top-down view. Since the photo only shows the garden and nothing around it, the garden area is accurately reconstructed, with the surrounding buildings extrapolated based on the model's trained knowledge.
The second photo captures a neighboring cottage, so there's no need for extrapolation in that area. In the third photo, more of the house is visible, requiring less extrapolation.
World Labs' conclusion is that the more it sees, the less it imagines. According to their data, two or three photos are usually sufficient to recreate a scene to a usable level, with the model capable of processing over a hundred at once.
This experiment combines two initially distinct processes into a single model.
To create a 3D scene on a computer, there used to be two main approaches. One was reconstruction, where photos taken on-site are used to faithfully recreate an existing place—if a corner wasn't photographed, it remains empty, preferring a hole to an assumption. The other was generation, where the model, drawing from its training on houses and lawns, constructs a place that never existed.
Atlas covers both ends of the spectrum. Missing areas are initially filled in based on common sense, and as new photos are added, the generated part is replaced with reality. If there are few photos, more generation is required; if there are many, less is needed.
I've always been a fan of a project called Funes World. They travel the world with a camera, capturing buildings—Pompeii, Troy, Cologne Cathedral, Temple of Heaven—as well as those nondescript houses that, once renovated one day, no one remembers how they used to look. They walk around the buildings, taking photos from all angles with enough overlap between photos, then reconstructing 3D models using photogrammetry, NeRF, and Gaussian Splatting. Their public archive now contains over 2,100 models, spanning 19 or more countries.
They claim to be creating a backup of the real world, building a GitHub for the physical world.
Funes took the path of pure reconstruction. They took the picture before the bulldozer arrived, aiming for the house to actually grow that way. Even if a piece was missing, the model couldn't stand in for it.
Atlas faced a different kind of user. In a kitchen, as long as the wall's position and size match, the result of the robotic arm reaching out will be the same, whether the wall's texture was captured or generated, the robotic arm doesn't care.
So the question shifted elsewhere. In a world partially built on generation, could you use it to train a machine meant to work in reality? As for whether the generated house looks real or not, that could be put aside for now.
In July, World Labs acquired the robotics company SceniX. The first set of results released after the acquisition addressed precisely this question.
Several groups of robots were trained entirely in a simulator, without using any real-world data, and then directly installed onto real robotic arms to perform tasks such as packing, wire routing, and test tube handling. Some of them operated autonomously for a continuous hour without any human intervention.
What's even more interesting is how they determined whether this simulator was qualified or not.
There was a seemingly natural standard — if the success rate in the simulator is 80%, it should be 80% on the real machine too. This standard was not used. Friction between the walls in the simulator and real walls was different, and the weight distribution of the parts did not match, so the success rate was inevitably lower.
They changed their approach. They first ran several scenarios in the simulator 2000 times each, ranked them based on performance, then took these scenarios to the real machine to run 100 times each, focusing on just two things: whether Scenario A, which performed better in the simulator than Scenario B, would also perform better on the real machine, and whether an action identified by the simulator as most prone to failure at a certain step would indeed fail at that step on the real machine.
In the tasks they disclosed, both of these matched up. As for how much the success rates differed between the two sides, it wasn't a concern. According to World Labs themselves, a useful simulator doesn't need to match the real-world success rate, it just needs to support the same decision-making as reality.
This method was used over a hundred years ago.
In the autumn of 1901, the Wright brothers, using Lilienthal's published wing data, built a glider with actual lift only a third of the predicted value. The predecessors' data was flawed, but to retest it would require sending people up into the air repeatedly. So, they made a six-foot-long wooden wind tunnel, where they could test several sets of wings in an afternoon and tried over 100 variations that winter.
The breeze in the wind tunnel is not the same as the sea breeze in North Carolina, but which wing is better can be compared.
A world model does not need to replicate the entire universe. It only needs to retain the part of the structure relevant to decision-making.
A Failed Experience
At this point, World Labs' business profile emerged. Their product Marble, launched in 2025, is able to generate an explorable 3D world from text, images, and videos. Once the World API is open, developers can directly access it, with Atlas being the underlying model of these products.
The $1 billion financing round in February this year had a diverse group of companies investing. NVIDIA is after Physical AI, Autodesk seeks applications in architecture, industrial design, and digital twins, and there are also AMD, Fidelity, and Sea. In the same 3D world, directors care about camera angles and lighting, architects focus on structure and scale, and robots need an environment for experimentation.
As long as a world is generic enough, it can span several software industries that are usually separate today.
It is now meaningless to squeeze companies claiming to be working on world models into the same benchmark chart.
Genie 3 is focused on image generation, Meta's V-JEPA 2 doesn't even create images but learns patterns directly from videos, NVIDIA's Cosmos integrates visual reasoning, world generation, and motion prediction into a single model, and Runway has transitioned from video generation to simulation. They use the same terminology but tackle different problems.
There is only one main difference. One camp believes that by predicting the world well enough, the 3D structure will emerge from the data itself, as these patterns are inherently present in videos. The other camp directly incorporates camera positions, geometry, and physical constraints into the model, enabling the world to have structure from day one. Atlas makes camera poses and depth native inputs; that's how it's done.
One thing is unavoidable on this path: where does the training data come from.
A large language model once received a gift. Over the decades, humans have been writing web pages, code, and arguing on forums, unintentionally amassing a huge corpus.
The 3D world doesn't have this. Spatial relationships are rarely systematically recorded, and the friction coefficient of a floor won't be written into a webpage on its own. Even more challenging is interaction. If a robot misplaces a wire, in reality, the wire is indeed misplaced; before the next attempt, someone has to walk over and correct it. Text can be endlessly replicated, but real-world experiences have to happen anew each time.
So what robots lacked most in the past was actually failure. In reality, making mistakes is too costly, but once the simulator is good enough, the robot can run in it thousands or tens of thousands of times, rapidly gaining experience.
Fei-Fei Li has been dealing with this issue for almost 20 years.
In 2009, she and her team created ImageNet, reorganizing the images scattered on the Internet according to the WordNet concept hierarchy. The complete index contains 14.19 million images and 21,841 synsets. The later ILSVRC that ignited deep learning only used 1000 classes from it. The tasks in that generation of computer vision were straightforward: give a machine a photo and let it say what's in it, which category it belongs to, and where in the picture.
Seventeen years have passed, and that question is no longer a problem. The cups in the kitchen in front can not only be recognized by today's models but can also magically generate 100 cups that have never existed.
The challenge now is not just recognizing the cup but also knowing where it is in the room, circling around to see if it's still the same cup when viewed from behind the table, predicting where it should be after someone has moved it, and what state it is in. Recognizing something and knowing how that thing exists in the world are two different things.
From ImageNet to Atlas, there is an unexpected thread: how much information about the world should humans cram into the machine for it to develop its own world.
In 1940, Argentinian writer Adolfo Bioy Casares wrote a novel called "The Invention of Morel."
The protagonist escapes to a deserted island and encounters a group of strange people. These people walk, chat, and dance every day, everything seems normal, but no one can see him. What's even weirder is that the same conversation will happen exactly the same again after a few days, not even missing a pause.
Later, he learned that the scientist Morel on the island had built a machine that recorded a complete week of these people on the island. The sound was recorded, the image was recorded, and Morel believed that even touch, smell, and temperature were also preserved. The machine was tidal-powered, and that week on the island played in a loop.
Morel believed that when all senses are synchronized, the soul appears.
This machine written over eighty years ago now resembles an exaggeratedly absurd multimodal model. It preserves all sensory evidence of reality, creating a replica that no observer can distinguish between true and false.
The protagonist fell in love with the woman in the projection, Faustine. He stood in front of her every day, trying to talk to her and make her see him. She never responded because her week in that projection had already been recorded. The machine could replay that week ten thousand times but could not produce something that did not happen at the time.
According to World Labs' own chart, Morel had created a perfect renderer.
Today's models can recreate a place flawlessly. The challenge is something that was never captured by anyone - someone walking in, reaching out, and knocking over the cup on the table.
Morel's island has everything except for that one moment.
-****END-
Welcome to join the official BlockBeats community:
Telegram Subscription Group: https://t.me/theblockbeats
Telegram Discussion Group: https://t.me/BlockBeats_App
Official Twitter Account: https://twitter.com/BlockBeatsAsia
