#UAE #AImodels - The Institute of Foundation Models (IFM), the AI research lab of Abu Dhabi’s Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), has released an open, general-purpose world model that predicts scenarios in the physical world. Users are able to give PAN 2 a scene and a written action, and the model produces video of the outcome, informed by its knowledge of physics, time, and space. According to IFM, PAN 2 ranks third overall on the Physics-IQ Verified leaderboard and first among open-weight models.
SO WHAT? - World models allow AI agents test actions in simulation before they are implemented in the real world, but few such models are released with their full training code. Potential future applications of world models range from robotics and autonomous vehicles, to urban planning and emergency response preparedness, through to healthcare and surgical simulation. PAN 2 has been open-sourced with weights, training code and inference code, so researchers can rebuild and adapt it. Despite its impressive capabilities, the research remains in its early stages and IFM notes that the world model’s long-term world consistency remains unsolved.
KEY POINTS:
The Institute of Foundation Models (IFM), the AI research lab of Abu Dhabi’s Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), has open-sourcedthe PAN 2 world model. The general-purpose world model has been released with open weights, training code and inference code.
Given a starting scene and an action described in words, the new world model can simulate a video of what happens next, grounded in physics, time and space.
IFM calls the approach simulative reasoning. People and AI agents can ask what happens if they take an action, and see the answer first. Agents can then learn and plan without costly, slow or irreversible real-world trials.
PAN 2 works in two stages. A large language model works out what will happen and when, producing a time-stamped description of events, actions and camera movement.
The video diffusion model is trained from scratch by IFM on real-world videos chosen for physical dynamics rather than visual appeal. The model fills in details such as how an ocean wave breaks or how a hand grips a pipette.
PAN 2 builds on the Generative Latent Prediction (GLP) architecture pioneered by IFM founder Eric Xing and IFM researchers. It predicts change in an internal representation, then reconstructs what would be observed, so predictions can be checked against the real world.
PAN 2 placed just behind MiniMax-H3 and ahead of four other models when tested across six core dimensions, including physical consistency, causal coherence and entity persistence. It also completed more full plans than any other model tested in multi-step planning.
PAN 2 ranks third overall on the Physics-IQ Verified leaderboard and first among open-weight models. It can also simulate situations absent from its training data, such as an egg dropped onto a wide range of surfaces.
Agents can use PAN 2 as a world to explore and learn in, such as cooking a pot of pasta in a kitchen they have never seen. IFM says long-term world consistency remains an open challenge.
IFM is releasing the video model weights, full training and inference code, a technical blog and documentation under the Apache 2.0 licence. PAN 2 is available now through IFM’s website, with robotics and education named as possible fields.
ZOOM OUT - PAN 2 follows another world-model release from MBZUAI, the university that established IFM. In October, MBZUAI unveiled WorldGuide, a video world model for procedural tasks such as folding origami, assembling furniture or cooking. It uses its own generated video to pick each next action and decide when to stop. On MBZUAI’s new WorldGuide Bench, it scored 33.33% task success, against 29.90% for MiniMax-H3. MBZUAI says execution and stopping errors remain. MiniMax-H3 also placed just ahead of PAN 2 in IFM’s tests.
[Written and edited with the assistance of AI]
Source: MBZUAI, IFM
LINKS
PAN 2 technical blog (IFM)
PAN 2 model weights (Hugging Face)
PAN 2 training code (GitHub)
PAN 2 inference code (Hugging Face)
PAN 2 inference code (GitHub)
PAN 2 inference code (FastVideo)
Read other MBZUAI / IFM news:
MBZUAI startup open-sources fast open decision model (Middle East AI News)
MBZUAI, Inception42 launch Jais 3 Arabic AI models (Middle East AI News)
MBZUAI builds video AI that plans its own next steps (Middle East AI News)
MBZUAI unveils causal AI for autonomous bio discovery (Middle East AI News)
MBZUAI’s releases world’s largest fully open AI model (Middle East AI News)