NTT Develops Physical AI Technology That Predicts the Future Before Robots Move to Prevent Failures
Physics-based world model predicts load collapse and object drops, advancing automated loading and unloading of diverse items such as airport baggage

Highlights

Figure 1. Position of the technology within physical AI. The technology works with existing robot control programs to predict and evaluate changes in the surrounding environment before a robot takes action.

Figure 2. Overview of action selection using the technology. The system predicts and evaluates the outcome of each candidate action and selects an action likely to succeed.

Background

World models that predict future changes in state resulting from a robot's actions are attracting attention in physical AI, which enables machines such as robots to perceive, decide, and act autonomously in the physical world. Conventional world models, however, do not explicitly represent the physical quantities of objects internally. As a result, objects may disappear or deform in generated video, and interactions such as collisions may not be predicted correctly. Their dependence on large volumes of training data also makes predictions more likely to fail in situations not represented in the training data.

These limitations are especially evident when loading and unloading items with diverse shapes, weights, and arrangements, including airport baggage and small parcels for delivery. Because the shapes and weights of these items vary, conventional robot control alone cannot readily determine in advance whether an action will cause a load to collapse or an item to fall. Collecting training data that covers every possible item is also impractical.

NTT has developed a proprietary world model that can address previously unseen situations by explicitly representing the physical quantities of objects and predicting changes in state based on physical laws.

Key Features of the NTT World Model

Feature 1: Works with existing robot control systems to support decisions before action

The technology can be used with existing robot control software to supplement its decision-making. It is also designed to operate at high speed while maintaining accuracy so that it can be incorporated directly into robot control flows. For each candidate action generated by a VLA⁴, motion planner⁵, or other system, the model predicts the future outcome and provides an evaluation of the action plan, including whether a load will collapse or an object will fall (Figure 3).

Figure 3. Overview of the NTT world model. The model predicts the future for each robot action plan and evaluates whether it will succeed before execution.

Feature 2: Compares multiple futures and selects an action likely to succeed

For each candidate action generated by a robot control program, the technology generates a video of the predicted outcome. It then predicts and evaluates results such as object drops and collisions, enabling the robot to select an action likely to succeed and reducing action failures (Figure 4).

Figure 4. Prediction and evaluation of multiple candidate actions by the NTT world model. The model generates a future video for each candidate and scores how likely it is to result in success.

Feature 3: Highly accurate future prediction based on physical laws

The model explicitly represents physical quantities for each object, including position, orientation, velocity, and mass, and predicts future motion in accordance with equations of motion. This reduces artifacts such as implausible object disappearance and physically implausible deformation and enables more accurate prediction of interactions between objects, such as collisions and load collapse (Figure 5).

Figure 5. Comparison of future prediction results on the public OBJ3D⁶ benchmark. In scenes involving object collisions, the conventional model fails to predict the collision (red boxes), while the NTT technology succeeds (green boxes).

Feature 4: Supports previously unseen situations not included in training data

By explicitly representing physical quantities and predicting changes in state based on physical laws, the model can address situations and motions not included in the training data (Figure 6). It is suited to environments where items vary in shape, weight, and arrangement, such as airport baggage and small parcels for delivery, and where comprehensive training data is difficult to obtain.

Figure 6. Prediction results for a scene not included in the training data, in which objects are occluded by the robot arm.

Technology Overview

Development of a physics-based "physics control adapter" and integration into a video generation model

NTT developed a "physics control adapter" that combines a mechanism for predicting the future from video with calculations based on physical laws. The adapter estimates the position, orientation, velocity, mass, and other properties of each object from video, and then calculates how the object will move next based on physical laws. NTT also became the first in the world to integrate the physics control adapter into the video generation model that serves as the foundation of the world model. By incorporating the adapter's calculation results into video generation, the model reduces object disappearance and deformation and more accurately predicts interactions between objects, including collisions and load collapse (Figure 7).

The technology offers four key advantages:

Figure 7. Architecture of the future prediction model at the core of the technology. The physics control adapter is integrated into the video generation model, and its estimates are reflected in video generation to produce highly accurate future video.

Experimental Results

NTT conducted a shelf-loading task using a small robot to simulate the stacking of items and compared task success rates with and without the technology. Applying the technology improved the success rate from 73.1% to 92.3%.

NTT also evaluated predicted video on the public OBJ3D⁶ benchmark, which includes collisions among multiple objects, to assess future prediction accuracy. The NTT world model outperformed conventional methods on LPIPS⁷, a metric that measures similarity in a manner aligned with human perception (Table 1). Lower LPIPS values indicate greater similarity.

Table 1. Accuracy of predicted video on the public OBJ3D benchmark.

Future Development

The first envisioned application is the automated loading and unloading of airport baggage with varying shapes and weights. NTT plans to advance implementation in collaboration with Mujin, Inc. and will also consider working with robot service providers and logistics and warehousing companies.

Building on baggage and parcel loading and unloading, NTT plans to expand the technology to a broad range of fields that handle diverse objects, including manufacturing, construction, and retail, helping address labor shortages and improve productivity.

The technology is scheduled to be exhibited at NTT R&D Forum 2026 in November 2026.
https://www.rd.ntt/e/forum/2026/

Notes

¹World model
A mechanism that models the state of and changes in the external environment and predicts future changes in state caused by its own actions. In this technology, the model generates video showing how the surrounding environment will change after an action, based on candidate robot actions and camera and sensor information, and scores the action plan before execution.

²Video generation model
A deep learning model trained on large volumes of data to generate video. It is used as the foundation of general world models.

³Physical AI
AI that enables machines such as robots to perceive the physical world through cameras and sensors and to make decisions and act autonomously.

⁴VLA (Vision-Language-Action) model
A foundation model that takes camera video and language instructions as input and directly outputs robot actions.

⁵Motion planner
A program that calculates robot controls needed to achieve a task objective based on the robot's goal and information about the surrounding environment.

⁶OBJ3D
A public benchmark for video prediction consisting of videos that include interactions such as collisions among multiple objects.

⁷LPIPS (Learned Perceptual Image Patch Similarity)
A metric that measures the similarity between ground-truth and predicted video in a manner aligned with human perception.

About NTT

NTT is a leading global technology innovator, providing a broad range of services to both consumers and businesses. As a mobile operator and provider of infrastructure, networks, and services, NTT is dedicated to promoting a sustainable future through cutting-edge innovations. Our portfolio includes business consulting, AI-powered solutions, application services, global networks, cybersecurity, data center and edge computing, all supported by our deep global industry expertise. Generating over $90 billion in revenue and employing 340,000 professionals, we allocate 30% of our annual profits to fundamental research and development. With operations spanning more than 70 countries and regions, our clients include over 75% of Fortune Global 100 companies, alongside thousands of enterprises, government organizations, and millions of consumers.