Sim to Real
Learn a robot-arm task from demonstrations recorded in Isaac Sim, generate additional examples with synthetic data generation (SDG), and train a policy for execution on a physical robot.
This tutorial follows the demonstration-learning workflow in Robotic Suite/Physical AI. ACT and GR00T N1.5 are available within the same workflow; choose the policy according to your task and deployment target.
What you will build
By the end of this tutorial, you will have a reviewed LeRobot dataset, a trained policy checkpoint, and a model evaluated on a physical Follower arm. If deployment uses a separate host, you will also transfer the selected model and verify it on that host.
Hardware at each stage
| Stage | Physical Leader | Physical Follower | Physical cameras |
|---|---|---|---|
| Simulation recording and SDG | Required | Not used for recording | Not used; images are rendered in simulation |
| Training | Not used | Not used | Not used; training reads the dataset |
| Real-world inference | Not used | Required | Required |
The table describes hardware used during execution. Follow the robot setup wizard for initial connection and identification. Check the system requirements for host support and the hardware preparation guide for stage-specific setup.
Follow the tutorial
| Step | Start here | Completion check |
|---|---|---|
| 1 | System requirements and installation | Robotic Suite is installed and the simulation images and viewer are ready |
| 2 | Hardware preparation | The Leader is configured; virtual and physical motion correspond after offset alignment |
| 3 | Policy selection | The policy is compatible with the intended deployment target |
| 4 | Simulation recording and SDG | The demonstration is reviewed, generated episodes are checked, and a LeRobot dataset is exported |
| 5 | Dataset management | The intended dataset is reviewed and selected for training |
| 6 | Model training | A usable checkpoint is saved and training behavior has been reviewed |
| 7 | Real-world validation | The physical arm is tested on the task and failure cases are recorded |
| 8 | Model deployment | The selected model works on its actual execution host |
Simulation data and physical data
Simulation recording provides a way to generate varied demonstrations. Physical validation is still required because the camera images, contacts, and mechanics of a real setup can differ from the simulated environment.
If needed, follow physical data supplementation to record examples from the real setup, then review and merge compatible data in dataset management. The Sim-to-Real gap section explains the differences to consider.
Walkthrough
The shared product walkthrough shows recording, training, and inference. Use the step table above to follow the simulation-specific route.
For the relationship between the two data routes, see the platform workflow overview. For troubleshooting, see the FAQ.