Learning to Drive via Real-World Simulation at Scale

Haochen Tian, Tianyu Li, Haochen Liu, Jiazhi Yang, Yihang Qiu, Guang Li, Junli Wang, Yinfeng Gao, Zhang Zhang, Liang Wang, Hangjun Ye, Tieniu Tan, Long Chen, Hongyang Li

• 📧 Primary Contact: Haochen Tian (tianhaochen2023@ia.ac.cn)
• 📜 Materials: 🌐 𝕏 | 📰 Media | 🗂️ Slides | 🪧 Poster | 🎬 CVPR Oral Pre / Talk (in Chinese)
• 🖊️ Joint effort by CASIA, OpenDriveLab at HKU, and Xiaomi EV.

---

🔥 Highlights

• 🏗️ A scalable simulation pipepline that synthesizes diverse and high-fidelity reactive driving scenarios with pseudo-expert demonstrations.
• 🚀 An effective sim-real co-training strategy that improves robustness and generalization synergistically across various end-to-end planners.
• 🔬 A comprehensive recipe that reveals crucial insights into the underlying scaling properties of sim-real learning systems for end-to-end autonomy.

📢 News

• [2026/4/9] 🎉🎉🎉 Selected as CVPR 2026 Oral Presentation!
• [2026/2/21] 🎉 Accepted to CVPR 2026.
• [2026/1/16] We released the data and models on 👾 ModelScope to better serve users in China.
• [2026/1/6] We released the code v1.0.
• [2025/12/31] We released the data and models v1.0 on 🤗 Hugging Face. Happy New Year ! 🎄
• [2025/12/1] We released our paper on arXiv.

📋 TODO List

• More Visualization Results.
• Future Sensors Data.
• Sim-Real Co-training Code release (Jan. 2026).
• Simulation Data release (Dec. 2025).
• Checkpoints release (Dec. 2025).

---

📌 Table of Contents

• 🏛️ Model Zoo
• 🎯 Getting Started
• 📦 Data Preparation
  • Download Dataset
  • Set Up Configuration
• ⚙️ Sim-Real Co-Training
  • Co-Training with Pseudo-Expert
  • Co-Training with Rewards Only
• 🔍 Inference
  • NAVSIM v2 navhard
  • NAVSIM v2 navtest
• ⭐ License and Citation

🏛️ Model Zoo

Model	Backbone	Sim-Real Config	NAVSIM v2 navhard		NAVSIM v2 navtest
			EPDMS	CKPT	EPDMS	CKPT
LTF	ResNet34	w/ pseudo-expert	30.3 | +6.9	HF / MS	84.4 | +2.9	HF / MS
DiffusionDrive	ResNet34	w/ pseudo-expert	32.6 | +5.1	HF / MS	85.9 | +1.7	HF / MS
GTRS-Dense	ResNet34	w/ pseudo-expert	46.1 | +7.8	HF / MS	84.0 | +1.7	HF / MS
		rewards only	46.9 | +8.6	HF / MS	84.6 | +2.3	HF / MS
	V2-99	w/ pseudo-expert	47.7 | +5.8	HF / MS	84.5 | +0.5	HF / MS
		rewards only	48.0 | +6.1	HF / MS	84.8 | +0.8	HF / MS

Note

We fixed a minor error in the simulation process without changing the method, resulting in better performance than the numbers reported in the early arXiv version v2. We have updated the arXiv version v2.

🎯 Getting Started

1. Clone SimScale Repo

git clone https://github.com/OpenDriveLab/SimScale.git
cd SimScale

2. Create Environment

conda env create --name simscale -f environment.yml
conda activate simscale
pip install -e .

📦 Data Preparation

Our released simulation data is based on nuPlan and NAVSIM. We recommend first preparing the real-world data by following the instructions in Download NAVSIM. If you plan to use GTRS, please directly refer Download NAVSIM.

1. Download Dataset

We provide 🤗 Script (Hugging Face) and 👾 Script (ModelScope) (users in China) for downloading the simulation data .

Our simulation data format follows that of OpenScene, with each clip/log has a fixed temporal horizon of 6 seconds at 2 Hz (2 s history + 4 s future), which are stored separately in sensor_blobs_hist and sensor_blobs_fut, respectively. For policy training, sensor_blobs_hist alone is sufficient.

📊 Overview Table of Simulated Synthetic Data

Split / Sim. Round	# Tokens	Logs	Sensors_Hist	Sensors_Fut	Link
Planner-based Pseudo-Expert
reaction_pdm_v1.0-0	65K	9.9GB	569GB	1.2T	HF+ HF_Fut / MS
reaction_pdm_v1.0-1	55K	8.5GB	448GB	964GB	HF+ HF_Fut / MS
reaction_pdm_v1.0-2	46K	6.9GB	402GB	801GB	HF+ HF_Fut / MS
reaction_pdm_v1.0-3	38K	5.6GB	333GB	663GB	HF+ HF_Fut / MS
reaction_pdm_v1.0-4	32K	4.7GB	279GB	554GB	HF+ HF_Fut / MS
Recovery-based Pseudo-Expert
reaction_recovery_v1.0-0	45K	6.8GB	395GB	789GB	HF+ HF_Fut / MS
reaction_recovery_v1.0-1	36K	5.5GB	316GB	631GB	HF+ HF_Fut / MS
reaction_recovery_v1.0-2	28K	4.3GB	244GB	488GB	HF+ HF_Fut / MS
reaction_recovery_v1.0-3	22K	3.3GB	189GB	378GB	HF+ HF_Fut / MS
reaction_recovery_v1.0-4	17K	2.7GB	148GB	296GB	HF+ HF_Fut / MS

Tip

Before downloading, we recommend checking the table above to select the appropriate split and sensor_blobs.

🏭 Simulation Data Pipeline

🧩 Examples of Simulated Synthetic Data

5c9694f15f9c5537	367cfa28901257ee	d37c49db3dcd59fa
Sim. 1 Sim. 2 Sim. 3	Sim. 1 Sim. 2 Sim. 3	Sim. 1 Sim. 2 Sim. 3

2. Set Up Configuration

We provide a Script for moving the download simulation data to create the following structure.

navsim_workspace/
├── simscale/
├── exp/
└── dataset/
    ├── maps/
    ├── navsim_logs/
    │   ├── test/
    │   ├── trainval/
    │   ├── synthetic_reaction_pdm_v1.0-*/
    │   │   ├── [log]-00*.pkl
    │   │   └── ...
    │   └── synthetic_reaction_recovery_v1.0-*/
    ├── sensor_blobs/
    │   ├── test/
    │   ├── trainval/
    │   ├── synthetic_reaction_pdm_v1.0-*/
    │   │   └── [token]-00*/
    │   │       ├── CAM_B0/
    │   │       └── ...
    │   └── synthetic_reaction_recovery_v1.0-*/
    └── navhard_two_stage/

⚙️ Sim-Real Co-Training Recipe

Preparation

1. Refer the Script to cache the real-world and simulation data.
2. Download pretrained image backbone weight, ResNet34 or V2_99.

Co-Training with Pseudo-Expert

We provide Scripts for sim–real co-training, e.g., run_diffusiondrive_training_syn.sh.

The main configuration options are as follows:

export SYN_IDX=0   # 0, 1, 2, 3, 4
export SYN_GT=pdm  # pdm, recovery

• SYN_IDX specifies which rounds of simulation data are included; e.g., SYN_IDX=2 means that rounds 0, 1, and 2 will be used.
• SYN_GT specifies the type of pseudo-expert used for supervision.

In addition, the cache path for simulation data is hard-coded in dataset.py#136. Please make sure the path is correctly set to your local simulation data directory before training.

• Regression-based Policy | LTF

We provide a Script to train LTF with 8 GPUs for 100 epochs.

• Diffusion-based Policy | DiffusionDrive

We provide a Script to train DiffusionDrive with 8 GPUs for 100 epochs.

• Scoring-based Policy | GTRS-Dense

We provide a Script to train GTRS_Dense on 4 nodes, each with 8 GPUs, for 50 epochs.

We also provide 🤗 Reward Files (Hugging Face) and 👾 Reward Files (ModelScope) (users in China) for rewards in simulation data. Please download correspending files first and move them to NAVSIM_TRAJPDM_ROOT/sim. The reward files path is hard-coded in gtrs_agent.py#223. Check it before training.

Co-Training with Rewards Only

• Scoring-based Policy | GTRS-Dense

It uses the same training Script, to train GTRS_Dense on 4 nodes, each with 8 GPUs, for 50 epochs.

The main configuration option is as follows:

syn_imi=false  # true, false

• syn_imi: When set to false, the imitation learning loss is disabled for simulation data, while it remains enabled for real-world data.

🔍 Inference

Preparation

Refer the Script to cache metric first.

NAVSIM v2 navhard

We provide Scripts to evaluate three policies on navhard using GPU inference.

NAVSIM v2 navtest

We provide Scripts to evaluate three policies on navtest using GPU inference.

❤️ Acknowledgements

We acknowledge all the open-source contributors for the following projects to make this work possible:

• NAVSIM | MTGS | GTRS | DiffusionDrive

⭐ License and Citation

All content in this repository is under the Apache-2.0 license. The released data is based on nuPlan and is under the CC-BY-NC-SA 4.0 license.

If any parts of our paper and code help your research, please consider citing us and giving a star to our repository.

@inproceedings{tian2026simscale,
  title={SimScale: Learning to Drive via Real-World Simulation at Scale},
  author={Tian, Haochen and Li, Tianyu and Liu, Haochen and Yang, Jiazhi and Qiu, Yihang and Li, Guang and Wang, Junli and Gao, Yinfeng and Zhang, Zhang and Wang, Liang and Ye, Hangjun and Chen, Long and Li, Hongyang},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2026}

Star History