Epona : Autoregressive Diffusion World Model for Autonomous Driving

ICCV 2025

Kaiwen Zhang1,2*, Zhenyu Tang1,3*, Xiaotao Hu1,5, Xingang Pan6,
Xiaoyang Guo1, Yuan Liu5, Jingwei Huang7, Yuan Li3, Qian Zhang1,
Xiao-Xiao Long4✝, Xun Cao4, Wei Yin1§
1Horizon Robotics 2Tsinghua University 3Peking University
4Nanjing University 5The Hong Kong University of Science and Technology
6Nanyang Technological University 7Tencent Hunyuan
*Equal Contribution ✝Project Advisor §Corresponding Author
Paper arXiv Code Hugging Face HuggingFace

Versatile capabilities of Epona : Given historical driving context, our Epona can generate consistent minutes-long future driving scenes at high resolution (A). It can be controlled by diverse trajectories (B), and understand real-world traffic knowledge (C). In addition, our world model can predict future trajectories and serve as an end-to-end real-time motion planner (D).

Abstract

Diffusion models have demonstrated exceptional visual quality in video generation, making them promising for autonomous driving world modeling. However, existing video diffusion-based world models struggle with flexible-length, long-horizon predictions and integrating trajectory planning. This is because conventional video diffusion models rely on global joint distribution modeling of fixed-length frame sequences rather than sequentially constructing localized distributions at each timestep. In this work, we propose Epona, an autoregressive diffusion world model that enables localized spatiotemporal distribution modeling through two key innovations: 1) Decoupled spatiotemporal factorization that separates temporal dynamics modeling from fine-grained future world generation, and 2) Modular trajectory and video prediction that seamlessly integrate motion planning with visual modeling in an end-to-end framework. Our architecture enables high-resolution, long-duration generation while introducing a novel chain-of-forward training strategy to address error accumulation in autoregressive loops. Experimental results demonstrate state-of-the-art performance with 7.4% FVD improvement and minutes longer prediction duration compared to prior works. The learned world model further serves as a real-time motion planner, outperforming strong end-to-end planners on NAVSIM benchmarks.

Method Overview


Our world model utilizes a multimodal spatiotemporal transformer to process the historical context of the first T frames and employs a next-frame prediction DiT to generate the frame at T+1 and a trajectory planning DiT to forecast the future N-frame pose trajectory. By adopting a chain-of-forward strategy, our approach enables high-quality and long-horizon video generation with an autoregressive manner.

Minutes-long Video Gneration

Trajectory-controlled Video Genration

Generalization to Diverse Driving Scenes in China

Note: The model is fintuned on a large in-house dataset of Chinese driving scenes.

End-to-end Trajectory Planning Evaluation


End-to-end motion planning performance on the NuScenes benchmark. Note that our model achieves a low collision rate, demonstrating its understanding of basic traffic rules via simple next-frame prediction. ∗ represents only using the front camera as input.


End-to-end motion planning performance on the NAVSIM benchmark. NC: no at-fault collision. DAC: drivable area compliance. TTC: time-to-collision. Comf.: comfort. EP: ego progress. PDMS: the predictive driver model score. LAW is in the perception-free setting. Our world model outperforms strong end-to-end planners in the overall PDMS score.

BibTeX


      @inproceedings{zhang2025epona,
        author = {Zhang, Kaiwen and Tang, Zhenyu and Hu, Xiaotao and Pan, Xingang and Guo, Xiaoyang and Liu, Yuan and Huang,
        Jingwei and Yuan, Li and Zhang, Qian and Long, Xiao-Xiao and Cao, Xun and Yin, Wei},
        title = {Epona: Autoregressive Diffusion World Model for Autonomous Driving},
        booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
        year = {2025}
      }