English | 简体中文

LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequences

Ao Liang     Youquan Liu     Yu Yang     Dongyue Lu     Linfeng Li
Lingdong Kong     Huaici Zhao     Wei Tsang Ooi

     

In this work, we introduce LiDARCrafter, a unified framework for 4D LiDAR generation and editing. We contribute:

• The first 4D generative world model dedicated to LiDAR data, with superior controllability and spatiotemporal consistency.
• We introduce a tri-branch 4D layout conditioned pipeline that turns language into an editable 4D layout and uses it to guide temporally stable LiDAR synthesis.
• We propose a comprehensive evaluation suite for LiDAR sequence generation, encompassing scene-level, object-level, and sequence-level metrics.
• We demonstrate best single-frame and sequence-level LiDAR point cloud generation performance on nuScenes, with improved foreground quality over existing methods.

📚 Citation If you find this work helpful for your research, please kindly consider citing our paper:

@article{liang2025lidarcrafter,
    title   = {LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequences},
    author  = {Ao Liang and Youquan Liu and Yu Yang and Dongyue Lu and Linfeng Li and Lingdong Kong and Huaici Zhao and Wei Tsang Ooi},
    journal = {arXiv preprint arXiv:2508.03692},
    year    = {2025},
}

Updates

• [08/2025] - The technical report of LiDARCrafter is available on arXiv.

Outline

• ⚙️ Installation
• ♨️ Data Preparation
• 🚀 Getting Started
• 🔧 Generation Framework
• 🐍 Model Zoo
• 📝 TODO List
• License
• Acknowledgements

⚙️ Installation

For details related to installation and environment setups, kindly refer to INSTALL.md.

♨️ Data Preparation

Kindly refer to our HuggingFace Dataset 🤗 page from here for more details.

🚀 Getting Started

To learn more usage of this codebase, kindly refer to GET_STARTED.md.

🔧 Generation Framework

Overall Framework

4D Layout Generation

Single-Frame Generation

🐍 Model Zoo

To be updated.

📝 TODO List

• Initial release. 🚀
• Release the training code.
• Release the inference code.
• Release the evaluation code.

License

This work is under the Apache License Version 2.0, while some specific implementations in this codebase might be with other licenses. Kindly refer to LICENSE.md for a more careful check, if you are using our code for commercial matters.

Acknowledgements

This work is developed based on the MMDetection3D codebase.

MMDetection3D is an open-source toolbox based on PyTorch, towards the next-generation platform for general 3D perception. It is a part of the OpenMMLab project developed by MMLab.

Part of the benchmarked models are from the OpenPCDet and 3DTrans projects.