Augmented LiDAR simulator for autonomous driving

Jin Fang, Dingfu Zhou, Feilong Yan, Tongtong Zhao, Feihu Zhang, Yu Ma, Liang Wang, Ruigang Yang

Research output: Contribution to journalArticlepeer-review

75 Scopus citations


In Autonomous Driving (AD), detection and tracking of obstacles on the roads is a critical task. Deep-learning based methods using annotated LiDAR data have been the most widely adopted approach for this. Unfortunately, annotating 3D point cloud is a very challenging, time- and money-consuming task. In this letter, we propose a novel LiDAR simulator that augments real point cloud with synthetic obstacles (e.g., vehicles, pedestrians, and other movable objects). Unlike previous simulators that entirely rely on CG (Computer Graphics) models and game engines, our augmented simulator bypasses the requirement to create high-fidelity background CAD (Computer Aided Design) models. Instead, we can deploy a vehicle with a LiDAR scanner to sweep the street of interests to obtain the background points cloud, based on which annotated point cloud can be automatically generated. This 'scan-and-simulate' capability makes our approach scalable and practical, ready for large-scale industrial applications. In this letter, we describe our simulator in detail, in particular the placement of obstacles that is critical for performance enhancement. We show that detectors with our simulated LiDAR point cloud alone can perform comparably (within two percentage points) with these trained with real data. Mixing real and simulated data can achieve over 95% accuracy.

Original languageEnglish
Article number8972449
Pages (from-to)1931-1938
Number of pages8
JournalIEEE Robotics and Automation Letters
Issue number2
StatePublished - Apr 2020

Bibliographical note

Publisher Copyright:
© 2016 IEEE.


  • Simulation and animation
  • computer vision for automation
  • object detection
  • segmentation and categorization

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Biomedical Engineering
  • Human-Computer Interaction
  • Mechanical Engineering
  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Control and Optimization
  • Artificial Intelligence


Dive into the research topics of 'Augmented LiDAR simulator for autonomous driving'. Together they form a unique fingerprint.

Cite this