Skip to content

Introduction

Welcome to the Dynamic Map Benchmark Wiki Page!

You can always press F or top right search bar to search for specific topics.

Please give us a star🌟 and cite our work📖 to support our work if you find this useful.

CHANGELOG:
  • 2024/08/24: Reorganize the wiki page. Add all scripts usage in the benchmark code, Data Overview with visualization, Method Overview with a demo to output clean map and Evaluation to evaluate the performance of the methods and also automatically output the visualization in the paper.
  • 2024/06/25: Qingwen is starting to work on the wiki page.
  • 2024/04/29 BeautyMap is accepted by RA-L'24. Updated benchmark: BeautyMap and DeFlow submodule instruction in the benchmark. Added the first data-driven method DeFlow into our benchmark. Feel free to check.
  • 2024/04/18 DUFOMap is accepted by RA-L'24. Updated benchmark: DUFOMap and dynablox submodule instruction in the benchmark. Two datasets w/o gt for demo are added in the download link. Feel free to check.
  • 2024/03/08 Fix statements on our ITSC'23 paper: KITTI sequences pose are also from SemanticKITTI which used SuMa. In the DUFOMap paper Section V-C, Table III, we present the dynamic removal result on different pose sources. Check discussion in DUFOMap paper if you are interested.
  • 2023/06/13 The benchmark paper Accepted by ITSC 2023 and release five methods (Octomap, Octomap w GF, ERASOR, Removert) and three datasets (01, 05, av2, semindoor) in benchmark paper.

Overview

Task: Detect and Remove dynamic points from Point Cloud Maps.

Here is a figure that illustrate of ghost points resulting from dynamic objects in KITTI sequence 7. The yellow points to the right represent points labeled as belonging to dynamic objects in the dataset. These ghost points negatively affect downstream tasks and overall point cloud quality

🏘️ Good to start from here

  • What kind of data format we use?

    PCD files (pose information saved in VIEWPOINT header). Read Data Section

  • How to evaluate the performance of a method?

    Two python scripts. Read Evaluation Section

  • How to run a benchmark method on my data?

    Format your data and run the method. Read Create data and Run method

🎁 Methods we included

Online (w/o prior map):

Learning-based (data-driven) (w pretrain-weights provided):

Offline (need prior map).

Please note that we provided the comparison methods also but modified a little bit for us to run the experiments quickly, but no modified on their methods' core. Please check the LICENSE of each method in their official link before using it.

You will find all methods in this benchmark under methods folder. So that you can easily reproduce the experiments. Or click here to check our score screenshot directly.

Last but not least, feel free to pull request if you want to add more methods. Welcome!

💖 Acknowledgements

This benchmark implementation is based on codes from several repositories as we mentioned in the beginning. Thanks for these authors who kindly open-sourcing their work to the community. Please see our paper reference section to get more information.

Thanks to HKUST Ramlab's members: Bowen Yang, Lu Gan, Mingkai Tang, and Yingbing Chen, who help collect additional datasets.

This work was partially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation

Cite Our Papers

Please cite our works if you find these useful for your research:

@inproceedings{zhang2023benchmark,
  author={Zhang, Qingwen and Duberg, Daniel and Geng, Ruoyu and Jia, Mingkai and Wang, Lujia and Jensfelt, Patric},
  booktitle={IEEE 26th International Conference on Intelligent Transportation Systems (ITSC)}, 
  title={A Dynamic Points Removal Benchmark in Point Cloud Maps}, 
  year={2023},
  pages={608-614},
  doi={10.1109/ITSC57777.2023.10422094}
}
@article{daniel2024dufomap,
  author={Duberg, Daniel and Zhang, Qingwen and Jia, Mingkai and Jensfelt, Patric},
  journal={IEEE Robotics and Automation Letters}, 
  title={{DUFOMap}: Efficient Dynamic Awareness Mapping}, 
  year={2024},
  volume={9},
  number={6},
  pages={5038-5045},
  doi={10.1109/LRA.2024.3387658}
}
@article{jia2024beautymap,
  author={Jia, Mingkai and Zhang, Qingwen and Yang, Bowen and Wu, Jin and Liu, Ming and Jensfelt, Patric},
  journal={IEEE Robotics and Automation Letters}, 
  title={{BeautyMap}: Binary-Encoded Adaptable Ground Matrix for Dynamic Points Removal in Global Maps}, 
  year={2024},
  volume={9},
  number={7},
  pages={6256-6263},
  doi={10.1109/LRA.2024.3402625}
}