Multi-Modal 3D Object Detection in Autonomous Driving: A Survey and Taxonomy

Autonomous vehicles require constant environmental perception to obtain the distribution of obstacles to achieve safe driving. Specifically, 3D object detection is a vital functional module as it can simultaneously predict surrounding objects' categories, locations, and sizes. Generally, autonomous vehicles are equipped with multiple sensors, including cameras and LiDARs. The fact that single-modal methods suffer from unsatisfactory detection performance motivates utilizing multiple modalities as inputs to compensate for single sensor faults. Although many multi-modal fusion detection algorithms exist, there is still a lack of comprehensive and in-depth analysis of these methods to clarify how to fuse multi-modal data effectively. Therefore, this paper surveys recent advancements in fusion detection methods. First, the authors present the broad background of multi-modal 3D object detection and identify the characteristics of widely used datasets along with their evaluation metrics. Second, instead of the traditional classification method of early, middle, and late fusion, the authors categorize and analyze all fusion methods from three aspects: feature representation, alignment, and fusion, which reveals how these fusion methods are implemented in an essential way. Third, the authors provide an in-depth comparison of their pros and cons and compare their performance in mainstream datasets. Finally, the authors further summarize current challenges and research trends for realizing the full potential of multi-modal 3D object detection.

• Record URL:
  http://dx.doi.org/10.1109/TIV.2023.3264658
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/issn/23798858
• Supplemental Notes:
  • Copyright © 2023, IEEE.
• Authors:
  • Wang, Li
  • Zhang, Xinyu
  • Song, Ziying
  • Bi, Jiangfeng
  • Zhang, Guoxin
  • Wei, Haiyue
  • Tang, Liyao
  • Yang, Lei
  • Li, Jun
  • Jia, Caiyan
  • Zhao, Lijun
• Publication Date: 2023-7

Language

• English

Media Info

• Media Type: Web
• Features: Figures; References;
• Pagination: pp 3781-3798
• Serial:
  • IEEE Transactions on Intelligent Vehicles
  • Volume: 8
  • Issue Number: 7
  • Publisher: Institute of Electrical and Electronics Engineers (IEEE)
  • ISSN: 2379-8858
  • Serial URL: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=7274857

Subject/Index Terms

• TRT Terms: Autonomous vehicles; In vehicle sensors; Laser radar; Level 5 driving automation; Perception; Stereoscopic cameras
• Subject Areas: Data and Information Technology; Highways; Safety and Human Factors; Vehicles and Equipment;

Filing Info

• Accession Number: 01892340
• Record Type: Publication
• Files: TRIS
• Created Date: Sep 5 2023 4:12PM