Context-guided ground truth sampling for multi-modality data augmentation in autonomous driving

Data augmentation is an important pre-processing step for object detection in 2D image and 3D point clouds. However, studies on multimodal data augmentation are extremely limited compared to single-modal work. Moreover, simultaneously ensuring consistency and rationality when pasting both image and point cloud samples is a major challenge in multimodal methods. In this study, a novel multimodal data augmentation method based on ground truth sampling (GT sampling) is proposed for generating content-rich synthetic scenes. A GT database and scene ground database based on the raw training set is initially built, following which the context of the image and point cloud is used to guide the paste location and filtering strategy of the GT samples. The proposed method can avoid the cluttered features caused by random pasting of samples; the image context information can help the model to learn the correlation between the object and the environment more comprehensively, and the point cloud context information can reduce occlusion in the case of long-distance objects. The effectiveness of the proposed strategy is demonstrated on the publicly available KITTI dataset. Utilizing the multimodal 3D detector MVXNet as an implementation tool, the authors' experiments evaluate different superimposition strategies ranging from context-free sample pasting methods to context-guided new training scenes. In comparison with existing GT sampling methods, the authors' method exhibits a relative performance improvement of 15% on benchmark datasets. In ablation studies, the authors' sample pasting strategy achieves a +2.81% gain compared with previous work. In conclusion, considering the multimodal context of modelled objects is crucial for placing them in the correct environment.

• Record URL:
  http://dx.doi.org/10.1049/itr2.12272
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/issn/1751956X
• Supplemental Notes:
  • Abstract reprinted with permission of the Institution of Engineering and Technology.
• Authors:
  • Shi, Peicheng
  • Qi, Heng
  • Liu, Zhiqiang
  • Yang, Aixi
• Publication Date: 2023-3

Language

• English

Media Info

• Media Type: Web
• Features: Figures; References; Tables;
• Pagination: pp 459-469
• Serial:
  • IET Intelligent Transport Systems
  • Volume: 17
  • Issue Number: 3
  • Publisher: Institution of Engineering and Technology (IET)
  • ISSN: 1751-956X
  • EISSN: 1751-9578
  • Serial URL: https://ietresearch.onlinelibrary.wiley.com/journal/17519578
• Publication flags:

  Open Access (libre)

Subject/Index Terms

• TRT Terms: Autonomous vehicles; Data files; Detection and identification; Image processing; Machine learning
• Subject Areas: Data and Information Technology; Highways; Vehicles and Equipment;

Filing Info

• Accession Number: 01876605
• Record Type: Publication
• Files: TRIS
• Created Date: Mar 23 2023 10:19AM