A Multimodality Fusion Deep Neural Network and Safety Test Strategy for Intelligent Vehicles

Multimodality fusion based on deep neural networks (DNN) is a significant method for intelligent vehicles. The special characteristics of DNN lead to the issue of AI safety and safety test. In this paper, the authors firstly propose a multimodality fusion framework called Integrated Multimodality Fusion Deep Neural Network (IMF-DNN), which can flexibly accomplish both object detection and end-to-end driving policy for prediction of steering angle and speed. Then, the authors propose a DNN safety test strategy, which systematically analyzes DNN's robustness and generalization ability in large amounts of diverse driving environment conditions. The test in this paper is based on the authors' IMF-DNN model and the strategy can be widely used for other DNNs. Finally, the experiment analysis is performed on KITTI for object detection and the dateset DBNet for end-to-end tasks. The results show the superior accuracy of the proposed IMF-DNN model and the test strategy's potential ability to improve the robustness and generalization of autonomous vehicle deep learning model. Code is available at https://github.com/ennisnie/IMF-DNN

• Record URL:
  http://dx.doi.org/10.1109/TIV.2020.3027319
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/issn/23798858
• Supplemental Notes:
  • Copyright © 2021, IEEE.
• Authors:
  • Nie, Jian
  • Yan, Jun
  • Yin, Huilin
  • Ren, Lei
  • Meng, Qian
• Publication Date: 2021-6

Language

• English

Media Info

• Media Type: Web
• Features: Figures; References;
• Pagination: pp 310-322
• Serial:
  • IEEE Transactions on Intelligent Vehicles
  • Volume: 6
  • Issue Number: 2
  • Publisher: Institute of Electrical and Electronics Engineers (IEEE)
  • ISSN: 2379-8858
  • Serial URL: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=7274857

Subject/Index Terms

• TRT Terms: Artificial intelligence; Autonomous vehicles; Data fusion; Neural networks; Vehicle safety; Vehicle tests
• Subject Areas: Data and Information Technology; Highways; Safety and Human Factors; Vehicles and Equipment;

Filing Info

• Accession Number: 01779866
• Record Type: Publication
• Files: TRIS
• Created Date: Aug 26 2021 4:54PM