End-to-End Autonomous Driving With Semantic Depth Cloud Mapping and Multi-Agent

Focusing on the task of point-to-point navigation for an autonomous driving vehicle, the authors propose a novel deep learning model trained with end-to-end and multi-task learning manners to perform both perception and control tasks simultaneously. The model is used to drive the ego vehicle safely by following a sequence of routes defined by the global planner. The perception part of the model is used to encode high-dimensional observation data provided by an RGBD camera while performing semantic segmentation, semantic depth cloud (SDC) mapping, and traffic light state and stop sign prediction. Then, the control part decodes the encoded features along with additional information provided by GPS and speedometer to predict waypoints that come with a latent feature space. Furthermore, two agents are employed to process these outputs and make a control policy that determines the level of steering, throttle, and brake as the final action. The model is evaluated on CARLA simulator with various scenarios made of normal-adversarial situations and different weathers to mimic real-world conditions. In addition, the authors do a comparative study with some recent models to justify the performance in multiple aspects of driving. Moreover, the authors also conduct an ablation study on SDC mapping and multi-agent to understand their roles and behavior. As a result, the authors' model achieves the highest driving score even with fewer parameters and computation load. To support future studies, the authors share their codes at https://github.com/oskarnatan/end-to-end-driving.

• Record URL:
  http://dx.doi.org/10.1109/TIV.2022.3185303
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/issn/23798858
• Supplemental Notes:
  • Copyright © 2023, IEEE.
• Authors:
  • Natan, Oskar
  • Miura, Jun
• Publication Date: 2023-1

Language

• English

Media Info

• Media Type: Web
• Features: References;
• Pagination: pp 557-571
• Serial:
  • IEEE Transactions on Intelligent Vehicles
  • Volume: 8
  • Issue Number: 1
  • Publisher: Institute of Electrical and Electronics Engineers (IEEE)
  • ISSN: 2379-8858
  • Serial URL: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=7274857

Subject/Index Terms

• TRT Terms: Automatic steering control; Autonomous vehicles; Data fusion; Highway safety; Image processing; Level 5 driving automation; Machine learning; Multi-agent systems
• Subject Areas: Data and Information Technology; Highways; Operations and Traffic Management; Vehicles and Equipment;

Filing Info

• Accession Number: 01875448
• Record Type: Publication
• Files: TRIS
• Created Date: Mar 13 2023 10:23AM