Behavioral-Adaptive Deep Q-Network for Autonomous Driving Decisions in Heavy Traffic

Deep reinforcement learning (DRL) is confronted with the significant problem of sparse rewards for autonomous driving in heavy traffic because of the dynamic and diverse nature of the driving environment as well as the complexity of the driving task. To mitigate the impact of sparse rewards on the convergence process of DRL, this paper proposes a novel behavioral-adaptive deep Q-network (BaDQN) for autonomous driving decisions in heavy traffic. BaDQN applies the idea of task decomposition to the DRL process. To break down the complexity of the driving task and achieve shorter exploration paths, BaDQN divides the driving task into three subtasks: Lane-Changing, Posture-Adjustment, and Wheel-Holding. BaDQN uses the finite state machine (FSM) to model the collaborative relationship between different subtasks, and abstracts each subtask separately using the Markov decision process (MDP). We used the Carla simulator to conduct experiments in a specific heavy traffic scenario. Compared with previous methods, BaDQN achieves a longer safe driving distance and a higher success rate. To discuss the adaptability of BaDQN to changes in traffic density and traffic velocity, we also conducted two extended experiments, which fully demonstrated the performance stability of BaDQN.

• Record URL:
  • https://doi.org/10.1177/03611981241262314
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/issn/03611981
• Supplemental Notes:
  • Hong Yu https://orcid.org/0000-0003-0667-8413© The Author(s) 2024.
• Authors:
  • Liu, Zhicheng
  • Yu, Hong
  • 0000-0003-0667-8413
• Publication Date: 2025-2

Language

• English

Media Info

• Media Type: Web
• Features: References;
• Pagination: pp 377-389
• Serial:
  • Transportation Research Record: Journal of the Transportation Research Board
  • Volume: 2679
  • Issue Number: 2
  • Publisher: Sage Publications, Incorporated
  • ISSN: 0361-1981
  • EISSN: 2169-4052
  • Serial URL: http://journals.sagepub.com/home/trr

Subject/Index Terms

• TRT Terms: Autonomous vehicle guidance; Behavior; Decision support systems; Machine learning; Neural networks; Traffic characteristics
• Subject Areas: Data and Information Technology; Highways; Vehicles and Equipment;

Filing Info

• Accession Number: 01926002
• Record Type: Publication
• Files: TRIS, TRB
• Created Date: Jul 30 2024 9:53AM