Longitudinal Control of Automated Vehicles: A Novel Approach by Integrating Deep Reinforcement Learning With Intelligent Driver Model

Deep reinforcement learning (DRL) provides a promising approach for the implementation of autonomous driving. By utilizing a trained DRL model as the longitudinal controller, the automated vehicle (AV) can generate optimal action outputs based on the state within a shorter time compared to traditional model predictive control (MPC) methods. However, the non-interpretability of neural networks poses a potential risk for real-world vehicle operation. This paper focuses on applying the Twin Delayed Deep Deterministic Policy Gradient (TD3), a state-of-the-art (SOTA) DRL algorithm, to train the longitudinal control model for AVs. The authors confirm the risks associated with the TD3-based longitudinal control model by assessing its violation of the rational driving constraint (RDC), which represents the basic conditions for normal driving behaviors. To mitigate these risks, the authors propose a novel model that integrates the TD3-based model with the intelligent driver model (IDM) using a new indicator called velocity response time (VRT). This indicator identifies risky outputs of the TD3-based model and calculates the combined weights of both the IDM and TD3-based models. This combination allows the authors to reduce risks associated with the non-interpretability of the neural network while also capturing the effect of engine time lag. Numerical simulations are conducted to evaluate the performance of the proposed combined model. The results demonstrate that the proposed combined model outperforms the TD3-based model, IDM, and another SOTA approach in terms of disturbance mitigation, safety improvement, and suppression of traffic oscillation. Additionally, the combined model exhibits greater computational efficiency than MPC, making it well-suited for real-time control of AVs.

• Record URL:
  • http://dx.doi.org/10.1109/TVT.2024.3376599
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/issn/00189545
• Supplemental Notes:
  • Copyright © 2024, IEEE.
• Authors:
  • Bai, Linhan
  • Zheng, Fangfang
  • Hou, Kangning
  • Liu, Xiaobo
  • Lu, Liang
  • Liu, Can
• Publication Date: 2024-8

Language

• English

Media Info

• Media Type: Web
• Features: References;
• Pagination: pp 11014-11028
• Serial:
  • IEEE Transactions on Vehicular Technology
  • Volume: 73
  • Issue Number: 8
  • Publisher: Institute of Electrical and Electronics Engineers (IEEE)
  • ISSN: 0018-9545
  • Serial URL: http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=25

Subject/Index Terms

• TRT Terms: Intelligent control systems; Intelligent vehicles; Machine learning
• Identifier Terms: Model Predictive Control
• Subject Areas: Data and Information Technology; Highways; Vehicles and Equipment;

Filing Info

• Accession Number: 01930099
• Record Type: Publication
• Files: TRIS
• Created Date: Sep 13 2024 10:33AM