Automated Driving Maneuvers Under Interactive Environment Based on Deep Reinforcement Learning

Safe and efficient autonomous driving maneuvers in an interactive and complex environment can be considerably challenging due to the unpredictable actions of other surrounding agents that may be cooperative or adversarial in their interactions with the ego vehicle. One of the state-of-the-art approaches is to apply Reinforcement Learning (RL) to learn a time-sequential driving policy, to execute proper control strategy or tracking trajectory in dynamic situations. However, direct application of RL algorithms is not satisfactorily enough to deal with the cases in the autonomous driving domain, mainly due to the complex driving environment and continuous action space. In this paper, the authors adopt Q-learning as their basic learning framework and design a unique format of the Q-function approximator that consists of neural networks to handle the continuous action space challenge. The learning model is present in a closed form of continuous control variables and trained in a simulation platform that the authors have developed with embedded properties of real-time vehicle interactions. The proposed algorithm avoids invoking an additional ‘actor’ network that learns to take actions, as in actor-critic algorithms. At the same time, some prior knowledge of vehicle dynamics is also fed into the model to assist learning. The authors test their algorithm with a challenging use case - lane change maneuver, to verify the practicability and feasibility of the proposed approach. Results from accumulated rewards and vehicle performance show that RL vehicle agents successfully learn a safe, comfort and efficient driving policy as defined in the reward function.

• Supplemental Notes:
  • This paper was sponsored by TRB committee AHB30 Standing Committee on Vehicle-Highway Automation. Alternate title: Driving Maneuver Learning via Reinforcement Learning for Automated Vehicles in An Interactive Environment
• Corporate Authors:

  Transportation Research Board

  ,    
• Authors:
  • Wang, Pin
  • Chan, Ching-Yao
• Conference:
  • Transportation Research Board 98th Annual Meeting
  • Location: Washington DC, United States
  • Date: 2019-1-13 to 2019-1-17
• Date: 2019

Language

• English

Media Info

• Media Type: Digital/other
• Features: Figures; References;
• Pagination: 5p

Subject/Index Terms

• TRT Terms: Advanced vehicle control systems; Autonomous vehicles; Intelligent agents; Lane changing; Machine learning; Neural networks
• Uncontrolled Terms: Deep learning; Reinforcement learning
• Subject Areas: Data and Information Technology; Highways; Vehicles and Equipment;

Filing Info

• Accession Number: 01697429
• Record Type: Publication
• Report/Paper Numbers: 19-01116
• Files: TRIS, TRB, ATRI
• Created Date: Mar 1 2019 3:50PM