A Deep-Reinforcement Learning Algorithm for Eco-Driving Control at Signalized Intersections with Prioritized Experience Replay, Target Network, and Double Learning

Eco-driving is a complex control problem where the driver’s actions are guided over a period of time or distance so as to achieve a certain goal such as optimizing fuel consumption. This paper presents an Eco-driving control system, developed to optimize vehicle trajectories near the signalized intersection by minimizing the acceleration noise and reducing stopped delay. The developed system interfaces with non-linear microscopic fuel consumption models. The model also utilizes the Deep Reinforced Learning (DRL) in developing a DRL algorithm using Artificial Neural Network concepts and novel techniques such as prioritized experience replay, target network, and double learning to overcome the expected instabilities during the training process. The developed model was applied to simulate vehicle movements along a 400-control section at a signalized intersection. Simulation was executed in Ptv-Vissim environment where driver’s actions were passed from the model to Vissim via Vissim-Com interface. Simulation results obtained for guided vehicles within the control section were compared with their counterparts of the typical movement without any guidance. Results showed that the estimated fuel consumption for guided case was about 17.5% less than that of the case with no guidance. Furthermore, the estimated acceleration noise for the guided case was less than that of the case with no guidance by about 16.9%. This reduction in fuel consumption and acceleration noise is an indicator for reducing vehicle emissions and improving traffic safety. This finding is considered promising for potential applications of the developed model and further enhancement of its features.

• Supplemental Notes:
  • This paper was sponsored by TRB committee ABJ00 Section - Data and Information Systems.
• Authors:
  • Mousa, Saleh R
  • Mousa, Ragab
  • Ishak, Sherif
• Conference:
  • Transportation Research Board 97th Annual Meeting
  • Location: Washington DC, United States
  • Date: 2018-1-7 to 2018-1-11
• Date: 2018

Language

• English

Media Info

• Media Type: Digital/other
• Pagination: 2p

Subject/Index Terms

• TRT Terms: Acceleration (Mechanics); Algorithms; Ecodriving; Fuel consumption; Machine learning; Neural networks; Optimization; Signalized intersections; Simulation; Vehicle trajectories
• Subject Areas: Energy; Environment; Highways; Operations and Traffic Management; Planning and Forecasting;

Filing Info

• Accession Number: 01659718
• Record Type: Publication
• Report/Paper Numbers: 18-06104
• Files: TRIS, TRB, ATRI
• Created Date: Feb 9 2018 4:52PM