A Deep Reinforcement Learning-based Ramp Metering Control Framework for Improving Traffic Operation at Freeway Weaving Sections

Ramp metering (RM) dynamically adjusts ramp flow merging into freeway mainline according to real-time traffic conditions to improve traffic operation. The effectiveness of RM is mainly determined by its control strategies, which decide how to calculate flow for various traffic states. The traditional RM control strategies are limited by the responding speed to traffic changes and the online computation workloads. They also require large and sufficient human knowledges about the traffic flow problems in the study segments. In this study, the authors aim at proposing a control framework that is deep reinforcement learning-based RM, named DQN-based RM. This new control framework incorporates the deep Q network (DQN) algorithm and the RM in order to reduce total travel time on freeways. A typical freeway weaving bottleneck section was simulated based on the Simulation of Urban Mobility (SUMO) platform. The results show that the proposed DQN-based RM strategy is able to response proactively to different traffic states and take immediate and correct actions to prevent traffic breakdown, without full prior knowledge of traffic flow theories. The DQN-based RM could reach the optimal control target within a short training time, and the total travel time was reduced by 51.48% and 50.58% with 15 s and 30 s as the control cycle. The authors also compare the performances of various RM strategies. The results show that the DQN-based RM outperforms the traditional fixed-time and the feedback-based RM control strategies in mitigating congestions and reducing travel time on freeways.

• Supplemental Notes:
  • This paper was sponsored by TRB committee ABJ70 Standing Committee on Artificial Intelligence and Advanced Computing Applications.
• Corporate Authors:

  Transportation Research Board

  ,    
• Authors:
  • Yang, Mofeng
  • Li, Zhibin
  • Ke, Zemian
  • Li, Meng
• Conference:
  • Transportation Research Board 98th Annual Meeting
  • Location: Washington DC, United States
  • Date: 2019-1-13 to 2019-1-17
• Date: 2019

Language

• English

Media Info

• Media Type: Digital/other
• Features: Figures; References; Tables;
• Pagination: 8p

Subject/Index Terms

• TRT Terms: Algorithms; Freeways; Ramp metering; Real time information; Traffic control; Traffic flow; Travel time; Weaving sections
• Subject Areas: Highways; Operations and Traffic Management;

Filing Info

• Accession Number: 01698023
• Record Type: Publication
• Report/Paper Numbers: 19-03788
• Files: TRIS, TRB, ATRI
• Created Date: Mar 1 2019 3:51PM