Reinforcement Learning Based Traffic Signal Control Using Trajectory Data of Connected Vehicles

As the rise of connected vehicles (CV), it becomes practical to train learning-based methods to better control traffic signals by utilizing the CV data. Reinforcement learning (RL) has shown great promise in agent-environment interactive optimal control. Current RL-based signal control methods usually assume 100% CV penetration to get the full knowledge of the environment and simplify the intersection scenario to reduce the state and action spaces, which make these methods not practical. This paper proposes a new RL-based signal control algorithm considering a real-world four-leg twelve-movement intersection and study the control performance under limited CV penetration. The positions, speeds, and accumulated waiting times are normalized and spatially discretized to represent the state in RL. The average waiting time and penalty of phase transitions are weighted up as the reward in RL. A two-stage deep Q-learning method is adopted to learn the Q-function. The numerical experiment results show that the proposed RL-based signal control algorithm could reduce the average queue length, average waiting time, and average time loss by up to 32.85%, 32.37%, and 19.14%, respectively. The relationship between CV penetration and RL-based control performance is also analyzed.

• Summary URL:
  https://c2smart.engineering.nyu.edu/vehicle-traffic-control-limited-capacity-cavs/
• Supplemental Notes:
  • This document was sponsored by the U.S. Department of Transportation, University Transportation Centers Program.
• Corporate Authors:

  University of Washington, Seattle

  Department of Civil and Environmental Engineering
  Seattle, WA  United States  98105

  C2SMART Connected Cities with Smart Transportation

  NYU Tandon School of Engineering
  Department of Civil and Urban Engineering
  Brooklyn, NY  United States 

  Office of the Assistant Secretary for Research and Technology

  University Transportation Centers Program
  Department of Transportation
  Washington, DC  United States  20590
• Authors:
  • Guo, Qiangqiang
  • 0000-0002-6461-5886
  • Ban, Xuegang (Jeff)
  • 0000-0003-3605-971X
• Publication Date: 2021

Language

• English

Media Info

• Media Type: Digital/other
• Features: Figures; References; Tables;
• Pagination: 15p

Subject/Index Terms

• TRT Terms: Algorithms; Connected vehicles; Machine learning; Market penetration; Stopped time delays; Traffic queuing; Traffic signal control systems; Travel time; Vehicle trajectories
• Subject Areas: Highways; Operations and Traffic Management; Vehicles and Equipment;

Filing Info

• Accession Number: 01767076
• Record Type: Publication
• Contract Numbers: 69A3551747124
• Files: UTC, TRIS, ATRI, USDOT
• Created Date: Mar 18 2021 10:02AM