Traffic Signal Control With Reinforcement Learning Based on Region-Aware Cooperative Strategy

With the increase of private cars, traditional traffic signal control methods cannot alleviate the traffic congestion problem. Reinforcement learning (RL) is increasingly used in adaptive traffic light control. As urban traffic becomes more complex, reinforcement learning algorithms solely based on value or policy are not suitable for such scenarios. Moreover, the centralized method does not show a good effect on the multi-intersection, in particular, when the traffic flow is in the high-dimensional continuous state space. In this article, the authors propose a decentralized framework based on the advantage actor-critic (A2C) algorithm by assigning global control to each local RL agent or intersection. A2C algorithm involved in this article is a product of combining policy function and value function, which has good convergence ability and can be applied to continuous state space. The decentralized methods may put a new challenge: from the perspective of each local agent, the environment becomes partially observable. The authors overcome this problem by putting forward a region-aware cooperative strategy (RACS) based on graph attention network (GAT), which can incorporate the spatial information of the surrounding agents. The authors carry out experiments on synthetic traffic grid and real-world traffic network of Monaco city to compare with the existing A2C and Q-learning algorithms. Experimental results confirm that the RACS method has a shorter queue length and less waiting time than the two existing algorithms, and can reduce the total vehicle travel time.

• Record URL:
  • http://dx.doi.org/10.1109/TITS.2021.3062072
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/oclc/41297384
• Supplemental Notes:
  • Copyright © 2022, IEEE.
• Authors:
  • Wang, Min
  • 0000-0002-6043-5191
  • Wu, Libing
  • Li, Jianxin
  • 0000-0002-9059-330X
  • He, Liu
• Publication Date: 2022-7

Language

• English

Media Info

• Media Type: Web
• Features: Figures; References;
• Pagination: pp 6774-6785
• Serial:
  • IEEE Transactions on Intelligent Transportation Systems
  • Volume: 23
  • Issue Number: 7
  • Publisher: Institute of Electrical and Electronics Engineers (IEEE)
  • ISSN: 1524-9050
  • Serial URL: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6979

Subject/Index Terms

• TRT Terms: Algorithms; Machine learning; Traffic queuing; Traffic signal control systems; Travel time
• Subject Areas: Highways; Operations and Traffic Management; Planning and Forecasting;

Filing Info

• Accession Number: 01853918
• Record Type: Publication
• Files: TRIS
• Created Date: Aug 4 2022 5:17PM