A Distributed Multi-Agent Reinforcement Learning With Graph Decomposition Approach for Large-Scale Adaptive Traffic Signal Control

With the emerging connected-vehicle technologies and smart roadways, the need for intelligent adaptive traffic signal controls (ATSC) is more than ever before. This paper first proposes an Accumulated Exponentially Weighted Waiting Time-based Adaptive Traffic Signal Control (AEWWT-ATSC) model to calculate priorities of roadways for signal scheduling. As the size of the traffic network grows, it adds great complexities and challenges to computational efficiencies. Considering this, the authors propose a novel Distributed Multi-agent Reinforcement Learning (DMARL) with a graph decomposition approach for large-scale ATSC problems. The decomposition clusters intersections by the level of connectivity (LoC), defined by the average residual capacities (ARC) between connected intersections, enabling us to train subgraphs instead of the entire network in a synchronized way. The problem is formulated as a Markov Decision Process (MDP), and the Double Dueling Deep Q Network with Prioritized Experience Replay is utilized to solve it. Under the optimal policy, the agents can select the optimal signal durations to minimize the waiting time and queue size. In evaluation, the authors show the superiority of the AEWWT-ATSC based RL methods in different densities and demonstrate the DMARL with a graph decomposition approach on a large graph in Manhattan, NYC. The approach is generic and can be extended to various types of use cases.

• Record URL:
  • https://doi.org/10.1109/TITS.2021.3131596
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/oclc/41297384
• Supplemental Notes:
  • Copyright © 2022, IEEE.
• Authors:
  • Jiang, Shan
  • Huang, Yufei
  • 0000-0002-9996-9190
  • Jafari, Mohsen
  • Jalayer, Mohammad
• Publication Date: 2022-9

Language

• English

Media Info

• Media Type: Web
• Pagination: pp 14689-14701
• Serial:
  • IEEE Transactions on Intelligent Transportation Systems
  • Volume: 23
  • Issue Number: 9
  • Publisher: Institute of Electrical and Electronics Engineers (IEEE)
  • ISSN: 1524-9050
  • Serial URL: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6979

Subject/Index Terms

• TRT Terms: Adaptive control; Connected vehicles; Intelligent control systems; Traffic signal control systems
• Geographic Terms: New York (New York)
• Subject Areas: Highways; Operations and Traffic Management;

Filing Info

• Accession Number: 01876471
• Record Type: Publication
• Files: TRIS
• Created Date: Mar 22 2023 5:07PM