An Attention Reinforcement Learning–Based Strategy for Large-Scale Adaptive Traffic Signal Control System

This paper proposes a reinforcement learning (RL)-based traffic control strategy integrated with attention mechanism for large-scale adaptive traffic signal control (ATSC) system. The proposed attention RL integrates attention mechanism into a multiagent RL model, namely multiagent proximal policy optimization (MAPPO), so as to enable more effective, scalable, and stable learning in complex ATSC environments. In the attention RL, decentralized policies are trained using a centrally computed critic that shares an attention model, while the attention model selects relevant intersections for each agent to estimate the global critic. This framework effectively reduces the computational complexity and stabilizes the training process, enhancing the ability of RL agents to control large-scale traffic networks. The proposed control strategy is tested in both a large synthetic traffic grid and a large real-world traffic network of Yangzhou city using the microscopic traffic simulation tool, SUMO. Experimental results demonstrate that the proposed approach learns stable and sustainable policies that achieve lower congestion level and faster recovery, which outperforms other state-of-art RL-based approaches, as well as a gap-based actuated controller.

• Record URL:
  • https://doi.org/10.1061/JTEPBS.TEENG-8261
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/issn/24732907
• Supplemental Notes:
  • © 2024 American Society of Civil Engineers.
• Authors:
  • Han, Gengyue
  • 0009-0007-3831-2084
  • Liu, Xiaohan
  • Wang, Hao
  • 0000-0001-7961-7588
  • Dong, Changyin
  • Han, Yu
• Publication Date: 2024-3

Language

• English

Media Info

• Media Type: Web
• Features: References;
• Pagination: 04024001
• Serial:
  • Journal of Transportation Engineering, Part A: Systems
  • Volume: 150
  • Issue Number: 3
  • Publisher: American Society of Civil Engineers
  • ISSN: 2473-2907
  • EISSN: 2473-2893
  • Serial URL: http://ascelibrary.org/journal/jtepbs

Subject/Index Terms

• TRT Terms: Adaptive control; High volume roads; Highway traffic control systems; Machine learning; Microscopic traffic flow; Multi-agent systems
• Identifier Terms: SUMO (Traffic simulation model)
• Geographic Terms: Yangzhou (China)
• Subject Areas: Data and Information Technology; Highways; Operations and Traffic Management;

Filing Info

• Accession Number: 01908331
• Record Type: Publication
• Files: TRIS, ASCE
• Created Date: Feb 15 2024 5:05PM