Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control
Reinforcement learning (RL) is a promising data-driven approach for adaptive traffic signal control (ATSC) in complex urban traffic networks, and deep neural networks further enhance its learning power. However, the centralized RL is infeasible for large-scale ATSC due to the extremely high dimension of the joint action space. The multi-agent RL (MARL) overcomes the scalability issue by distributing the global control to each local RL agent, but it introduces new challenges: now, the environment becomes partially observable from the viewpoint of each local agent due to limited communication among agents. Most existing studies in MARL focus on designing efficient communication and coordination among traditional Q-learning agents. This paper presents, for the first time, a fully scalable and decentralized MARL algorithm for the state-of-the-art deep RL agent, advantage actor critic (A2C), within the context of ATSC. In particular, two methods are proposed to stabilize the learning procedure, by improving the observability and reducing the learning difficulty of each local agent. The proposed multi-agent A2C is compared against independent A2C and independent Q-learning algorithms, in both a large synthetic traffic grid and a large real-world traffic network of Monaco city, under simulated peak-hour traffic dynamics. The results demonstrate its optimality, robustness, and sample efficiency over the other state-of-the-art decentralized MARL algorithms.
- Record URL:
-
Availability:
- Find a library where document is available. Order URL: http://worldcat.org/oclc/41297384
-
Supplemental Notes:
- Copyright © 2020, IEEE.
-
Authors:
- Chu, Tianshu
- Wang, Jie
- Codecà, Lara
- Li, Zhaojian
- Publication Date: 2020
Language
- English
Media Info
- Media Type: Digital/other
- Features: Figures; References; Tables;
- Pagination: pp 1086-1095
-
Serial:
- IEEE Transactions on Intelligent Transportation Systems
- Volume: 21
- Issue Number: 3
- Publisher: Institute of Electrical and Electronics Engineers (IEEE)
- ISSN: 1524-9050
- Serial URL: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6979
Subject/Index Terms
- TRT Terms: Adaptive control; Highway traffic control systems; Machine learning; Multi-agent systems; Neural networks; Traffic flow; Traffic simulation
- Geographic Terms: Monaco
- Subject Areas: Data and Information Technology; Highways; Operations and Traffic Management; Planning and Forecasting;
Filing Info
- Accession Number: 01737773
- Record Type: Publication
- Files: TLIB, TRIS
- Created Date: Apr 24 2020 5:29PM