eMARLIN: Distributed Coordinated Adaptive Traffic Signal Control with Topology-Embedding Propagation

In this paper, we examine the practical problem of minimizing the delay in traffic networks that are controlled at each intersection independently, without a centralized supervisory computer and with limited communication bandwidth. We find that existing learning algorithms have lackluster performance or are too computationally complex to be implemented in the field. Instead, we introduce a simple yet efficient and effective approach using multi-agent reinforcement learning (MARL) that applies the Deep Q-Network (DQN) learning algorithm in a fully decentralized setting. First, we decouple the DQN into per-intersection Q-networks and then transmit the output of each Q-network’s hidden layer to its intersection neighbors. We show that our method is computationally efficient compared with other MARL methods, with minimal additional overhead compared with a naive isolated learning approach with no communication. This property enables our method to be implemented in real-world scenarios with less computation power. Finally, we conduct experiments for both synthetic and real-world scenarios and show that our method achieves better performance in minimizing intersection delay than other methods.

• Record URL:
  • https://doi.org/10.1177/03611981231184250
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/issn/03611981
• Supplemental Notes:
  • Xiaoyu Wang, https://orcid.org/0000-0003-1587-6307© National Academy of Sciences: Transportation Research Board 2023.
• Authors:
  • Wang, Xiaoyu
  • 0000-0003-1587-6307
  • Taitler, Ayal
  • 0000-0003-3919-6883
  • Smirnov, Ilia
  • Sanner, Scott
  • 0000-0001-7984-8394
  • Abdulhai, Baher
  • 0000-0002-8787-2578
• Publication Date: 2024-4

Language

• English

Media Info

• Media Type: Web
• Features: References;
• Pagination: pp 189-202
• Serial:
  • Transportation Research Record: Journal of the Transportation Research Board
  • Volume: 2678
  • Issue Number: 4
  • Publisher: Sage Publications, Incorporated
  • ISSN: 0361-1981
  • EISSN: 2169-4052
  • Serial URL: http://journals.sagepub.com/home/trr

Subject/Index Terms

• TRT Terms: Adaptive control; Distributed control; Machine learning; Topology; Traffic signal control systems
• Subject Areas: Data and Information Technology; Highways; Operations and Traffic Management;

Filing Info

• Accession Number: 01887399
• Record Type: Publication
• Files: TRIS, TRB, ATRI
• Created Date: Jul 17 2023 9:06AM