Adaptive Multi-Agent Deep Mixed Reinforcement Learning for Traffic Light Control

Despite significant advancements in Multi-Agent Deep Reinforcement Learning (MADRL) approaches for Traffic Light Control (TLC), effectively coordinating agents in diverse traffic environments remains a challenge. Studies in MADRL for TLC often focus on repeatedly constructing the same intersection models with sparse experience. However, real road networks comprise Multi-Type of Intersections (MTIs) rather than being limited to intersections with four directions. In the scenario with MTIs, each type of intersection exhibits a distinctive topology structure and phase set, leading to disparities in the spaces of state and action. This article introduces Adaptive Multi-agent Deep Mixed Reinforcement Learning (AMDMRL) for addressing tasks with multiple types of intersections in TLC. AMDMRL adopts a two-level hierarchy, where high-level proxies guide low-level agents in decision-making and updating. All proxies are updated by value decomposition to obtain the globally optimal policy. Moreover, the AMDMRL approach incorporates a mixed cooperative mechanism to enhance cooperation among agents, which adopts a mixed encoder to aggregate the information from correlated agents. The authors conduct comparative experiments involving four traditional and four DRL-based approaches, utilizing three training and four testing datasets. The results indicate that the AMDMRL approach achieves average reductions of 41% than traditional approaches, and 16% compared to DRL-based approaches in traveling time on three training datasets. During testing, the AMDMRL approach exhibits a 37% improvement in reward compared to the MADRL-based approaches.

• Record URL:
  • http://dx.doi.org/10.1109/TVT.2023.3319698
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/issn/00189545
• Supplemental Notes:
  • Copyright © 2024, IEEE.
• Authors:
  • Li, Lulu
  • Zhu, Ruijie
  • Wu, Shuning
  • Ding, Wenting
  • Xu, Mingliang
  • Lu, Jiwen
• Publication Date: 2024-2

Language

• English

Media Info

• Media Type: Web
• Features: References;
• Pagination: pp 1803-1816
• Serial:
  • IEEE Transactions on Vehicular Technology
  • Volume: 73
  • Issue Number: 2
  • Publisher: Institute of Electrical and Electronics Engineers (IEEE)
  • ISSN: 0018-9545
  • Serial URL: http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=25

Subject/Index Terms

• TRT Terms: Computer models; Decision making; Machine learning; Traffic signal control systems; Traffic signals
• Subject Areas: Highways; Operations and Traffic Management;

Filing Info

• Accession Number: 01911216
• Record Type: Publication
• Files: TRIS
• Created Date: Mar 8 2024 3:32PM