Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events

The transportation demand is rapidly growing in metropolises, resulting in chronic traffic congestions in dense downtown areas. Adaptive traffic signal control as the principle part of intelligent transportation systems has a primary role to effectively reduce traffic congestion by making a real-time adaptation in response to the changing traffic network dynamics. Reinforcement learning (RL) is an effective approach in machine learning that has been applied for designing adaptive traffic signal controllers. One of the most efficient and robust type of RL algorithms are continuous state actor-critic algorithms that have the advantage of fast learning and the ability to generalize to new and unseen traffic conditions. These algorithms are utilized in this paper to design adaptive traffic signal controllers called actor-critic adaptive traffic signal controllers (A-CATs controllers). The contribution of the present work rests on the integration of three threads: (a) showing performance comparisons of both discrete and continuous A-CATs controllers in a traffic network with recurring congestion (24-h traffic demand) in the upper downtown core of Tehran city, (b) analyzing the effects of different traffic disruptions including opportunistic pedestrians crossing, parking lane, non-recurring congestion, and different levels of sensor noise on the performance of A-CATS controllers, and (c) comparing the performance of different function approximators (tile coding and radial basis function) on the learning of A-CATs controllers. To this end, first an agent-based traffic simulation of the study area is carried out. Then six different scenarios are conducted to find the best A-CATs controller that is robust enough against different traffic disruptions. The authors observe that the A-CATs controller based on radial basis function networks (RBF (5)) outperforms others. This controller is benchmarked against controllers of discrete state Q-learning, Bayesian Q-learning, fixed time and actuated controllers; and the results reveal that it consistently outperforms them.

• Record URL:
  https://doi.org/10.1016/j.trc.2017.09.020
• Record URL:
  https://www.sciencedirect.com/science/article/pii/S0968090X17302681
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/issn/0968090X
• Supplemental Notes:
  • Abstract reprinted with permission of Elsevier.
• Authors:
  • Aslani, Mohammad
  • Mesgari, Mohammad Saadi
  • Wiering, Marco
• Publication Date: 2017-12

Language

• English

Media Info

• Media Type: Web
• Features: Appendices; Figures; References; Tables;
• Pagination: pp 732-752
• Serial:
  • Transportation Research Part C: Emerging Technologies
  • Volume: 85
  • Issue Number: 0
  • Publisher: Elsevier
  • ISSN: 0968-090X
  • Serial URL: http://www.sciencedirect.com/science/journal/0968090X

Subject/Index Terms

• TRT Terms: Adaptive control; Machine learning; Service disruption; Traffic signal controllers; Traffic simulation
• Geographic Terms: Tehran (Iran)
• Subject Areas: Data and Information Technology; Highways; Operations and Traffic Management; Pedestrians and Bicyclists;

Filing Info

• Accession Number: 01656734
• Record Type: Publication
• Files: TRIS
• Created Date: Jan 22 2018 10:48AM