Cooperative bus holding and stop-skipping: A deep reinforcement learning framework

The bus control problem that combines holding and stop-skipping strategies is formulated as a multi-agent reinforcement learning (MARL) problem. Traditional MARL methods, designed for settings with joint action-taking, are incompatible with the asynchronous nature of at-stop control tasks. On the other hand, using a fully decentralized approach leads to environment non-stationarity, since the state transition of an individual agent may be distorted by the actions of other agents. To address it, the authors propose a design of the state and reward function that increases the observability of the impact of agents’ actions during training. An event-based mesoscopic simulation model is built to train the agents. The authors evaluate the proposed approach in a case study with a complex route from the Chicago transit network. The proposed method is compared to a standard headway-based control and a policy trained with MARL but with no cooperative learning. The results show that the proposed method not only improves level of service but it is also more robust towards uncertainties in operations such as travel times and operator compliance with the recommended action.

• Record URL:
  • https://doi.org/10.1016/j.trc.2023.104308
• Record URL:
  • http://www.sciencedirect.com/science/article/pii/S0968090X23002978
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/issn/0968090X
• Supplemental Notes:
  • © 2023 Elsevier Ltd. All rights reserved. Abstract reprinted with permission of Elsevier.
• Authors:
  • Rodriguez, Joseph
  • 0000-0001-9806-1423
  • Koutsopoulos, Haris N
  • Wang, Shenhao
  • 0000-0003-4374-8193
  • Zhao, Jinhua
  • 0000-0002-1929-7583
• Publication Date: 2023-10

Language

• English

Media Info

• Media Type: Web
• Features: Figures; References; Tables;
• Pagination: 104308
• Serial:
  • Transportation Research Part C: Emerging Technologies
  • Volume: 155
  • Issue Number: 0
  • Publisher: Elsevier
  • ISSN: 0968-090X
  • Serial URL: http://www.sciencedirect.com/science/journal/0968090X

Subject/Index Terms

• TRT Terms: Bus transit operations; Level of service; Machine learning; Real time control; Travel demand management; Travel time reliability
• Identifier Terms: Chicago Transit Authority
• Geographic Terms: Chicago (Illinois)
• Subject Areas: Data and Information Technology; Highways; Motor Carriers; Operations and Traffic Management; Public Transportation;

Filing Info

• Accession Number: 01894351
• Record Type: Publication
• Files: TRIS
• Created Date: Sep 25 2023 2:46PM