Agent-based Traffic Management and Reinforcement Learning in Congested Intersection Network

This study evaluates the performance of traffic control systems based on reinforcement learning (RL), also called approximate dynamic programming (ADP). Two algorithms have been selected for testing: 1) Q-learning and 2) approximate dynamic programming (ADP) with a post-decision state variable. The algorithms were tested in increasingly complex scenarios, from an oversaturated isolated intersection, to an arterial in undersaturated conditions, to a 2x5 network in both undersaturation and oversaturation, and finally to a 4x5 network in oversaturation with even and uneven directional demands. Potential benefits of these algorithms include signal systems that not only quickly respond to the actual conditions found in the field, but also learn about them and truly adapt through flexible cycle-free strategies. Moreover, these signal systems are decentralized, providing greater scalability and lower vulnerability at the network level. Results showed that agents with RL algorithms (ADP and Q-learning) were able to manage the traffic signals efficiently in both undersaturation and oversaturation.

• Record URL:
  http://www.purdue.edu/discoverypark/nextrans/assets/pdfs/Final%20Report%20072IY03.pdf
• 
• Supplemental Notes:
  • This document was sponsored by the U.S. Department of Transportation, University Transportation Centers Program.
• Corporate Authors:

  University of Illinois, Urbana-Champaign

  Department of Civil and Environmental Engineering
  205 North Mathews Avenue
  Urbana, IL  United States  61801-2352

  NEXTRANS

  Purdue University
  3000 Kent Avenue
  Lafayette, IN  United States  47906-1075

  Research and Innovative Technology Administration

  1200 New Jersey Avenue, SE
  Washington, DC  United States  20590
• Authors:
  • Medina, Juan C
  • Benekohal, Rahim (Ray) F
• Publication Date: 2012-8-23

Language

• English

Media Info

• Media Type: Web
• Edition: Final Report
• Features: Figures; References; Tables;
• Pagination: 152p

Subject/Index Terms

• TRT Terms: Algorithms; Dynamic programming; Machine learning; Oversaturation (Traffic flow); Signalized intersections; Traffic congestion; Traffic signal control systems
• Uncontrolled Terms: Undersaturated conditions (Traffic)
• Subject Areas: Highways; Operations and Traffic Management; I73: Traffic Control;

Filing Info

• Accession Number: 01481626
• Record Type: Publication
• Report/Paper Numbers: 072IY03
• Contract Numbers: DTRT07-G-005
• Files: UTC, TRIS, RITA, ATRI, USDOT
• Created Date: May 20 2013 3:30PM