Agent-based Traffic Management and Reinforcement Learning in Congested Intersection Network

This study evaluates the performance of traffic control systems based on reinforcement learning (RL), also called approximate dynamic programming (ADP). Two algorithms have been selected for testing: 1) Q-learning and 2) approximate dynamic programming (ADP) with a post-decision state variable. The algorithms were tested in increasingly complex scenarios, from an oversaturated isolated intersection, to an arterial in undersaturated conditions, to a 2x5 network in both undersaturation and oversaturation, and finally to a 4x5 network in oversaturation with even and uneven directional demands. Potential benefits of these algorithms include signal systems that not only quickly respond to the actual conditions found in the field, but also learn about them and truly adapt through flexible cycle-free strategies. Moreover, these signal systems are decentralized, providing greater scalability and lower vulnerability at the network level. Results showed that agents with RL algorithms (ADP and Q-learning) were able to manage the traffic signals efficiently in both undersaturation and oversaturation.

  • Record URL:
  • Supplemental Notes:
    • This document was sponsored by the U.S. Department of Transportation, University Transportation Centers Program.
  • Corporate Authors:

    University of Illinois, Urbana-Champaign

    Department of Civil and Environmental Engineering
    205 North Mathews Avenue
    Urbana, IL  United States  61801-2352


    Purdue University
    3000 Kent Avenue
    Lafayette, IN  United States  47906-1075

    Research and Innovative Technology Administration

    1200 New Jersey Avenue, SE
    Washington, DC  United States  20590
  • Authors:
    • Medina, Juan C
    • Benekohal, Rahim (Ray) F
  • Publication Date: 2012-8-23


  • English

Media Info

  • Media Type: Web
  • Edition: Final Report
  • Features: Figures; References; Tables;
  • Pagination: 152p

Subject/Index Terms

Filing Info

  • Accession Number: 01481626
  • Record Type: Publication
  • Report/Paper Numbers: 072IY03
  • Contract Numbers: DTRT07-G-005
  • Created Date: May 20 2013 3:30PM