Visible & Thermal Imaging and Deep Learning Based Approach for Automated Robust Detection of Potholes to Prioritize Highway Maintenance

Potholes are a significant pavement distress compromising safety and causing costly damage. They result from pavement deterioration due to aging, weather, and traffic overloads, with the Mountain Plains region particularly affected due to freeze/thaw cycles. Timely identification and repair of potholes are critical for effective highway maintenance. This research develops an automated deep learning-based pothole detection and mapping tool using the fusion of visible and thermal images. Visible images alone often fail in poor lighting or adverse weather conditions, whereas thermal images offer robust detection but lack texture details. Integrating both image types enhanced detection accuracy. The authors created a database of geotagged and labeled trios of visible, thermal, and fused images using a low-cost FLIR ONE thermal camera connected to a smartphone. Three machine-learning algorithms were proposed and compared: Anisotropic Diffusion Fusion (ADF) + Mask R-CNN, RTFNet, and RTFNet with Enhancement Parameters (EPs). The RTFNet method achieved the best F1-score of 93.7% in daytime and 90.9% in nighttime scenarios. A Bright-Dark detector was developed to optimize algorithm selection based on lighting conditions. Detected potholes were mapped using global positioning system (GPS) data, and the trained algorithm was packaged into a graphical user interface (GUI) tool that can be used by highway maintenance teams.

• Record URL:
  • https://www.ugpti.org/resources/reports/details.php?id=1205
• Record URL:
  • https://rosap.ntl.bts.gov/view/dot/78916
• Summary URL:
  • https://rosap.ntl.bts.gov/view/dot/78917
• Supplemental Notes:
  • This document was sponsored by the U.S. Department of Transportation, University Transportation Centers Program.
• Corporate Authors:

  Colorado State University, Fort Collins

  Department of Civil and Environmental Engineering
  Fort Collins, CO  United States  80525

  Mountain-Plains Consortium

  North Dakota State University
  Fargo, ND  United States  58108

  Office of the Assistant Secretary for Research and Technology

  University Transportation Centers Program
  Department of Transportation
  Washington, DC  United States  20590
• Authors:
  • Jia, Gaofeng
  • Chen, Wei-Hsiang
• Publication Date: 2024-9

Language

• English

Media Info

• Media Type: Digital/other
• Edition: Final Report
• Features: Figures; Photos; References; Tables;
• Pagination: 53p

Subject/Index Terms

• TRT Terms: Data fusion; Detection and identification technologies; Image analysis; Machine learning; Pavement maintenance; Potholes; Thermal imagery
• Subject Areas: Data and Information Technology; Highways; Maintenance and Preservation; Pavements;

Filing Info

• Accession Number: 01940368
• Record Type: Publication
• Report/Paper Numbers: MPC-620, MPC-24-559
• Files: UTC, NTL, TRIS, USDOT
• Created Date: Dec 19 2024 2:14PM