Automated Detection and Classification of Pavement Distresses using 3D Pavement Surface Images and Deep Learning

Pavement distresses lead to pavement deterioration and failure. Accurate identification and classification of distresses helps agencies evaluate the condition of their pavement infrastructure and assists in decision-making processes on pavement maintenance and rehabilitation. The state of the art is automated pavement distress detection using vision-based methods. This study implements two deep learning techniques, Faster Region-based Convolutional Neural Networks (R-CNN) and You Only Look Once (YOLO) v3, for automated distress detection and classification of high resolution (1,800?×?1,200) three-dimensional (3D) asphalt and concrete pavement images. The training and validation dataset contained 625 images that included distresses manually annotated with bounding boxes representing the location and types of distresses and 798 no-distress images. Data augmentation was performed to enable more balanced representation of class labels and prevent overfitting. YOLO and Faster R-CNN achieved 89.8% and 89.6% accuracy respectively. Precision-recall curves were used to determine the average precision (AP), which is the area under the precision-recall curve. The AP values for YOLO and Faster R-CNN were 90.2% and 89.2% respectively, indicating strong performance for both models. Receiver operating characteristic (ROC) curves were also developed to determine the area under the curve, and the resulting area under the curve values of 0.96 for YOLO and 0.95 for Faster R-CNN also indicate robust performance. Finally, the models were evaluated by developing confusion matrices comparing our proposed model with manual quality assurance and quality control (QA/QC) results performed on automated pavement data. A very high level of match to manual QA/QC, namely 97.6% for YOLO and 96.9% for Faster R-CNN, suggest the proposed methodology has potential as a replacement for manual QA/QC.

  • Record URL:
  • Availability:
  • Supplemental Notes:
    • Because of the proprietary nature of the data and license agreements, the data cannot be made freely available. Requests for access to the data will be granted subject to approval by the Iowa DOT and CTRE. Requests for data should be made to the authors of this paper. © National Academy of Sciences: Transportation Research Board 2021.
  • Authors:
    • Ghosh, Rohit
    • Smadi, Omar
  • Publication Date: 2021


  • English

Media Info

Subject/Index Terms

Filing Info

  • Accession Number: 01764454
  • Record Type: Publication
  • Report/Paper Numbers: TRBAM-21-03383
  • Files: TRIS, TRB, ATRI
  • Created Date: Dec 23 2020 11:05AM