Semantic Image Retrieval and Clustering for Supporting Domain-Specific Bridge Component and Defect Classification

Automatic defect detection and classification from images is becoming increasingly important for bridge deterioration prediction and maintenance decision making. The majority of existing defect detection efforts have developed their datasets for training a machine-learning algorithm for detection/classification. However, the majority of these datasets suffer from two main limitations. First, most of the datasets are relatively small in size, which is not sufficient to build a well-trained, accurate image classifier. Second, most of the datasets lack the needed variety in scenes, angles, and backgrounds, which is not adaptable to different application contexts and environments. To address these limitations, this paper proposes a semantic image retrieval and clustering method to collect a large size of relevant images with various scenes, angles, and backgrounds from the Web and cluster these images for supporting domain-specific bridge component and defect detection. The proposed method includes three primary steps: query formation and image search and retrieval, image representation, and image clustering. First, a set of domain-specific words were extracted from bridge inspection documents and used as queries for retrieving a large number of images from the Web. Second, a transfer learning technique was used to transfer knowledge in a pre-trained model for general image classification to the bridge component and defect-related image clustering task. A deep convolutional neural network (CNN) with pre-trained weights was used to extract the visual features of the images for image representation. Third, a clustering technique was used to cluster the images based on the extracted features. The performance of the proposed method was evaluated using the silhouette coefficient. The evaluation results show that the proposed method is promising.

• Record URL:
  • https://doi.org/10.1061/9780784482858.087
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/isbn/9780784482858
• Supplemental Notes:
  • © 2020 American Society of Civil Engineers.
• Corporate Authors:

  American Society of Civil Engineers

  1801 Alexander Bell Drive
  Reston, VA  United States  20191-4400
• Authors:
  • Liu, Peter Cheng-Yang
  • El-Gohary, Nora
• Conference:
  • Construction Research Congress 2020
  • Location: Tempe Arizona, United States
  • Date: 2020-3-8 to 2020-3-10
• Publication Date: 2020

Language

• English

Media Info

• Media Type: Web
• Features: References;
• Pagination: pp 809-818
• Monograph Title: Construction Research Congress 2020: Infrastructure Systems and Sustainability

Subject/Index Terms

• TRT Terms: Bridges; Classification; Cluster analysis; Image analysis; Image processing; Machine learning; Neural networks; Structural deterioration and defects
• Subject Areas: Bridges and other structures; Highways; Maintenance and Preservation; Planning and Forecasting;

Filing Info

• Accession Number: 01760733
• Record Type: Publication
• ISBN: 9780784482858
• Files: TRIS, ASCE
• Created Date: Dec 21 2020 10:06AM