Improving visual question answering for bridge inspection by pre-training with external data of image–text pairs

This paper explores the application of visual question answering (VQA) in bridge inspection using recent advancements in multimodal artificial intelligence (AI) systems. VQA involves an AI model providing natural language answers to questions about the content of an input image. However, applying VQA to bridge inspection poses challenges due to the high cost of creating training data that requires expert knowledge. To address this, the authors propose leveraging existing bridge inspection reports, which already include image–text pairs, as external knowledge to enhance VQA performance. The authors' approach involves training the model on a large collection of image–text pairs, followed by fine-tuning it on a limited amount of training data specifically designed for the VQA task. The results demonstrate a significant improvement in VQA accuracy using this approach. These findings highlight the potential of AI models for VQA as valuable tools for assessing the condition of bridges.

• Record URL:
  • https://doi.org/10.1111/mice.13086
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/issn/10939687
• Supplemental Notes:
  • © 2023 The Authors. Computer-Aided Civil and Infrastructure Engineering published by Wiley Periodicals LLC on behalf of Editor.
• Authors:
  • Kunlamai, Thannarot
  • Yamane, Tatsuro
  • Suganuma, Masanori
  • Chun, Pang-jo
  • Okatani, Takayaki
• Publication Date: 2024-2-1

Language

• English

Media Info

• Media Type: Web
• Features: Figures; Photos; References; Tables;
• Pagination: pp 345-361
• Serial:
  • Computer-Aided Civil and Infrastructure Engineering
  • Volume: 39
  • Issue Number: 3
  • Publisher: Elsevier
  • ISSN: 1093-9687
  • Serial URL: https://www.sciencedirect.com/journal/computer-aided-civil-and-infrastructure-engineering
• Publication flags:

  Open Access (libre)

Subject/Index Terms

• TRT Terms: Artificial intelligence; Bridge management systems; Datasets; Inspection; Reports
• Subject Areas: Bridges and other structures; Data and Information Technology; Highways; Maintenance and Preservation;

Filing Info

• Accession Number: 01915940
• Record Type: Publication
• Files: TRIS
• Created Date: Apr 19 2024 9:38AM