Depth Video-Based Secondary Action Recognition in Vehicles via Convolutional Neural Network and Bidirectional Long Short-Term Memory with Spatial Enhanced Attention Mechanism

Secondary actions in vehicles are activities that drivers engage in while driving that are not directly related to the primary task of operating the vehicle. Secondary Action Recognition (SAR) in drivers is vital for enhancing road safety and minimizing accidents related to distracted driving. It also plays an important part in modern car driving systems such as Advanced Driving Assistance Systems (ADASs), as it helps identify distractions and predict the driver's intent. Traditional methods of action recognition in vehicles mostly rely on RGB videos, which can be significantly impacted by external conditions such as low light levels. In this research, the authors introduce a novel method for SAR. The authors' approach utilizes depth-video data obtained from a depth sensor located in a vehicle. The authors' methodology leverages the Convolutional Neural Network (CNN), which is enhanced by the Spatial Enhanced Attention Mechanism (SEAM) and combined with Bidirectional Long Short-Term Memory (Bi-LSTM) networks. This method significantly enhances action recognition ability in depth videos by improving both the spatial and temporal aspects. The authors conduct experiments using K-fold cross validation, and the experimental results show that on the public benchmark dataset Drive&Act, the authors' proposed method shows significant improvement in SAR compared to the state-of-the-art methods, reaching an accuracy of about 84% in SAR in depth videos.

• Record URL:
  • http://dx.doi.org/10.3390/s24206604
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/issn/14248220
• Supplemental Notes:
  • Copyright: © 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
• Authors:
  • Shao, Weirong
  • Bouazizi, Mondher
  • Tomoaki, Ohtuski
• Publication Date: 2024

Language

• English

Media Info

• Media Type: Web
• Features: Figures; References; Tables;
• Pagination: 6604
• Serial:
  • Sensors
  • Volume: 24
  • Issue Number: 20
  • Publisher: MDPI AG
  • ISSN: 1424-8220
  • Serial URL: http://www.mdpi.com/journal/sensors
• Publication flags:

  Open Access (libre)

Subject/Index Terms

• TRT Terms: Attention; Distracted driving; Highway safety; Machine learning; Neural networks; Sensors; Video
• Subject Areas: Highways; Safety and Human Factors; Vehicles and Equipment;

Filing Info

• Accession Number: 01937241
• Record Type: Publication
• Files: TRIS
• Created Date: Nov 18 2024 2:21PM