A Multimodal Approach for Monitoring Driving Behavior and Emotions

Studies have indicated that emotions can significantly be influenced by environmental factors; these factors can also significantly influence drivers’ emotional state and, accordingly, their driving behavior. Furthermore, as the demand for autonomous vehicles is expected to significantly increase within the next decade, a proper understanding of drivers’/passengers’ emotions, behavior, and preferences will be needed in order to create an acceptable level of trust with humans. This paper proposes a novel semi-automated approach for understanding the effect of environmental factors on drivers’ emotions and behavioral changes through a naturalistic driving study. This setup includes a frontal road and facial camera, a smart watch for tracking physiological measurements, and a Controller Area Network (CAN) serial data logger. The results suggest that the driver’s affect is highly influenced by the type of road and the weather conditions, which have the potential to change driving behaviors. For instance, when the research defines emotional metrics as valence and engagement, results reveal there exist significant differences between human emotion in different weather conditions and road types. Participants’ engagement was higher in rainy and clear weather compared to cloudy weather. More-over, engagement was higher on city streets and highways compared to one-lane roads and two-lane highways.

• Record URL:
  https://doi.org/10.31979/mti.2020.1928
• Record URL:
  https://scholarworks.sjsu.edu/cgi/viewcontent.cgi?article=1311&context=mti_publications
• Record URL:
  https://merritt.cdlib.org/api/presign-file/ark%253A%252F13030%252Fm5t20g9b/1/producer%252FCA-MTI-1928.pdf
• Supplemental Notes:
  • A version of this paper was also presented at the 2019 Transportation Research Board 98th Annual Meeting.
• Corporate Authors:

  Mineta Transportation Institute

  College of Business
  San José State University
  San Jose, CA  United States  95192-0219

  State of California SB1 2017/2018

  Trustees of the California State University. Sponsored Programs Administration
  401 Golden Shore, 5th Floor
  Long Beach, CA  United States  90802

  California State University Transportation Consortium

  San José State University
  San Jose, CA  United States 

  California State University, Long Beach

  ,    
• Authors:
  • Balali, Vahid
  • 0000-0002-5553-7599
• Publication Date: 2020-7

Language

• English

Media Info

• Media Type: Digital/other
• Edition: Final Report
• Features: Bibliography; Figures;
• Pagination: 20p

Subject/Index Terms

• TRT Terms: Automatic data collection systems; Behavior; Computer vision; Driver monitoring; Drivers; Emotions; Human machine systems; In vehicle sensors
• Subject Areas: Data and Information Technology; Highways; Planning and Forecasting; Safety and Human Factors;

Filing Info

• Accession Number: 01752854
• Record Type: Publication
• Report/Paper Numbers: 20-27, CA-MTI-1928
• Contract Numbers: ZSB12017-SJAUX
• Files: BTRIS, TRIS
• Created Date: Sep 24 2020 6:24PM