Global-Local-Feature-Fused Driver Speech Emotion Detection for Intelligent Cockpit in Automated Driving

Affective interaction between the intelligent cockpit and humans is becoming an emerging topic full of opportunities. Robust recognition of the driver's emotions is the first step for affective interaction, and the intelligent cockpit recognizes emotions through the driver's speech, which has a wide range of technical application potential. In this paper, the authors first proposed a multi-feature fusion parallel structure speech emotion recognition network, which complementarily fuses the global acoustic features and local spectral features of the entire speech. Second, the authors designed and conducted the speech data collection under the driver's emotion and established the driver's speech emotion (SpeechEmo) dataset in the dynamic driving environment including 40 participants. Finally, the proposed model was validated on the SpeechEmo and public datasets, and quantitative analysis was carried out. It was found that the proposed model achieved advanced recognition performance, and the ablation experiments verified the importance of different components of the model. The proposed model and dataset are beneficial to the realization of human-vehicle affective interaction in intelligent cockpits in the future toward a better human experience.

• Record URL:
  http://dx.doi.org/10.1109/TIV.2023.3259988
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/issn/23798858
• Supplemental Notes:
  • Copyright © 2023, IEEE.
• Authors:
  • Li, Wenbo
  • Xue, Jiyong
  • Tan, Ruichen
  • Wang, Cong
  • Deng, Zejian
  • Li, Shen
  • Guo, Gang
  • Cao, Dongpu
• Publication Date: 2023-4

Language

• English

Media Info

• Media Type: Web
• Features: References;
• Pagination: pp 2684-2697
• Serial:
  • IEEE Transactions on Intelligent Vehicles
  • Volume: 8
  • Issue Number: 4
  • Publisher: Institute of Electrical and Electronics Engineers (IEEE)
  • ISSN: 2379-8858
  • Serial URL: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=7274857

Subject/Index Terms

• TRT Terms: Driver information systems; Emotions; Human machine systems; Intelligent vehicles; Machine learning; Speech recognition
• Subject Areas: Data and Information Technology; Highways; Operations and Traffic Management; Vehicles and Equipment;

Filing Info

• Accession Number: 01897214
• Record Type: Publication
• Files: TRIS
• Created Date: Oct 23 2023 4:52PM