Closed-Loop Brain Machine Interface System for In-Vehicle Function Controls Using Head-Up Display and Deep Learning Algorithm

Human-vehicle interfaces have evolved with technologies such as touchscreens, voice commands, and advanced steering wheels with control panels. However, these technologies can increase the driver’s cognitive workload and distraction, potentially compromising road safety. Brain-machine interfaces (BMI) offer an alternative way of controlling in-vehicle features with minimal distraction. This study presents a closed-loop steady-state visual evoked potentials (SSVEP) based BMI for controlling in-vehicle features via a windshield head-up display to control multiple vehicle features such as music, temperature, settings, and navigation system. The custom software synchronizes visual stimuli, processes electroencephalogram (EEG) signals, and selects target icons in real-time. A convolutional neural network (CNN) based on the SE-ResNet architecture efficiently detects SSVEPs and the driver’s intended target icon in under 1 second. Comparative evaluations against traditional and deep learning methods using time and frequency features show superior performance. With just 225 seconds of calibration data, the proposed compact deep-learning model enables drivers to adjust their environment within 0.5 seconds, achieving an Information Transfer Rate (ITR) of 93.43±7.26 bits/min. This SSVEP-based BMI system demonstrates multi-layer in-vehicle feature control with ten human participants, highlighting its potential to enhance road safety by allowing drivers to make adjustments without diverting their attention from the road. The authors' code and dataset can be downloaded from https://github.com/hosseinhamidi92.

• Record URL:
  • http://dx.doi.org/10.1109/TITS.2023.3345855
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/oclc/41297384
• Supplemental Notes:
  • Copyright © 2024, IEEE.
• Authors:
  • Shishavan, Hossein Hamidi
  • Behzadi, Mohammad Mahdi
  • Lohan, Danny J
  • Dede, Ercan M
  • Kim, Insoo
• Publication Date: 2024-7

Language

• English

Media Info

• Media Type: Web
• Features: References;
• Pagination: pp 6594-6603
• Serial:
  • IEEE Transactions on Intelligent Transportation Systems
  • Volume: 25
  • Issue Number: 7
  • Publisher: Institute of Electrical and Electronics Engineers (IEEE)
  • ISSN: 1524-9050
  • Serial URL: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6979

Subject/Index Terms

• TRT Terms: Autonomous vehicles; Driver vehicle interfaces; Electroencephalography; Feedback control; Machine learning; Visualization
• Subject Areas: Data and Information Technology; Highways; Vehicles and Equipment;

Filing Info

• Accession Number: 01935514
• Record Type: Publication
• Files: TRIS
• Created Date: Oct 30 2024 11:08AM