Implicit Hand Gestures in Aeronautics Cockpit as a Cue for Crew State and Workload Inference

This paper aims at improving advanced aeronautic cockpit by raising its awareness of the crew's state and workload level. The authors' approach is based on visual analysis of pilot's upper body movements. The authors define the term of "implicit gestures" and further observe its subclasses. The authors collected a simulator dataset of practical implicit gestures, annotated semi-automatically a dataset for Human pose estimation training, and the authors offer these datasets for public use. Based on experiments on this data, the authors propose a method for recognition of implicit gestures - full interactions, touch-and-go interactions, and unfinished gestures. The authors' approach is purely visual (no depth data, which are hardly usable in the cockpit environment due to regulations). This method is based on human pose estimation by a hierarchical approach named Pose machine whose subsampled output is used for recognition of implicit gesture presence from sequences of frames by random forest. The experiments show that the classification works reliably and the method is able to recognize these implicit gestures in the cockpit.


  • English

Media Info

  • Media Type: Web
  • Features: Figures; References;
  • Pagination: 632-637
  • Monograph Title: 18th International IEEE Conference on Intelligent Transportation Systems (ITSC 2015)

Subject/Index Terms

Filing Info

  • Accession Number: 01615198
  • Record Type: Publication
  • ISBN: 9781467365956
  • Files: TRIS
  • Created Date: May 2 2016 3:17PM