Mixed-initiative mission planning considering human operator state estimation based on physiological sensors

Abstract

Missions involving humans with automated systems become increasingly common and are subject to risk of failing due to human factors. In fact, missions workload may generate stress or mental fatigue increasing the accident risk. The idea of our project is to refine human-robot supervision by using data from physiological sensors(eye tracking and heart rate monitoring devices) giving information about the operator's state. The proof of concept mission consists of a ground robot, autonomous or controlled by a human operator, which has to fight fires that catch randomly. We proposed to use the planning framework called Partially Observable Markov Decision Process (POMDP) along with machine learning techniques to improve human-machine interactions by optimizing the decision of the mode (autonomous or controlled robot) and of the display of alarms in the form of visual stimuli.A dataset of demonstrations produced by remote volunteers through an online video game simulating the mission allows to learn a POMDP that infers human state and to optimize the associated strategy. Cognitive availability, current task, type of behavior, situation awareness or involvement in the mission are examples of studied human operator states. Finally, scores of the missions, consisting in the number of extinguished fires, will quantify the improvement made by using physiological data.