OATAO - Open Archive Toulouse Archive Ouverte Open Access Week

A multi-modal perception based assistive robotic system for the elderly

Mollaret, Christophe and Mekonnen, Alhayat Ali and Lerasle, Frédéric and Ferrané, Isabelle and Pinquier, Julien and Boudet, Blandine and Rumeau, Pierre A multi-modal perception based assistive robotic system for the elderly. (2016) Computer Vision and Image Understanding, 149. 78-97. ISSN 1077-3142

(Document in English)

PDF (Author's version) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader

Official URL: https://doi.org/10.1016/j.cviu.2016.03.003


In this paper, we present a multi-modal perception based framework to realize a non-intrusive domestic assistive robotic system. It is non-intrusive in that it only starts interaction with a user when it detects the user’s intention to do so. All the robot’s actions are based on multi-modal perceptions which include user detection based on RGB-D data, user’s intention-for-interaction detection with RGB-D and audio data, and communication via user distance mediated speech recognition. The utilization of multi-modal cues in different parts of the robotic activity paves the way to successful robotic runs (94% success rate). Each presented perceptual component is systematically evaluated using appropriate dataset and evaluation metrics. Finally the complete system is fully integrated on the PR2 robotic platform and validated through system sanity check runs and user studies with the help of 17 volunteer elderly participants.

Item Type:Article
Audience (journal):International peer-reviewed journal
Uncontrolled Keywords:
Institution:French research institutions > Centre National de la Recherche Scientifique - CNRS (FRANCE)
Université de Toulouse > Institut National Polytechnique de Toulouse - Toulouse INP (FRANCE)
Université de Toulouse > Université Toulouse III - Paul Sabatier - UT3 (FRANCE)
Université de Toulouse > Université Toulouse - Jean Jaurès - UT2J (FRANCE)
Université de Toulouse > Université Toulouse 1 Capitole - UT1 (FRANCE)
Other partners > Centre Hospitalier Universitaire de Toulouse - CHU Toulouse (FRANCE)
Laboratory name:
Agence Nationale de Recherche - ANR (FRANCE)
Deposited On:27 Apr 2020 09:06

Repository Staff Only: item control page