Towards soundpainting gesture recognition

Abstract

In this article, we describe our recent research activities on gesture recognition for soundpainting applications. Soundpainting is a multidisciplinary live composing sign language for musicians, actors, dancers, and visual artists. These gestures are produced by a soundpainter, which plays the role of a conductor, in order to lead a live performance. Soundpainting gestures are normalized and well defined, thus they are a very interesting case study in automatic gesture recognition. We describe a first gesture recognition system based on hidden Markov Models. We also report on the creation of a pilot corpus of soundpainting RGB/depth videos. The use of a computer could have many interesting applications listed in the paper. These applications are not limited to live performance, in which the computer would act as a performer. It could also help to investigate the balance between improvisation and planned creation in the particular context of soundpainting.