OATAO - Open Archive Toulouse Archive Ouverte Open Access Week

Temporal relation algebra for audiovisual content analysis

Ibrahim, Zein Al Abidin and Ferrané, Isabelle and Joly, Philippe Temporal relation algebra for audiovisual content analysis. (2018) Journal of Multimedia Tools and Applications, 78 (309). 1-42. ISSN 1380-7501

(Document in English)

PDF (Author's version) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader

Official URL: https://doi.org/10.1007/s11042-018-6771-1


The context of this work is to characterize the content and the structure of audiovisual documents by analysing the temporal relationships between basic events resulted from different segmentations of the same document. For this objective, we need to represent and reason about time. We propose a parametric representation of temporal relation between segments (points or intervals) in which the parameters are used to characterize the relationship between two non-convex intervals corresponding to two segmentations in the video analysis domain. The relationship is represented by a co-occurrences matrix noted as Temporal Relation Matrix (TRM). Each document is represented by a set of TRMs computed between each couple of segmentations of the same document using different features. The TRMs are analysed later to detect semantic events, highlight clues about the video content structure or to classify documents based on their types. For higher-level semantic events and documents’ structure, we needed to apply some operations on the basic temporal relations and TRMs such as composition, disjunction, complement, intersection, etc. These operations brought to light more complex patterns; e.g. event 1 occurs at the same time of event 2 followed by event 3. In the work presented in this paper, we define a temporal relation algebra including its set of operations based on the parametric representation and TRM defined above. Several experimentations have been done on different audio and video documents to show the efficiency of the proposed representation and the defined operations for audiovisual content analysing.

Item Type:Article
Additional Information:Thanks to Springer editor. This papers appears in Volume 78 of Journal of Multimedia Tools and Applications ISSN : 1380-7501 The original PDF is available at: https://link.springer.com/article/10.1007%2Fs11042-018-6771-1
HAL Id:hal-02089343
Audience (journal):International peer-reviewed journal
Uncontrolled Keywords:
Institution:French research institutions > Centre National de la Recherche Scientifique - CNRS (FRANCE)
Université de Toulouse > Institut National Polytechnique de Toulouse - Toulouse INP (FRANCE)
Other partners > Lebanese University - LU (LEBANON)
Université de Toulouse > Université Toulouse III - Paul Sabatier - UT3 (FRANCE)
Université de Toulouse > Université Toulouse - Jean Jaurès - UT2J (FRANCE)
Université de Toulouse > Université Toulouse 1 Capitole - UT1 (FRANCE)
Laboratory name:
Deposited On:18 Mar 2019 15:49

Repository Staff Only: item control page