Semantic Event Fusion of Different Visual Modality Concepts for Activity Recognition

Abstract : Combining multimodal concept streams from heterogeneous sensors is a problem superficially explored for activity recognition. Most studies explore simple sensors in nearly perfect conditions, where temporal synchronization is guaranteed. Sophisticated fusion schemes adopt problem-specific graphical representations of events that are generally deeply linked with their training data and focused on a single sensor. This paper proposes a hybrid framework between knowledge-driven and probabilistic-driven methods for event representation and recognition. It separates semantic modeling from raw sensor data by using an intermediate semantic representation, namely concepts. It introduces an algorithm for sensor alignment that uses concept similarity as a surrogate for the inaccurate temporal information of real life scenarios. Finally, it proposes the combined use of an ontology language, to overcome the rigidity of previous approaches at model definition, and a probabilistic interpretation for ontological models, which equips the framework with a mechanism to handle noisy and ambiguous concept observations, an ability that most knowledge-driven methods lack. We evaluate our contributions in multimodal recordings of elderly people carrying out IADLs. Results demonstrated that the proposed framework outperforms baseline methods both in event recognition performance and in delimiting the temporal boundaries of event instances.
Type de document :
Article dans une revue
IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers, 2016, 38, pp.1598 - 1611. 〈10.1109/TPAMI.2016.2537323〉
Liste complète des métadonnées

Littérature citée [37 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01399025
Contributeur : Carlos Crispim <>
Soumis le : vendredi 18 novembre 2016 - 14:59:57
Dernière modification le : mardi 24 juillet 2018 - 15:48:03

Fichier

crispim_etal_pami_spec_issu_v7...
Fichiers produits par l'(les) auteur(s)

Identifiants

Citation

Carlos Crispim-Junior, Vincent Buso, Konstantinos Avgerinakis, Georgios Meditskos, Alexia Briassouli, et al.. Semantic Event Fusion of Different Visual Modality Concepts for Activity Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers, 2016, 38, pp.1598 - 1611. 〈10.1109/TPAMI.2016.2537323〉. 〈hal-01399025〉

Partager

Métriques

Consultations de la notice

265

Téléchargements de fichiers

392