Stochastic Coherence Over Attention Trajectory For Continuous Learning In Video Streams - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2022

Stochastic Coherence Over Attention Trajectory For Continuous Learning In Video Streams

Résumé

Devising intelligent agents able to live in an environment and learn by observing the surroundings is a longstanding goal of Artificial Intelligence. From a bare Machine Learning perspective, challenges arise when the agent is prevented from leveraging large fully-annotated dataset, but rather the interactions with supervisory signals are sparsely distributed over space and time. This paper proposes a novel neural-network-based approach to progressively and autonomously develop pixel-wise representations in a video stream. The proposed method is based on a human-like attention mechanism that allows the agent to learn by observing what is moving in the attended locations. Spatio-temporal stochastic coherence along the attention trajectory, paired with a contrastive term, leads to an unsupervised learning criterion that naturally copes with the considered setting. Differently from most existing works, the learned representations are used in open-set class-incremental classification of each frame pixel, relying on few supervisions. Our experiments leverage 3D virtual environments and they show that the proposed agents can learn to distinguish objects just by observing the video stream. Inheriting features from state-of-the art models is not as powerful as one might expect.

Dates et versions

hal-03874819 , version 1 (28-11-2022)

Identifiants

Citer

Matteo Tiezzi, Simone Marullo, Lapo Faggi, Enrico Meloni, Alessandro Betti, et al.. Stochastic Coherence Over Attention Trajectory For Continuous Learning In Video Streams. 2022. ⟨hal-03874819⟩
12 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More