Hebbian Learning of Recurrent Connections: A Geometrical Perspective

Abstract : We show how a Hopfield network with modifiable recurrent connections undergoing slow Hebbian learning can extract the underlying geometry of an input space. First, we use a slow and fast analysis to derive an averaged system whose dynamics derives from an energy function and therefore always converges to equilibrium points. The equilibria reflect the correlation structure of the inputs, a global object extracted through local recurrent interactions only. Second, we use numerical methods to illustrate how learning extracts the hidden geometrical structure of the inputs. Indeed, multidimensional scaling methods make it possible to project the final connectivity matrix onto a Euclidean distance matrix in a high-dimensional space, with the neurons labeled by spatial position within this space. The resulting network structure turns out to be roughly convolutional. The residual of the projection defines the nonconvolutional part of the connectivity, which is minimized in the process. Finally, we show how restricting the dimension of the space where the neurons live gives rise to patterns similar to cortical maps. We motivate this using an energy efficiency argument based on wire length minimization. Finally, we show how this approach leads to the emergence of ocular dominance or orientation columns in primary visual cortex via the self-organization of recurrent rather than feedforward connections. In addition, we establish that the nonconvolutional (or long-range) connectivity is patchy and is co-aligned in the case of orientation learning.
Type de document :
Article dans une revue
Neural Computation, Massachusetts Institute of Technology Press (MIT Press), 2012, 24 (9), pp.2346--2383. 〈10.1162/NECO_a_00322〉
Liste complète des métadonnées

https://hal.inria.fr/hal-00845588
Contributeur : Pierre Kornprobst <>
Soumis le : mercredi 17 juillet 2013 - 14:13:33
Dernière modification le : jeudi 3 mai 2018 - 13:32:58

Lien texte intégral

Identifiants

Collections

Citation

Mathieu Galtier, Olivier Faugeras, Paul Bressloff. Hebbian Learning of Recurrent Connections: A Geometrical Perspective. Neural Computation, Massachusetts Institute of Technology Press (MIT Press), 2012, 24 (9), pp.2346--2383. 〈10.1162/NECO_a_00322〉. 〈hal-00845588〉

Partager

Métriques

Consultations de la notice

295