M. Elena-acevedo-mosqueda, C. Y. Márquez, and M. , Bidirectional associative memories, ACM Computing Surveys, vol.45, issue.2, pp.1-1830, 2013.
DOI : 10.1145/2431211.2431217

D. J. Amit, Modeling brain function?the world of attractor neural networks, 1989.

K. J. Astrom, Theory and Applications of Adaptive Control, IFAC Proceedings Volumes, vol.14, issue.2, pp.471-486, 1983.
DOI : 10.1016/S1474-6670(17)63572-9

D. Balduzzi and M. Ghifary, Strongly-typed recurrent neural networks. CoRR, abs/1602, 2016.

Y. Bengio, Learning Deep Architectures for AI, Foundations and Trends?? in Machine Learning, vol.2, issue.1, 2009.
DOI : 10.1561/2200000006
URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.150.527

Y. Bengio, A. Courville, and P. Vincent, Representation Learning: A Review and New Perspectives, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.35, issue.8, pp.1798-1828, 2012.
DOI : 10.1109/TPAMI.2013.50
URL : http://arxiv.org/pdf/1206.5538

Y. Bengio and Y. Lecun, Scaling learning algorithms towards ai, Large-Scale Kernel Machines, 2007.

Y. Bengio, D. Lee, J. Bornschein, and Z. Lin, Towards biologically plausible deep learning. arXiv preprint, p.10, 2016.

N. Bertschinger and T. Natschläger, Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks, Neural Computation, vol.7, issue.7, pp.1413-1436, 2004.
DOI : 10.1126/science.274.5293.1724

B. Cessac, A VIEW OF NEURAL NETWORKS AS DYNAMICAL SYSTEMS, International Journal of Bifurcation and Chaos, vol.97, issue.06, pp.1585-1629, 1920.
DOI : 10.1162/089976602760408008
URL : https://hal.archives-ouvertes.fr/inria-00534326

B. Cessac, A discrete time neural network model with spiking neurons, Journal of Mathematical Biology, vol.18, issue.26, pp.311-345, 2008.
DOI : 10.1017/CBO9781107359987
URL : https://hal.archives-ouvertes.fr/inria-00530115

B. Cessac, H. Paugam-moisy, and T. Viéville, Overview of facts and issues about neural coding by spikes, Journal of Physiology-Paris, vol.104, issue.1-2, pp.5-18, 2010.
DOI : 10.1016/j.jphysparis.2009.11.002
URL : https://hal.archives-ouvertes.fr/inria-00407915

B. Cessac, R. Salas, and T. Viéville, Using event-based metric for event-based neural network weight adjustment. page 18 pp, 2000.
URL : https://hal.archives-ouvertes.fr/hal-00755345

D. S. Chen and R. C. Jain, A robust backpropagation learning algorithm for function approximation, IEEE Transactions on Neural Networks, vol.5, issue.3, pp.467-479, 1994.
DOI : 10.1109/72.286917

K. Cho, B. Van-merrienboer, Ç. Gülçehre, F. Bougares, H. Schwenk et al., Learning Phrase Representations using RNN Encoder???Decoder for Statistical Machine Translation, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1078.
DOI : 10.3115/v1/D14-1179
URL : https://hal.archives-ouvertes.fr/hal-01433235

R. Cofre and B. Cessac, Exact computation of the maximum-entropy potential of spiking neural-network models, Physical Review E, vol.36, issue.5, 2014.
DOI : 10.1007/s00285-010-0358-4

Y. Le-cun, A Theoretical Framework for Back-Propagation, 1988.

L. Deng, Deep Learning: Methods and Applications. Foundations and Trends® in Signal Processing, pp.197-387
DOI : 10.1561/2000000039
URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.650.4684

J. L. Elman, Finding Structure in Time, Cognitive Science, vol.49, issue.2, pp.179-211, 1990.
DOI : 10.1007/BF00308682

F. Thalita and . Drumond, Thierry Viéville, and Frédéric Alexandre Alexandre. Not-so-big data deep learning: a review, 2017.

F. Thalita and . Drumond, Thierry Viéville, and Frédéric Alexandre. From shortcuts to architecture optimization in deep-learning, 2017.

C. Farabet, C. Couprie, L. Najman, and Y. Lecun, Learning Hierarchical Features for Scene Labeling, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.35, issue.8, pp.1915-1929, 0578.
DOI : 10.1109/TPAMI.2012.231
URL : https://hal.archives-ouvertes.fr/hal-00742077

Y. Freund, R. Iyer, E. Robert, Y. Schapire, and . Singer, An efficient boosting algorithm for combining preferences. The Journal of machine learning research, pp.933-969, 2003.

A. Felix, N. N. Gers, J. Schraudolph, and . Schmidhuber, Learning precise timing with lstm recurrent networks, J. Mach. Learn. Res, vol.3, pp.115-143, 2003.

I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, 2016.

K. He, X. Zhang, S. Ren, and J. Sun, Deep Residual Learning for Image Recognition, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.770-778, 2016.
DOI : 10.1109/CVPR.2016.90
URL : http://arxiv.org/pdf/1512.03385

E. Geoffrey, J. L. Hinton, and . Mcclelland, Learning representations by recirculation, Neural Information Processing Systems, pp.358-366, 1988.

S. Hochreiter and J. Schmidhuber, Long Short-Term Memory, Neural Computation, vol.4, issue.8, pp.1735-1780, 1997.
DOI : 10.1016/0893-6080(88)90007-X

S. Hochreiter and J. Schmidhuber, Long Short-Term Memory, Neural Computation, vol.4, issue.8, pp.1735-1780, 1997.
DOI : 10.1016/0893-6080(88)90007-X

J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proc. National Academy of Sciences, pp.2554-2558, 1982.

. Guang-bin, Q. Huang, C. Zhu, and . Siew, Extreme learning machine: Theory and applications, Neurocomputing, vol.70, issue.1, pp.489-501, 2006.

J. Håstad and M. Goldmann, On the power of small-depth threshold circuits, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science, pp.113-129, 1991.
DOI : 10.1109/FSCS.1990.89582

H. Jaeger, Adaptive nonlinear system identification with Echo State Networks, Advances in Neural Information Processing Systems, pp.593-600, 2002.

M. I. Jordan, Attractor dynamics and parallelism in a connectionist sequential machine, Proceedings of the 8th Annular Conference Cognitive Science Society, pp.531-546, 1986.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, ImageNet classification with deep convolutional neural networks, Advances in Neural Information Processing Systems 25, pp.1097-1105, 2012.
DOI : 10.1162/neco.2009.10-08-881
URL : http://dl.acm.org/ft_gateway.cfm?id=3065386&type=pdf

Y. Lecun, Y. Bottou, P. Bengio, and . Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE, pp.2278-2324, 1998.
DOI : 10.1109/5.726791
URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.138.1115

R. Legenstein and W. Maass, Edge of chaos and prediction of computational performance for neural circuit models, Echo State Networks and Liquid State Machines, pp.323-334, 2007.
DOI : 10.1016/j.neunet.2007.04.017

W. Maass, T. Natschläger, and H. Markram, Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations, Neural Computation, vol.7, issue.11, pp.2531-2560, 2002.
DOI : 10.1038/35009102

J. Martens and I. Sutskever, Learning Recurrent Neural Networks with Hessian-Free Optimization
DOI : 10.1007/978-3-642-35289-8_27
URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.296.4704

M. Nielsen, Neural Networks and Deep Learning, 2015.

R. C. O-'reilly, Biologically Plausible Error-Driven Learning Using Local Activation Differences: The Generalized Recirculation Algorithm, Neural Computation, vol.13, issue.2, pp.895-938, 1996.
DOI : 10.1126/science.150.3700.1187

H. Paugam-moisy, R. Martinez, and S. Bengio, Delay learning and polychronization for reservoir computing, Neurocomputing, vol.71, issue.7-9, pp.1143-1158, 2008.
DOI : 10.1016/j.neucom.2007.12.027
URL : https://hal.archives-ouvertes.fr/hal-01500331

F. J. Pineda, Generalization of back-propagation to recurrent neural networks, Physical Review Letters, vol.81, issue.19, pp.2229-2232, 1987.
DOI : 10.1073/pnas.81.10.3088

T. Poggio, H. Mhaskar, L. Rosasco, B. Miranda, and Q. Liao, Why and when can deep-but not shallow-networks avoid the curse of dimensionality: A review, International Journal of Automation and Computing, vol.37, issue.3, 2017.
DOI : 10.1007/BF00355455

H. Rostro-gonzalez, B. Cessac, and T. Viéville, Exact spike-train reproduction with a neural network model, Journal of Computational Neuroscience, 2010.

J. Schmidhuber, Deep learning in neural networks: An overview, Neural Networks, vol.61, pp.85-117, 2015.
DOI : 10.1016/j.neunet.2014.09.003
URL : http://arxiv.org/pdf/1404.7828

P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus et al., Overfeat: Integrated recognition, localization and detection using convolutional networks, 2013.

S. , M. Sherman, and R. W. Guillery, The role of the thalamus in the flow of information to the cortex, Philosophical Transactions of the Royal Society B: Biological Sciences, vol.357, pp.1695-1708, 1428.

T. Hava, E. D. Siegelmann, and . Sontag, Turing Computability With Neural Nets, 1991.

K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, International Conference on Learning Representations (ICRL), pp.1-14, 2015.

G. Stuart, N. Spruston, B. Sakmann, and M. Häusser, Action potential initiation and backpropagation in neurons of the mammalian CNS, Trends in Neurosciences, vol.20, issue.3, pp.125-131, 1997.
DOI : 10.1016/S0166-2236(96)10075-8

C. Szegedy, S. Ioffe, V. Vanhoucke, and A. Alemi, Inception-v4, inception-resnet and the impact of residual connections on learning, Arxiv, p.12, 2016.

C. Szegedy, W. Liu, Y. Jia, and P. Sermanet, Going deeper with convolutions, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
DOI : 10.1109/CVPR.2015.7298594
URL : http://arxiv.org/abs/1409.4842

M. Topalidou, A. Leblois, T. Boraud, and N. P. Rougier, A long journey into reproducible computational neuroscience, Frontiers in Computational Neuroscience, vol.515, p.30, 2015.
DOI : 10.1038/515151a
URL : https://hal.archives-ouvertes.fr/hal-01109483

J. A. Tropp, Greed is Good: Algorithmic Results for Sparse Approximation, IEEE Transactions on Information Theory, vol.50, issue.10, pp.2231-2242, 2004.
DOI : 10.1109/TIT.2004.834793
URL : http://authors.library.caltech.edu/9035/1/TROieeetit04a.pdf

J. A. Tropp, Just relax: Convex programming methods for subset selection and sparse approximation, Texas Institute for Computational Engineering and Sciences, 2004.
DOI : 10.1109/tit.2005.864420
URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.184.2957

J. C. Vasquez, T. Viéville, and B. Cessac, Parametric Estimation of Gibbs distributions as general Maximum-entropy models for the analysis of spike train statistics This work corresponds to an extended and revisited version of a previous Arxiv preprint, submitted to HAL as http, 2011.

D. Verstraeten, B. Schrauwen, M. D. Haene, and D. Stroobandt, An experimental unification of reservoir computing methods, Neural Networks, vol.20, issue.3, pp.391-403, 2007.
DOI : 10.1016/j.neunet.2007.04.003

X. Xie and H. S. Seung, Equivalence of Backpropagation and Contrastive Hebbian Learning in a Layered Network, Neural Computation, vol.1, issue.2, pp.441-454, 2003.
DOI : 10.1038/331679a0

D. Matthew, R. Zeiler, and . Fergus, Visualizing and understanding convolutional networks, Lecture Notes in Computer Science, vol.8689, pp.818-833, 2014.

C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals, Understanding deep learning requires rethinking generalization, 2016.