P. Baldi and L. Itti, Of bits and wows: A Bayesian theory of surprise with applications to attention, Neural Networks, vol.23, issue.5, pp.649-666
DOI : 10.1016/j.neunet.2009.12.007

A. Baranès and P. Oudeyer, R-IAC: Robust intrinsically motivated exploration and active learning. Autonomous Mental Development, IEEE Transactions on, vol.1, issue.3, pp.155-169, 2009.

A. Baranes and P. Y. Oudeyer, Active learning of inverse models with intrinsically motivated goal exploration in robots, Robotics and Autonomous Systems, vol.61, issue.1, pp.49-73, 2013.
DOI : 10.1016/j.robot.2012.05.008
URL : https://hal.archives-ouvertes.fr/hal-00788440

A. F. Baranes, P. Y. Oudeyer, and J. Gottlieb, The effects of task difficulty, novelty and the size of the search space on intrinsically motivated exploration, Frontiers in Neuroscience, vol.86, issue.6, 2014.
DOI : 10.1016/j.neuroimage.2013.08.019
URL : https://hal.archives-ouvertes.fr/hal-01087227

A. F. Baranes, P. Y. Oudeyer, and J. Gottlieb, Eye movements encode epistemic curiosity in human observers, Vis Res, 2015.
DOI : 10.1016/j.visres.2015.10.009
URL : http://doi.org/10.1016/j.visres.2015.10.009

A. Barto, M. Mirolli, and G. Baldassare, Novelty or surprise? Frontiers in Psychology, 2013.
DOI : 10.3389/fpsyg.2013.00907

J. Bisley and M. Goldberg, Attention, Intention, and Priority in the Parietal Lobe, Annual Review of Neuroscience, vol.33, issue.1, pp.1-21, 2010.
DOI : 10.1146/annurev-neuro-060909-152823

T. C. Blanchard, B. Y. Hayden, and E. S. Bromberg-martin, Orbitofrontal Cortex Uses Distinct Codes for Different Choice Attributes in Decisions Motivated by Curiosity, Neuron, vol.85, issue.3, pp.602-614, 2015.
DOI : 10.1016/j.neuron.2014.12.050

R. I. Brafman and M. Tennenholtz, R-max-a general polynomial time algorithm for near-optimal reinforcement learning, The Journal of Machine Learning Research, vol.3, pp.213-231, 2003.

E. S. Bromberg-martin and O. Hikosaka, Midbrain Dopamine Neurons Signal Preference for Advance Information about Upcoming Rewards, Neuron, vol.63, issue.1, pp.119-126, 2009.
DOI : 10.1016/j.neuron.2009.06.009

D. C. Castro and K. C. Berridge, Advances in the neurobiological bases for food ???liking??? versus ???wanting???, Physiology & Behavior, vol.136, pp.22-30, 2014.
DOI : 10.1016/j.physbeh.2014.05.022

N. Daddoua, M. Lopes, and J. Gottlieb, Intrinsically motivated visual exploration is sensitive to reward and uncertainty in non-human primates, Scientific Reports, 2016.

P. Dayan, Y. Niv, B. Seymour, and N. D. Daw, The misbehavior of value and the discipline of the will, Neural Networks, vol.19, issue.8, pp.1153-1160, 2006.
DOI : 10.1016/j.neunet.2006.03.002

S. B. Flagel, J. J. Clark, T. E. Robinson, L. Mayo, A. Czul et al., A selective role for dopamine in stimulus???reward learning, Nature, vol.76, issue.7328, pp.469-53, 2011.
DOI : 10.1038/nature09588

N. C. Foley, D. C. Jangraw, C. Peck, and J. Gottlieb, Novelty Enhances Visual Salience Independently of Reward in the Parietal Lobe, Journal of Neuroscience, vol.34, issue.23, pp.34-7947, 2014.
DOI : 10.1523/JNEUROSCI.4171-13.2014

K. Friston, F. Rigoli, D. Ognibene, C. Mathys, T. Fitzgerald et al., Active inference and epistemic value, Cognitive Neuroscience, vol.6, issue.2, pp.1-28, 2015.
DOI : 10.1016/j.neuron.2008.04.027

K. Friston, P. Schwartenbeck, T. Fitzgerald, M. Moutoussis, T. Behrens et al., The anatomy of choice: active inference and agency, Frontiers in Human Neuroscience, vol.7, p.598, 2013.
DOI : 10.3389/fnhum.2013.00598

J. Gottlieb, M. Hayhoe, O. Hikosaka, and A. Rangel, Attention, Reward, and Information Seeking, Journal of Neuroscience, vol.34, issue.46, pp.15497-154504, 2014.
DOI : 10.1523/JNEUROSCI.3270-14.2014

M. J. Gruber, B. D. Gelman, and C. Ranganath, States of Curiosity Modulate Hippocampus-Dependent Learning via the Dopaminergic Circuit, Neuron, vol.84, issue.2, pp.486-496, 2014.
DOI : 10.1016/j.neuron.2014.08.060

M. Hayhoe and D. Ballard, Modeling Task Control of Eye Movements, Current Biology, vol.24, issue.13, p.24, 2014.
DOI : 10.1016/j.cub.2014.05.020

C. Hickey, L. Chelazzi, and J. Theeuwes, Reward Changes Salience in Human Vision via the Anterior Cingulate, Journal of Neuroscience, vol.30, issue.33, pp.11096-11103, 2010.
DOI : 10.1523/JNEUROSCI.1026-10.2010

C. Hickey, L. Chelazzi, and J. Theeuwes, Reward Guides Vision when It's Your Thing: Trait Reward-Seeking in Reward-Mediated Visual Priming, PLoS ONE, vol.5, issue.11, p.14087, 2010.
DOI : 10.1371/journal.pone.0014087.t001

J. C. Horvitz, Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events, Neuroscience, vol.96, issue.4, pp.651-656, 2000.
DOI : 10.1016/S0306-4522(00)00019-1

L. Itti and P. Baldi, Bayesian surprise attracts human attention, Vision Research, vol.49, issue.10, pp.1295-1306, 2009.
DOI : 10.1016/j.visres.2008.09.007

S. Kakade and P. Dayan, Dopamine: generalization and bonuses, Neural Networks, vol.15, issue.4-6, pp.4-6, 2002.
DOI : 10.1016/S0893-6080(02)00048-5

M. J. Kang, M. Hsu, I. M. Krajbich, G. Loewenstein, S. M. Mcclure et al., The wick in the candle of learning: epistemic curiosity activates reward circuitry and enhances memory, Psychol Sci, issue.8, pp.20-963, 2009.

P. A. Laurent, The emergence of saliency and novelty responses from Reinforcement Learning principles, Neural networks : the official journal of the International Neural Network Society, pp.1493-1499, 2008.
DOI : 10.1016/j.neunet.2008.09.004

J. A. Litman, Curiosity as a feeling of interest and feeling of deprivation: the I/D model of curiosity, 2007.

M. Lopes, T. Lang, M. Toussaint, and P. Oudeyer, Exploration in model-based reinforcement learning by empirically estimating learning progress, Paper presented at the Neural Information Processing Systems, 2012.
URL : https://hal.archives-ouvertes.fr/hal-00755248

M. Lopes and L. Montesano, Active learning for autonomous intelligent agents: exploration, curiosity and interaction arXiv: 1403, p.1497, 2014.

M. Lopes and P. Oudeyer, Guest Editorial Active Learning and Intrinsically Motivated Exploration in Robots: Advances and Challenges, IEEE Transactions on Autonomous Mental Development, vol.2, issue.2, pp.65-69, 2010.
DOI : 10.1109/TAMD.2010.2052419
URL : https://hal.archives-ouvertes.fr/inria-00541788

M. Lopes and P. Oudeyer, The strategic student approach for life-long exploration and learning, 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL), 2012.
DOI : 10.1109/DevLrn.2012.6400807
URL : https://hal.archives-ouvertes.fr/hal-00755216

G. Lowenstein, The psychology of curiosity: A review and reinterpretation., Psychological Bulletin, vol.116, issue.1, pp.75-98, 1994.
DOI : 10.1037/0033-2909.116.1.75

C. Moulin-frier, S. M. Nguyen, and P. Oudeyer, Self-organization of early vocal development in infants and machines: the role of intrinsic motivation, Frontiers in Psychology, vol.4, 2013.
DOI : 10.3389/fpsyg.2013.01006
URL : https://hal.archives-ouvertes.fr/hal-00927940

S. M. Nguyen and P. Oudeyer, Socially guided intrinsic motivation for robot learning of motor skills, Autonomous Robots, vol.24, issue.3, 2013.
DOI : 10.1007/s10514-013-9339-y
URL : https://hal.archives-ouvertes.fr/hal-00936938

P. Oudeyer, F. Kaplan, and V. V. Hafner, Intrinsic Motivation Systems for Autonomous Mental Development, IEEE Transactions on Evolutionary Computation, vol.11, issue.2, pp.265-286, 2007.
DOI : 10.1109/TEVC.2006.890271
URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.107.666

P. Y. Oudeyer, F. Kaplan, and V. V. Hafner, Intrinsic Motivation Systems for Autonomous Mental Development, IEEE Transactions on Evolutionary Computation, vol.11, issue.2, pp.265-286, 2007.
DOI : 10.1109/TEVC.2006.890271
URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.107.666

P. Y. Oudeyer and L. Smith, How Evolution May Work Through Curiosity-Driven Developmental Process, Topics in Cognitive Science, vol.8, issue.11
DOI : 10.3389/neuro.12.006.2007
URL : https://hal.archives-ouvertes.fr/hal-01404334

C. J. Peck, D. C. Jangraw, M. Suzuki, R. Efem, and J. Gottlieb, Reward Modulates Attention Independently of Action Value in Posterior Parietal Cortex, Journal of Neuroscience, vol.29, issue.36, pp.29-11182, 2009.
DOI : 10.1523/JNEUROSCI.1929-09.2009

H. Rieiro, S. Martinez-conde, and S. L. Macknik, Perceptual elements in Penn & Teller???s ???Cups and Balls??? magic trick, PeerJ, vol.5, p.19, 2013.
DOI : 10.7717/peerj.19/supp-8

E. F. Risko, N. C. Anderson, S. Lanthier, and A. Kingstone, Curious eyes: Individual differences in personality predict eye movement behavior in scene-viewing, Cognition, vol.122, issue.1, pp.86-90, 2012.
DOI : 10.1016/j.cognition.2011.08.014

R. M. Ryan and E. L. Deci, Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions, Contemporary Educational Psychology, vol.25, issue.1, pp.54-67, 2000.
DOI : 10.1006/ceps.1999.1020

R. M. Ryan and E. L. Deci, Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being., American Psychologist, vol.55, issue.1, pp.68-78, 2000.
DOI : 10.1037/0003-066X.55.1.68

J. Schmidhuber, Curious model-building control systems, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks, 1991.
DOI : 10.1109/IJCNN.1991.170605

R. K. Srivastava, B. R. Steunebrink, and J. Schmidhuber, First experiments with PowerPlay, Neural Networks, vol.41, pp.130-136, 2013.
DOI : 10.1016/j.neunet.2013.01.022

B. T. Sullivan, L. Johnson, C. A. Rothkopf, D. Ballard, and M. Hayhoe, The role of uncertainty and reward on eye movements in a virtual driving task, Journal of Vision, vol.12, issue.13, 2012.
DOI : 10.1167/12.13.19

B. W. Tatler, M. N. Hayhoe, M. F. Land, and D. H. Ballard, Eye guidance in natural vision: Reinterpreting salience, Journal of Vision, vol.11, issue.5, pp.5-25, 2011.
DOI : 10.1167/11.5.5

M. L. Vo and J. M. Wolfe, The role of memory for visual search in scenes, Ann NY Acad Sci, vol.1339, pp.72-81, 2015.

B. C. Wittmann, N. Bunzeck, R. J. Dolan, and E. Düzel, Anticipation of novelty recruits reward system and hippocampus while promoting recollection, NeuroImage, vol.38, issue.1, pp.194-202, 2007.
DOI : 10.1016/j.neuroimage.2007.06.038

B. C. Wittmann, N. D. Daw, B. Seymour, and R. J. Dolan, Striatal activity underlies noveltybased choice in humans, Neuron, issue.6, pp.58-967, 2008.
DOI : 10.1016/j.neuron.2008.04.027

B. Clement, D. Roy, P. Oudeyer, and M. Lopes, Multi-Armed Bandits for Intelligent Tutoring Systems, Journal of Educational Data Mining (JEDM), vol.2015, issue.72, pp.20-48
URL : https://hal.archives-ouvertes.fr/hal-00913669