M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, 1994.
DOI : 10.1002/9780470316887

P. Auer, N. Cesa-bianchi, and P. Fischer, Finite-time analysis of the multiarmed bandit problem, Machine Learning, vol.47, issue.2/3, pp.235-256, 2002.
DOI : 10.1023/A:1013689704352

N. Abe and A. Nakamura, Learning to optimally schedule Internet banner advertisements, Proceedings of the 16th International Conference on Machine Learning, pp.12-21, 1999.

O. Granmo, A Bayesian learning automaton for solving two-armed Bernoulli bandit problems, Proceedings of the 7th International Conference on Machine Learning and Applications, pp.23-30, 2008.

M. Langheinrich, A. Nakamura, N. Abe, T. Kamba, and Y. Koseki, Unintrusive customization techniques for Web advertising, Computer Networks, vol.31, issue.11-16, pp.31-42, 1999.
DOI : 10.1016/S1389-1286(99)00033-X

A. Nakamura and N. Abe, Improvements to the Linear Programming Based Scheduling of Web Advertisements, Electronic Commerce Research, vol.5, issue.1, pp.75-98, 2005.
DOI : 10.1023/B:ELEC.0000045974.88926.88

S. Pandey, D. Agarwal, D. Chakrabarti, and V. Josifovski, Bandits for Taxonomies: A Model-based Approach, Proceedings of the 7th SIAM International Conference on Data Mining, 2007.
DOI : 10.1137/1.9781611972771.20

J. Langford and T. Zhang, The epoch-greedy algorithm for multi-armed bandits with side information, Proceedings of 20th Advances in Neural Information Processing Systems, pp.817-824, 2008.

C. C. Wang, S. R. Kulkarni, S. Poor, and H. , Bandit problems with side observations, IEEE Transactions on Automatic Control, issue.3, pp.50-338, 2005.

S. M. Kakade, S. Shalev-shwartz, and A. Tewari, Efficient bandit algorithms for online multiclass prediction, Proceedings of the 25th international conference on Machine learning, ICML '08, pp.440-447, 2008.
DOI : 10.1145/1390156.1390212

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=

W. Li, X. Wang, R. Zhang, Y. Cui, J. Mao et al., Exploitation and exploration in a performance based contextual advertising system, Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, KDD '10, pp.27-36, 2010.
DOI : 10.1145/1835804.1835811

S. Pandey and C. Olston, Handling advertisements of unknown quality in search advertising, Proceedings of 18th Advances in Neural Information Processing Systems, pp.1065-1072, 2006.

D. Agarwal, B. Chen, and P. Elango, Explore/Exploit Schemes for Web Content Optimization, 2009 Ninth IEEE International Conference on Data Mining, pp.1-10, 2009.
DOI : 10.1109/ICDM.2009.52

L. Li, W. Chu, J. Langford, and R. Schapire, A contextual-bandit approach to personalized news article recommendation, Proceedings of the 19th international conference on World wide web, WWW '10, pp.661-670, 2010.
DOI : 10.1145/1772690.1772758

M. Richardson, E. Dominowska, and R. Ragno, Predicting clicks, Proceedings of the 16th international conference on World Wide Web , WWW '07, pp.521-530, 2007.
DOI : 10.1145/1242572.1242643

D. Agarwal, B. C. Chen, and P. Elango, Spatio-temporal models for estimating click-through rate, Proceedings of the 18th international conference on World wide web, WWW '09, pp.21-30, 2009.
DOI : 10.1145/1526709.1526713

D. Agarwal, A. Broder, D. Chakrabarti, D. Diklic, V. Josifovski et al., Estimating rates of rare events at multiple resolutions, Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining , KDD '07, pp.16-25, 2007.
DOI : 10.1145/1281192.1281198

X. Wang, W. Li, Y. Cui, B. Zhang, and J. Mao, Clickthrough rate estimation for rare events in online advertising, Online Multimedia Advertising: Techniques and Technologies. Hershey: IGI Global, 2010.

T. K. Fan and C. Chang, Sentiment-oriented contextual advertising, Knowledge and Information Systems, vol.20, issue.3, pp.321-344, 2010.
DOI : 10.1007/s10115-009-0222-2

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=

A. Mehta, A. Saberi, U. Vazirani, and V. Vazirani, AdWords and Generalized On-line Matching, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05), pp.264-273, 2005.
DOI : 10.1109/SFCS.2005.12

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=

M. Mahdian and H. Nazerzadeh, Allocating online advertisement space with unreliable estimates, Proceedings of the 8th ACM conference on Electronic commerce , EC '07, pp.288-294, 2007.
DOI : 10.1145/1250910.1250952

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=

J. Langford, A. Strehl, and J. Wortman, Exploration scavenging, Proceedings of the 25th international conference on Machine learning, ICML '08, pp.528-535, 2008.
DOI : 10.1145/1390156.1390223

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=