E. Altman, Constrained Markov Decision Processes, 1999.
URL : https://hal.archives-ouvertes.fr/inria-00074109

K. Avrachenkov and V. Borkar, Whittle Index Policy for Crawling Ephemeral Content, Proceedings of IEEE CDC, 2015.
URL : https://hal.archives-ouvertes.fr/hal-01937994

K. Avrachenkov, U. Ayesta, J. Doncel, and P. Jacko, Congestion Control of TCP Flows in Internet Routers by Means of Index Policy, Computer Networks, vol.57, issue.17, pp.3463-3478, 2013.
URL : https://hal.archives-ouvertes.fr/hal-00926612

K. Avrachenkov, O. Habachi, A. Piunovskiy, and Y. Zhang, Infinite Horizon Optimal Impulsive Control with Applications to Internet Congestion Control, International Journal of Control, vol.88, issue.4, pp.703-716, 2015.
URL : https://hal.archives-ouvertes.fr/hal-01259259

K. Avrachenkov, A. Dudin, V. Klimenok, P. Nain, and O. Semenova, Optimal Threshold Control by the Robots of Web Search Engines with Obsolescence of Documents, Computer Networks, vol.55, issue.8, pp.1880-1893, 2011.
URL : https://hal.archives-ouvertes.fr/hal-00648830

V. S. Borkar, Convex Analytic Methods in Markov Decision Processes, pp.347-375, 2002.

V. S. Borkar, Stochastic Approximation: A Dynamical Systems Viewpoint, 2008.

J. Cho and H. Garcia-molina, Synchronizing a Database to Improve Freshness, Proceedings of ACM SIGMOD 2000, vol.29, pp.117-128

J. Cho and H. Garcia-molina, Effective Page Refresh Policies for Web Crawlers, ACM Transactions on Database Systems (TODS), vol.28, issue.4, pp.390-426

J. Cho and A. Ntoulas, Effective Change Detection Using Sampling, Proceedings of VLDB 2002, pp.514-525

A. Goyal, F. Bonchi, and L. V. Lakshmanan, Learning Influence Probabilities in Social Networks, Proceedings of ACM WSDM 2010, pp.241-250, 2010.

O. Hernández-lerma and J. Lasserre, Discrete Time Markov Control Processes: Basic Optimality Criteria, 1996.

P. Jacko, Dynamic Priority Allocation in Restless Bandit Models, 2010.

P. Jacko and B. Sanso, Congestion Avoidance with Future-Path Information, Proceedings of EuroFGI Workshop on IP QoS and Traffic Control, pp.153-160, 2007.

M. Larranaga, U. Ayesta, and I. M. Verloop, Stochastic and Fluid Index Policies for Resource Allocation Problems, Proceedings of IEEE INFOCOM 2015, pp.1-9

D. Lefortier, L. Ostroumova, E. Samosvat, and P. Serdyukov, Timely Crawling of High-quality Ephemeral New Content, Proceedings of CIKM 2013, pp.745-750, 2013.

Z. Liu and P. Nain, Optimization Issues in Web Search Engines, Handbook of Optimization in Telecommunications, pp.981-1015, 2006.

K. Liu and Q. Zhao, Indexability of Restless Bandit Problems and Optimality of Whittle Index for Dynamic Multichannel Access, IEEE Trans. Info. Theory, vol.56, issue.11, pp.5547-5567, 2010.

P. Milgrom and I. Segal, Envelope Theorems for Arbitrary Choice Sets, Econometrica, vol.70, pp.583-603, 2002.

T. Moon, W. Chu, L. Li, Z. Zheng, and Y. Chang, Refining Recency Search Results with User Click Feedback, 2011.

J. Nino-mora and S. S. Villar, Sensor Scheduling for Hunting Elusive Hiding Targets via Whittle's Restless Bandit Index Policy, Proceedings of NetGCoop, pp.1-8, 2011.

J. L. Ny, M. Dahleh, and E. Feron, Multi-UAV Dynamic Routing with Partial Observations Using Restless Bandit Allocation Indices, Proceedings of American Control Conf. (ACC, pp.4220-4225, 2008.

C. Olston and M. Najork, Web Crawling, In Foundations and Trends in Information Retrieval, vol.4, issue.3, pp.175-246, 2010.

A. B. Piunovskiy, Optimal Control of Random Sequences in Problems with Constraints, p.348, 1997.

M. I. Puterman, Markov Decision Processes, 1994.

V. Raghunathan, V. S. Borkar, M. Cao, and P. R. Kumar, Index Policies for Real-time Multicast Scheduling for Wireless Bradcast Systems, Proceedings of IEEE INFOCOM 2008, pp.2243-2251, 2008.

D. Ruiz-hernandez, , 2008.

N. Salodkar, A. Karandikar, and V. S. Borkar, A Stable Online Algorithm for energy-efficient multi-user scheduling, IEEE Transactions on Mobile Computing, vol.9, issue.10, pp.1391-1406, 2010.

J. Talim, Z. Liu, P. Nain, and E. G. Coffman, Controlling the Robots of Web Search Engines, Performance Evaluation Review, vol.29, issue.1, pp.236-244, 2001.
DOI : 10.1145/378420.378788
URL : http://www.ee.columbia.edu/~egc/webpapers/newrob1.ps

I. M. Verloop, Asymptotically Optimal Priority Policies for Indexable and Non-indexable Restless Bandits, Annals of Applied Probability, vol.26, issue.4, pp.1947-1995, 2016.
DOI : 10.1214/15-aap1137
URL : https://hal.archives-ouvertes.fr/hal-00743781

R. R. Weber and G. Weiss, On an Index Policy for Restless Bandits, J. Appl. Prob, vol.27, pp.637-648, 1990.
DOI : 10.1017/s0021900200039176

P. Whittle, Restless Bandits: Activity Allocation in a Changing World, J. Appl. Prob, vol.25, pp.287-298, 1988.
DOI : 10.1017/s0021900200040420

K. E. , Ph.D. degree in Mathematics from University of South Australia (2000) and Habilitation, 1996.

. Currently, He is an associate editor of International Journal of Performance Evaluation and ACM TOMPECS. His main research interests are Markov processes, singular perturbation theory, queueing theory, mathematical programming, game theory and performance evaluation of communication networks

S. ;. Vivek and P. D. , He has held positions at TIFR Centre for Applicable Math. and Indian Institute of Science in Bangalore and TIFR and IIT Bombay in Mumbai. He is currently an Institute Chair Professor of Electrical Eng. in the latter. He has held visiting positions at Uni. of Twente, MIT, Uni. of Maryland at College Park and Uni. of California at Berkeley. He is a Fellow of IEEE, TWAS and the science and engineering academies in India. His research interests are stochastic optimization-theory, applications and algorithms