E. Altman, Constrained Markov Decision Processes, 1999.
URL : https://hal.archives-ouvertes.fr/inria-00074109

C. Baier, M. Größer, M. Leucker, B. Bollig, and F. Ciesinski, Controller Synthesis for Probabilistic Systems (Extended Abstract), pp.493-506, 2004.
DOI : 10.1007/1-4020-8141-3_38

A. G. Barto, S. J. Bradtke, and S. P. Singh, Learning to act using real-time dynamic programming, Artificial Intelligence, vol.72, issue.1-2, pp.81-138, 1995.
DOI : 10.1016/0004-3702(94)00011-O

D. Bertsekas, Dynamic Programming and Optimal Control Volumes), 1995.

D. Bertsekas and J. Tsitsiklis, Neurodynamic Programming, 1996.

B. Bonet and H. Geffner, Planning as heuristic search, Artificial Intelligence, vol.129, issue.1-2, pp.5-33, 2001.
DOI : 10.1016/S0004-3702(01)00108-4
URL : http://doi.org/10.1016/s0004-3702(01)00108-4

B. Bonet and H. Geffner, Faster heuristic search algorithms for planning with uncertainty and full feedback, Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI'03), pp.1233-1238, 2003.

B. Bonet and H. Geffner, Labeled RTDP: Improving the convergence of real-time dynamic programming, Proceedings of the 13th International Conference on Automated Planning and Scheduling (ICAPS'03), pp.12-21, 2003.

B. Bonet and H. Geffner, mgpt: A probabilistic planner based on heuristic search, Journal of Artificial Intelligence Research, vol.24, pp.933-944, 2005.

B. Bonet and H. Geffner, Learning depth-first search: A unified approach to heuristic search in deterministic and non-deterministic settings, and its application to MDPs, Proceedings of the 16th International Conference on Automated Planning and Scheduling (ICAPS'06), pp.142-151, 2006.

B. Bonet and H. Geffner, Action selection for MDPs: Anytime AO* versus UCT, Proceedings of the 26th AAAI Conference on Artificial Intelligence (AAAI'12), 2012.

D. Bryce and O. Buffet, 6th international planning competition: Uncertainty part, Proceedings of the 6th International Planning Competition (IPC'08), 2008.

A. Camacho, C. Muise, and S. A. Mcilraith, From FOND to robust probabilistic planning: Computing compact policies that bypass avoidable deadends, Proceedings of the 26th International Conference on Automated Planning and Scheduling (ICAPS'16, 2016.

K. Chatterjee, M. Chmelik, R. Gupta, and A. Kanodia, Optimal cost almost-sure reachability in POMDPs, Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI'15), pp.3496-3502, 2015.
DOI : 10.1016/j.artint.2016.01.007

K. Chatterjee, M. Chmelik, R. Gupta, and A. Kanodia, Optimal cost almost-sure reachability in POMDPs, Artificial Intelligence, vol.234, pp.26-48, 2016.
DOI : 10.1016/j.artint.2016.01.007

A. J. Coles, Opportunistic branched plans to maximise utility in the presence of resource uncertainty, Proceedings of the 20th European Conference on Artificial Intelligence (ECAI'12), pp.252-257, 2012.

A. J. Coles, A. Coles, M. Fox, and D. Long, A hybrid LP-RPG heuristic for modelling numeric resource flows in planning, Journal of Artificial Intelligence Research, vol.46, pp.343-412, 2013.

A. J. Coles, A. Coles, A. García-olaya, S. Jiménez, C. Linares-lópez et al., A survey of the seventh international planning competition. The AI Magazine, 2012.

P. Dai, . Mausam, D. S. Weld, and J. Goldsmith, Topological value iteration algorithms, Journal of Artificial Intelligence Research, vol.42, pp.181-209, 2011.

T. L. Dean and R. Givan, Model minimization in markov decision processes, Proceedings of the 14th National Conference of the American Association for Artificial Intelligence (AAAI'97), pp.106-111, 1997.

C. Domshlak and V. Mirkis, Deterministic oversubscription planning as heuristic search: Abstractions and reformulations, Journal of Artificial Intelligence Research, vol.52, pp.97-169, 2015.

K. Dräger, B. Finkbeiner, and A. Podelski, Directed model checking with distance-preserving abstractions, International Journal on Software Tools for Technology Transfer, vol.1, issue.9, pp.27-37, 2009.
DOI : 10.1007/s10009-008-0092-z

S. Edelkamp, Planning with pattern databases, Proceedings of the 6th European Conference on Planning (ECP'01), pp.13-24, 2001.

A. Gefen and R. I. Brafman, Pruning methods for optimal delete-free planning, Proceedings of the 22nd International Conference on Automated Planning and Scheduling (ICAPS'12, 2012.

R. Givan, S. M. Leach, and T. Dean, Bounded-parameter Markov decision processes, Artificial Intelligence, vol.122, issue.1-2, pp.71-109, 2000.
DOI : 10.1016/S0004-3702(00)00047-3
URL : http://doi.org/10.1016/s0004-3702(00)00047-3

E. A. Hansen and S. Zilberstein, LAO???: A heuristic search algorithm that finds solutions with loops, Artificial Intelligence, vol.129, issue.1-2, pp.35-62, 2001.
DOI : 10.1016/S0004-3702(01)00106-0

P. Haslum, A. Botea, M. Helmert, B. Bonet, and S. Koenig, Domain-independent construction of pattern database heuristics for cost-optimal planning, Proceedings of the 22nd National Conference of the American Association for Artificial Intelligence (AAAI'07), pp.1007-1012, 2007.

P. Haslum and H. Geffner, Heuristic planning with time and resources, Proceedings of the 6th European Conference on Planning (ECP'01), pp.121-132, 2001.

M. Helmert, The Fast Downward planning system, Journal of Artificial Intelligence Research, vol.26, pp.191-246, 2006.

M. Helmert and C. Domshlak, Landmarks, critical paths and abstractions: What's the difference anyway, Proceedings of the 19th International Conference on Automated Planning and Scheduling (ICAPS'09), pp.162-169, 2009.

M. Helmert, P. Haslum, J. Hoffmann, and R. Nissim, Merge-and-Shrink Abstraction, Journal of the ACM, vol.61, issue.3, p.61, 2014.
DOI : 10.1145/2559951
URL : https://hal.archives-ouvertes.fr/inria-00592438

J. Hoffmann, Simulated penetration testing: From " Dijkstra " to " Turing Test++, Proceedings of the 25th International Conference on Automated Planning and Scheduling (ICAPS'15, 2015.

J. Hoffmann, P. Kissmann, and ´. A. Torralba, Distance " ? Who Cares? Tailoring merge-andshrink heuristics to detect unsolvability, Proceedings of the 21st European Conference on Artificial Intelligence (ECAI'14), 2014.

J. Hoffmann and B. Nebel, The FF planning system: Fast plan generation through heuristic search, Journal of Artificial Intelligence Research, vol.14, pp.253-302, 2001.

P. Hou, W. Yeoh, and P. Varakantham, Revisiting risk-sensitive MDPs: New algorithms and results, Proceedings of the 24th International Conference on Automated Planning and Scheduling (ICAPS'14, 2014.

S. Jimenez, A. Coles, and A. Smith, Planning in probabilistic domains using a deterministic numeric planner, Proceedings of the 25th Workshop of the UK Planning and Scheduling Special Interest Group (PlanSig'06), 2006.

E. Karpas and C. Domshlak, Cost-optimal planning with landmarks, Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI'09), pp.1728-1733, 2009.

M. Katz, J. Hoffmann, and M. Helmert, How to relax a bisimulation, Proceedings of the 22nd International Conference on Automated Planning and Scheduling (ICAPS'12), pp.101-109, 2012.
URL : https://hal.archives-ouvertes.fr/hal-00765027

T. Keller and P. Eyerich, PROST: Probabilistic planning based on UCT, Proceedings of the 22nd International Conference on Automated Planning and Scheduling (ICAPS'12, 2012.

T. Keller and M. Helmert, Trial-based heuristic tree search for finite horizon MDPs, Proceedings of the 23rd International Conference on Automated Planning and Scheduling (ICAPS'13), 2013.

A. Kolobov, Scalable Methods and Expressive Models for Planning Under Uncertainty, 2013.

A. Kolobov, . Mausam, and D. S. Weld, Sixthsense: Fast and reliable recognition of dead ends in MDPs, Proceedings of the 24th National Conference of the American Association for Artificial Intelligence (AAAI'10), 2010.

A. Kolobov, . Mausam, and D. S. Weld, A theory of goal-oriented MDPs with dead ends, Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence (UAI'12), pp.438-447, 2012.

A. Kolobov, . Mausam, D. S. Weld, and H. Geffner, Heuristic search for generalized stochastic shortest path MDPs, Proceedings of the 21st International Conference on Automated Planning and Scheduling (ICAPS'11, 2011.

H. Kurniawati, D. Hsu, and W. S. Lee, SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces, Robotics: Science and Systems IV, 2008.
DOI : 10.15607/RSS.2008.IV.009
URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.149.450

U. Kuter and J. Hu, Computing and Using Lower and Upper Bounds for Action Elimination in MDP Planning, Proceedings of the 7th International Symposium on Abstraction, Reformulation, and Approximation (SARA-07), 2007.
DOI : 10.1007/978-3-540-73580-9_20

M. Kwiatkowska, D. Parker, and H. Qu, Incremental quantitative verification for Markov decision processes, 2011 IEEE/IFIP 41st International Conference on Dependable Systems & Networks (DSN), pp.359-370, 2011.
DOI : 10.1109/DSN.2011.5958249
URL : https://hal.archives-ouvertes.fr/hal-00647057

M. Z. Kwiatkowska, G. Norman, and D. Parker, PRISM 4.0: Verification of Probabilistic Real-Time Systems, Proceedings of the 23rd International on Conference Computer Aided Verification (CAV'11), pp.585-591, 2011.
DOI : 10.1007/3-540-45657-0_17
URL : https://hal.archives-ouvertes.fr/hal-00648035

I. Little, D. Aberdeen, and S. Thiébaux, Prottle: A probabilistic temporal planner, Proceedings of the 20th National Conference of the American Association for Artificial Intelligence (AAAI'05), pp.1181-1186, 2005.

I. Little and S. Thiebaux, Probabilistic planning vs replanning, ICAPS Workshop on the International Planning Competition: Past, Present and Future, 2007.

J. Marecki and M. Tambe, Towards faster planning with continuous resources in stochastic domains, Proceedings of the 23rd National Conference of the American Association for Artificial Intelligence (AAAI'08), pp.1049-1055, 2008.

H. B. Mcmahan, M. Likhachev, and G. J. Gordon, Bounded real-time dynamic programming, Proceedings of the 22nd international conference on Machine learning , ICML '05, 2005.
DOI : 10.1145/1102351.1102423

N. Meuleau, E. Benazera, R. I. Brafman, E. A. Hansen, and M. Mausam, A heuristic search approach to planning with continuous resources in stochastic domains, Journal of Artificial Intelligence Research, vol.34, issue.1, pp.27-59, 2009.

R. Milner, Operational and Algebraic Semantics of Concurrent Processes, Handbook of Theoretical Computer Science, Volume B: Formal Models and Sematics, pp.1201-1242, 1990.
DOI : 10.1016/B978-0-444-88074-1.50024-X

C. J. Muise, S. A. Mcilraith, and J. C. Beck, Improved non-deterministic planning by exploiting state relevance, Proceedings of the 22nd International Conference on Automated Planning and Scheduling (ICAPS'12, 2012.

H. Nakhost, J. Hoffmann, and M. Müller, Resource-constrained planning: A monte carlo random walk approach, Proceedings of the 22nd International Conference on Automated Planning and Scheduling (ICAPS'12), pp.181-189, 2012.
URL : https://hal.archives-ouvertes.fr/hal-00765030

N. J. Nilsson, Problem Solving Methods in Artificial Intelligence, 1971.

R. Nissim, J. Hoffmann, and M. Helmert, Computing perfect heuristics in polynomial time: On bisimulation and merge-and-shrink abstraction in optimal planning, Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI'11), pp.1983-1990, 2011.
URL : https://hal.archives-ouvertes.fr/inria-00592438

F. Pommerening and M. Helmert, Optimal planning for delete-free tasks with incremental lm-cut, Proceedings of the 22nd International Conference on Automated Planning and Scheduling (ICAPS'12, 2012.

S. Richter and M. Helmert, Preferred operators and deferred evaluation in satisficing planning, Proceedings of the 19th International Conference on Automated Planning and Scheduling (ICAPS'09), pp.273-280, 2009.

S. Sanner, Relational dynamic influence diagram language (rddl): Language description, 2010.

P. Santana, S. Thibaux, and B. Williams, RAO*: An algorithm for chance-constrained POMDPs, Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI'16), pp.3308-3314, 2016.

C. Sarraute, O. Buffet, and J. Hoffmann, POMDPs make better hackers: Accounting for uncertainty in penetration testing, Proceedings of the 26th AAAI Conference on Artificial Intelligence (AAAI'12), pp.1816-1824, 2012.
URL : https://hal.archives-ouvertes.fr/hal-00755265

J. Seipp and M. Helmert, Counterexample-guided Cartesian abstraction refinement, Proceedings of the 23rd International Conference on Automated Planning and Scheduling (ICAPS'13), pp.347-351, 2013.

J. Seipp and M. Helmert, Diverse and additive cartesian abstraction heuristics, Proceedings of the 24th International Conference on Automated Planning and Scheduling (ICAPS'14, 2014.

T. Smith and R. G. Simmons, Focused real-time dynamic programming for MDPs: Squeezing more out of a heuristic, Proceedings of the 21st National Conference of the American Association for Artificial Intelligence (AAAI'06), pp.1227-1232, 2006.

M. Steinmetz, J. Hoffmann, and O. Buffet, Revisiting goal probability analysis in probabilistic planning, Proceedings of the 26th International Conference on Automated Planning and Scheduling (ICAPS'16, 2016.
URL : https://hal.archives-ouvertes.fr/hal-01413035

R. E. Tarjan, Depth-First Search and Linear Graph Algorithms, SIAM Journal on Computing, vol.1, issue.2, pp.146-160, 1972.
DOI : 10.1137/0201010

F. Teichteil-königsbuch, Stochastic safest and shortest path problems, Proceedings of the 26th AAAI Conference on Artificial Intelligence (AAAI'12), 2012.

F. Teichteil-königsbuch, U. Kuter, and G. Infantes, Incremental plan aggregation for generating policies in MDPs, Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems (AAMAS'10), pp.1231-1238, 2010.

F. Teichteil-königsbuch, V. Vidal, and G. Infantes, Extending classical planning heuristics to probabilistic planning with dead-ends, Proceedings of the 25th National Conference of the American Association for Artificial Intelligence (AAAI'11), 2011.

´. A. Torralba and J. Hoffmann, Simulation-based admissible dominance pruning, Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI'15), pp.1689-1695, 2015.