HAL - INRIA :: [inria-00185311, version 2] Fitted Q-iteration in continuous action-space MDPs

inria-00185311, version 2

<?xml version="1.0" encoding="UTF-8"?>
<listBibl xmlns="http://www.tei-c.org/ns/1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML">
<listBibl type="report"><head>Research reports</head>
<biblStruct type="report" xml:id="inria-00185311"><monogr>
<author><persName><forename>Andras</forename><surname>Antos</surname></persName>
<affiliation>
<orgName type="laboratory">SZTAKI</orgName>
<orgName type="team">Computer and Automation Research Institute of the Hungarian Academy of Sciences</orgName>
<country>HU</country>
<orgName type="institution">Computer and Automation Research Institute of the Hungarian Academy of Sciences</orgName>
</affiliation>
</author>
<author><persName><forename>Rémi</forename><surname>Munos</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA Futurs</orgName>
<orgName type="team">SEQUEL</orgName>
<country>FR</country>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">CNRS : UMR8022</orgName>
<orgName type="institution">CNRS : UMR8146</orgName>
<orgName type="institution">Université des Sciences et Technologies de Lille - Lille I</orgName>
<orgName type="institution">Université Charles de Gaulle - Lille III</orgName>
<orgName type="institution">Ecole Centrale de Lille</orgName>
</affiliation>
</author>
<author><persName><forename>Csaba</forename><surname>Szepesvari</surname></persName>
<affiliation>
<orgName type="team">Department of Computing Science, University of Alberta</orgName>
<country>CA</country>
<orgName type="institution">Department of Computing Science, University of Alberta</orgName>
</affiliation>
</author>
<title type="main" level="m">Fitted Q-iteration in continuous action-space MDPs</title><title type="main" level="m">Fitted Q-iteration in continuous action-space MDPs</title><imprint><biblScope type="publicationDate">2007</biblScope><biblScope type="pp">24</biblScope></imprint></monogr><idno type="HALid">http://hal.inria.fr/inria-00185311/en/</idno><idno type="HALFile">http://hal.inria.fr/inria-00185311/PDF/rlca.pdf</idno></biblStruct>
</listBibl>
</listBibl>

Fitted Q-iteration in continuous action-space MDPs

Andras Antos (

) ², Rémi Munos (

) ¹, Csaba Szepesvari (

) ³

(2007)

Access to full text
PDF

Abstract: We consider continuous state, continuous action batch reinforcement learning where the goal is to learn a good policy from a sufficiently rich trajectory generated by some policy. We study a variant of fitted Q-iteration, where the greedy action selection is replaced by searching for a policy in a restricted set of candidate policies by maximizing the average action values. We provide a rigorous analysis of this algorithm, proving what we believe is the first finite-time bound for value-function based algorithms for continuous state and action problems.

1:	SEQUEL (INRIA Futurs)
	INRIA – CNRS : UMR8022 – CNRS : UMR8146 – Université des Sciences et Technologies de Lille - Lille I – Université Charles de Gaulle - Lille III – Ecole Centrale de Lille
2:	Computer and Automation Research Institute of the Hungarian Academy of Sciences (SZTAKI)
	Computer and Automation Research Institute of the Hungarian Academy of Sciences
3:	Department of Computing Science, University of Alberta
	Department of Computing Science, University of Alberta

Domain

Computer Science/Learning

Available versions:

v1 (2007-11-05)

v2 (2008-01-08)


inria-00185311, version 2
http://hal.inria.fr/inria-00185311/en/
oai:hal.inria.fr:inria-00185311_v2
From: Rémi Munos <>
Submitted on: Tuesday, 8 January 2008 16:52:29
Updated on: Tuesday, 8 January 2008 17:02:01