HAL - INRIA :: [inria-00117075, version 1] Sparse Temporal Difference Learning using LASSO

inria-00117075, version 1

%% inria-00117075, version 1
%% http://hal.inria.fr/inria-00117075/en/
@inproceedings{LOTH:2007:INRIA-00117075:1,
	title = { {S}parse {T}emporal {D}ifference {L}earning using {LASSO}},
	author = {{L}oth, {M}anuel and {D}avy, {M}anuel and {P}reux, {P}hilippe},
	abstract = {{W}e consider the problem of on-line value function estimation in reinforcement learning. {W}e concentrate on the function approximator to use. {T}o try to break the curse of dimensionality, we focus on non parametric function approximators. {W}e propose to fit the use of kernels into the temporal difference algorithms by using regression via the {LASSO}. {W}e introduce the equi-gradient descent algorithm ({EGD}) which is a direct adaptation of the one recently introduced in the {LARS} algorithm family for solving the {LASSO}. {W}e advocate our choice of the {EGD} as a judicious algorithm for these tasks. {W}e present the {EGD} algorithm in details as well as some experimental results. {W}e insist on the qualities of the {EGD} for reinforcement learning.},
	language = {{A}nglais},
	affiliation = {{SEQUEL} - {INRIA} {F}uturs - {INRIA} - {CNRS} : {UMR}8022 - {CNRS} : {UMR}8146 - {U}niversit{\'e} des {S}ciences et {T}echnologies de {L}ille - {L}ille {I} - {U}niversit{\'e} {C}harles de {G}aulle - {L}ille {III} - {E}cole {C}entrale de {L}ille - {L}aboratoire d'{A}utomatique, {G}{\'e}nie {I}nformatique et {S}ignal - {LAGIS} - {CNRS} : {UMR}8146 - {U}niversit{\'e} des {S}ciences et {T}echnologies de {L}ille - {L}ille {I} - {E}cole {C}entrale de {L}ille },
	booktitle = {{IEEE} {I}nternational {S}ymposium on {A}pproximate {D}ynamic {P}rogramming and {R}einforcement {L}earning },
	address = {{H}awa{\"i}, {USA} },
	note = {{I}.: {C}omputing {M}ethodologies/{I}.2: {ARTIFICIAL} {INTELLIGENCE}/{I}.2.8: {P}roblem {S}olving, {C}ontrol {M}ethods, and {S}earch/{I}.2.8.2: {D}ynamic programming },
	audience = {non sp{\'e}cifi{\'e}e },
    day = {01},
    month = {04},
    year = {2007},
    URL = {http://hal.inria.fr/inria-00117075/en/},
    URL = {http://hal.inria.fr/inria-00117075/PDF/lassoTd.pdf},
}

<?xml version="1.0" encoding="UTF-8"?>
<listBibl xmlns="http://www.tei-c.org/ns/1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML">
<listBibl type="inproceedings"><head>Communications avec acte</head>
<biblStruct type="inproceedings" xml:id="inria-00117075"><analytic>
<author><persName><forename>Manuel</forename><surname>Loth</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA Futurs</orgName>
<orgName type="team">SEQUEL</orgName>
<country>FR</country>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">CNRS : UMR8022</orgName>
<orgName type="institution">CNRS : UMR8146</orgName>
<orgName type="institution">Université des Sciences et Technologies de Lille - Lille I</orgName>
<orgName type="institution">Université Charles de Gaulle - Lille III</orgName>
<orgName type="institution">Ecole Centrale de Lille</orgName>
</affiliation>
</author>
<author><persName><forename>Manuel</forename><surname>Davy</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA Futurs</orgName>
<orgName type="team">SEQUEL</orgName>
<country>FR</country>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">CNRS : UMR8022</orgName>
<orgName type="institution">CNRS : UMR8146</orgName>
<orgName type="institution">Université des Sciences et Technologies de Lille - Lille I</orgName>
<orgName type="institution">Université Charles de Gaulle - Lille III</orgName>
<orgName type="institution">Ecole Centrale de Lille</orgName>
</affiliation>
<affiliation>
<orgName type="laboratory">LAGIS</orgName>
<orgName type="team">Laboratoire d'Automatique, Génie Informatique et Signal</orgName>
<country>FR</country>
<orgName type="institution">CNRS : UMR8146</orgName>
<orgName type="institution">Université des Sciences et Technologies de Lille - Lille I</orgName>
<orgName type="institution">Ecole Centrale de Lille</orgName>
</affiliation>
</author>
<author><persName><forename>Philippe</forename><surname>Preux</surname></persName>
<affiliation>
<orgName type="laboratory">INRIA Futurs</orgName>
<orgName type="team">SEQUEL</orgName>
<country>FR</country>
<orgName type="institution">INRIA</orgName>
<orgName type="institution">CNRS : UMR8022</orgName>
<orgName type="institution">CNRS : UMR8146</orgName>
<orgName type="institution">Université des Sciences et Technologies de Lille - Lille I</orgName>
<orgName type="institution">Université Charles de Gaulle - Lille III</orgName>
<orgName type="institution">Ecole Centrale de Lille</orgName>
</affiliation>
</author>
<title type="main" level="a">Sparse Temporal Difference Learning using LASSO</title></analytic><monogr><meeting>IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning</meeting><imprint><pubPlace><settlement>Hawaï</settlement><country> USA</country></pubPlace><biblScope type="publicationDate">2007-04-01</biblScope></imprint></monogr><idno type="HALid">http://hal.inria.fr/inria-00117075/en/</idno><idno type="HALFile">http://hal.inria.fr/inria-00117075/PDF/lassoTd.pdf</idno></biblStruct>
</listBibl>
</listBibl>

Sparse Temporal Difference Learning using LASSO

Manuel Loth (

)^a^, 1, Manuel Davy^b^, 1, 2, Philippe Preux (

)^c^, 1

(2007)

Accès au texte intégral
PDF

IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning (2007)

Résumé : We consider the problem of on-line value function estimation in reinforcement learning. We concentrate on the function approximator to use. To try to break the curse of dimensionality, we focus on non parametric function approximators. We propose to fit the use of kernels into the temporal difference algorithms by using regression via the LASSO. We introduce the equi-gradient descent algorithm (EGD) which is a direct adaptation of the one recently introduced in the LARS algorithm family for solving the LASSO. We advocate our choice of the EGD as a judicious algorithm for these tasks. We present the EGD algorithm in details as well as some experimental results. We insist on the qualities of the EGD for reinforcement learning.

a –	Université des Sciences et Technologie de Lille - Lille I
b –	Ecole Centrale de Lille
c –	Université Charles de Gaulle - Lille III
1 :	SEQUEL (INRIA Futurs)
	INRIA – CNRS : UMR8022 – CNRS : UMR8146 – Université des Sciences et Technologies de Lille - Lille I – Université Charles de Gaulle - Lille III – Ecole Centrale de Lille
2 :	Laboratoire d'Automatique, Génie Informatique et Signal (LAGIS)
	CNRS : UMR8146 – Université des Sciences et Technologies de Lille - Lille I – Ecole Centrale de Lille

Domaine

Informatique/Apprentissage


inria-00117075, version 1
http://hal.inria.fr/inria-00117075/fr/
oai:hal.inria.fr:inria-00117075_v1
Contributeur : Manuel Loth <>
Soumis le : Jeudi 30 Novembre 2006, 13:15:51
Dernière modification le : Jeudi 25 Janvier 2007, 16:23:22