Exploiting Additive Structure in Factored MDPs for Reinforcement Learning

Degris, Thomas
LIP6 - Pôle IA
Université Paris 6
8, rue du capitaine Scott F-75015 Paris, France
email: thomas.degris@lip6.fr   

Sigaud, Olivier
LIP6 - Pôle IA
Université Paris 6
8, rue du capitaine Scott F-75015 Paris, France
email: olivier.sigaud@lip6.fr   

Wuillemin, Pierre-Henri
LIP6 - Pôle IA
Université Paris 6
8, rue du capitaine Scott F-75015 Paris, France
email: pierre-henri.wuillemin@lip6.fr   home: www-desir.lip6.fr/~phw

Degris, Thomas and Sigaud, Olivier and Wuillemin, Pierre-Henri (2008) "Exploiting Additive Structure in Factored MDPs for Reinforcement Learning". In European Workshop on Reinforcement Learning, Lille, France.

SDYNA is a framework able to address large, discrete and stochastic reinforcement learning problems. It incrementally learns a FMDP representing the problem to solve while using FMDP planning techniques to build an efficient policy. SPITI, an instantiation of SDYNA, uses a planning method based on dynamic programming which cannot exploit the additive structure of a FMDP. In this paper, we present two new instantiations of SDYNA, namely ULP and UNATLP, using a linear programming based planning method that can exploit the additive structure of a FMDP and address problems out of reach of SPITI.

@InProceedings{,
  author = {Degris, Thomas and Sigaud, Olivier and Wuillemin, Pierre-Henri},
  title = {Exploiting Additive Structure in Factored MDPs for Reinforcement Learning},
  booktitle = {European Workshop on Reinforcement Learning},
  year = {2008},
  month = {7},
  address = {Lille, France}
}