OATAO - Open Archive Toulouse Archive Ouverte Open Access Week

Extending the Bellman equation for MDPs to continuous actions and continuous time in the discounted case

Rachelson, Emmanuel and Garcia, Frédérick and Fabiani, Patrick Extending the Bellman equation for MDPs to continuous actions and continuous time in the discounted case. (2008) In: International Symposium on Artificial Intelligence and Mathematics - 2008, 2008 - 2008 (Fort Lauderdale, United States).

(Document in English)

PDF (Author's version) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader


Recent work on Markov Decision Processes (MDPs) covers the use of continuous variables and resources, including time. This work is usually done in a framework of bounded resources and finite temporal horizon for which a total reward criterion is often appropriate. However, most of this work considers discrete effects on continuous variables while considering continuous variables often allows for parametric (possibly continuous) quantification of actions effects. On top of that, infinite horizon MDPs often make use of discounted criterions in order to insure convergence and to account for the difference between a reward obtained now and a reward obtained later. In this paper, we build on the standard MDP framework in order to extend it to continuous time and resources and to the corresponding parametric actions. We aim at providing a framework and a sound set of hypothesis under which a classical Bellman equation holds in the discounted case, for parametric continuous actions and hybrid state spaces, including time. We illustrate our approach by applying it to the TMDP representation of Boyan and Littman.

Item Type:Conference or Workshop Item (Paper)
Audience (conference):International conference proceedings
Uncontrolled Keywords:
Institution:French research institutions > Institut National de la Recherche Agronomique - INRA (FRANCE)
French research institutions > Office National d'Etudes et Recherches Aérospatiales - ONERA (FRANCE)
Laboratory name:
Deposited On:29 Nov 2017 11:03

Repository Staff Only: item control page