Article (Scientific journals)
Point-Based Value Iteration for Continuous POMDPs
Porta, Josep M.; VLASSIS, Nikos; Spaan, Matthijs T. J. et al.
2006In Journal of Machine Learning Research, 7, p. 2329-2367
Peer Reviewed verified by ORBi
 

Files


Full Text
download.pdf
Publisher postprint (527.86 kB)
http://jmlr.org/papers/volume7/porta06a/porta06a.pdf
Download

All documents in ORBilu are protected by a user license.

Send to



Details



Abstract :
[en] We propose a novel approach to optimize Partially Observable Markov Decisions Processes (POMDPs) defined on continuous spaces. To date, most algorithms for model-based POMDPs are restricted to discrete states, actions, and observations, but many real-world problems such as, for instance, robot navigation, are naturally defined on continuous spaces. In this work, we demonstrate that the value function for continuous POMDPs is convex in the beliefs over continuous state spaces, and piecewise-linear convex for the particular case of discrete observations and actions but still continuous states. We also demonstrate that continuous Bellman backups are contracting and isotonic ensuring the monotonic convergence of value-iteration algorithms. Relying on those properties, we extend the algorithm, originally developed for discrete POMDPs, to work in continuous state spaces by representing the observation, transition, and reward models using Gaussian mixtures, and the beliefs using Gaussian mixtures or particle sets. With these representations, the integrals that appear in the Bellman backup can be computed in closed form and, therefore, the algorithm is computationally feasible. Finally, we further extend to deal with continuous action and observation sets by designing effective sampling approaches.
Disciplines :
Computer science
Identifiers :
UNILU:UL-ARTICLE-2011-714
Author, co-author :
Porta, Josep M.
VLASSIS, Nikos ;  University of Luxembourg > Luxembourg Centre for Systems Biomedicine (LCSB)
Spaan, Matthijs T. J.
Poupart, Pascal
Language :
English
Title :
Point-Based Value Iteration for Continuous POMDPs
Publication date :
2006
Journal title :
Journal of Machine Learning Research
ISSN :
1532-4435
eISSN :
1533-7928
Publisher :
MIT Press, United States - Massachusetts
Volume :
7
Pages :
2329-2367
Peer reviewed :
Peer Reviewed verified by ORBi
Available on ORBilu :
since 17 November 2013

Statistics


Number of views
44 (0 by Unilu)
Number of downloads
119 (0 by Unilu)

Scopus citations®
 
193
Scopus citations®
without self-citations
184
WoS citations
 
143

Bibliography


Similar publications



Contact ORBilu