Reference : Optimal and approximate Q-value functions for decentralized POMDPs
Scientific journals : Article
Engineering, computing & technology : Computer science
http://hdl.handle.net/10993/11026
Optimal and approximate Q-value functions for decentralized POMDPs
English
Oliehoek, Frans A. [> >]
Spaan, Matthijs T. J. [> >]
Vlassis, Nikos mailto [University of Luxembourg > Luxembourg Centre for Systems Biomedicine (LCSB) > >]
2008
Journal of Artificial Intelligence Research
Morgan Kaufmann Publishers
32
289-353
Yes (verified by ORBilu)
1076-9757
1943-5037
San Francisco
CA
[en] Decision-theoretic planning is a popular approach to sequential decision making problems, because it treats uncertainty in sensing and acting in a principled way. In single-agent frameworks like MDPs and POMDPs, planning can be carried out by resorting to Q-value functions: an optimal Q-value function Q* is computed in a recursive manner by dynamic programming, and then an optimal policy is extracted from Q*. In this paper we study whether similar Q-value functions can be defined for decentralized POMDP models (Dec-POMDPs), and how policies can be extracted from such value functions. We define two forms of the optimal Q-value function for Dec-POMDPs: one that gives a normative description as the Q-value function of an optimal pure joint policy and another one that is sequentially rational and thus gives a recipe for computation. This computation, however, is infeasible for all but the smallest problems. Therefore, we analyze various approximate Q-value functions that allow for efficient computation. We describe how they relate, and we prove that they all provide an upper bound to the optimal Q-value function Q*. Finally, unifying some previous approaches for solving Dec-POMDPs, we describe a family of algorithms for extracting policies from such Q-value functions, and perform an experimental evaluation on existing test problems, including a new firefighting benchmark problem.
http://hdl.handle.net/10993/11026
http://www.jair.org/media/2447/live-2447-3856-jair.pdf
MARKOV DECISION-PROCESSES COMPLEXITY SYSTEMS

File(s) associated to this reference

Fulltext file(s):

FileCommentaryVersionSizeAccess
Open access
live-2447-3856-jair.pdfhttp://www.jair.org/media/2447/live-2447-3856-jair.pdfPublisher postprint613.04 kBView/Open

Bookmark and Share SFX Query

All documents in ORBilu are protected by a user license.