Reference : Collaborative multiagent reinforcement learning by payoff propagation
Scientific journals : Article
Engineering, computing & technology : Computer science
Collaborative multiagent reinforcement learning by payoff propagation
Kok, Jelle R. [> >]
Vlassis, Nikos mailto [University of Luxembourg > Luxembourg Centre for Systems Biomedicine (LCSB) > >]
Journal of Machine Learning Research
Yes (verified by ORBilu)
[en] collaborative multiagent system ; coordination graph ; reinforcement learning ; Q-learning ; belief propagation
[en] In this article we describe a set of scalable techniques for learning the behavior of a group of agents in a collaborative multiagent setting. As a basis we use the framework of coordination graphs of Guestrin, Koller, and Parr (2002a) which exploits the dependencies between agents to decompose the global payoff function into a sum of local terms. First, we deal with the single-state case and describe a payoff propagation algorithm that computes the individual actions that approximately maximize the global payoff function. The method can be viewed as the decision-making analogue of belief propagation in Bayesian networks. Second, we focus on learning the behavior of the agents in sequential decision-making tasks. We introduce different model-free reinforcement-learning techniques, unitedly called Sparse Cooperative Q-learning, which approximate the global action-value function based on the topology of a coordination graph, and perform updates using the contribution of the individual agents to the maximal global action value. The combined use of an edge-based decomposition of the action-value function and the payoff propagation algorithm for efficient action selection, result in an approach that scales only linearly in the problem size. We provide experimental evidence experimental evidence that our method outperforms related multiagent reinforcement-learning methods based on temporal differences.

File(s) associated to this reference

Fulltext file(s):

Open access
download.pdf postprint509.05 kBView/Open

Bookmark and Share SFX Query

All documents in ORBilu are protected by a user license.