References of "Whiteson, Shimon"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailMultiagent Reinforcement Learning for Urban Traffic Control Using Coordination Graphs
Kuyer, Lior; Whiteson, Shimon; Bakker, Bram et al

in Proceedings of 19th European Conference on Machine Learning (2008)

Since traffic jams are ubiquitous in the modern world, optimizing, the behavior of traffic lights for efficient traffic flow is a critically important goal. Though most current traffic lights use simple ... [more ▼]

Since traffic jams are ubiquitous in the modern world, optimizing, the behavior of traffic lights for efficient traffic flow is a critically important goal. Though most current traffic lights use simple heuristic protocols, more efficient controllers can be discovered automatically via multiagent reinforcement learning where each agent controls a single traffic light. However, in previous work on this approach, agents select only locally optimal actions without coordinating their behavior. This paper extends this approach to include explicit coordination between neighboring traffic lights. Coordination is achieved using the max-plus algorithm, which estimates the optimal joint action by sending locally optimized messages among connected agents. This paper presents the first application of max-plus to a large-scale problem and thus verifies its efficacy in realistic settings. It also provides empirical evidence that max-plus performs well on cyclic graphs, though it has been proven to converge only for tree-structured graphs. Furthermore, it provides a new understanding of the properties a traffic network must have for such coordination to be beneficial and shows that max-plus outperforms previous methods on networks that possess those properties. [less ▲]

Detailed reference viewed: 75 (0 UL)
Full Text
Peer Reviewed
See detailExploiting locality of interaction in factored Dec-POMDPs
Oliehoek, Frans A.; Spaan, Matthijs T. J.; Vlassis, Nikos UL et al

in Int. Joint Conf. on Autonomous Agents and Multi-Agent Systems (2008)

Decentralized partially observable Markov decision processes (Dec-POMDPs) constitute an expressive framework for multiagent planning under uncertainty, but solving them is provably intractable. We ... [more ▼]

Decentralized partially observable Markov decision processes (Dec-POMDPs) constitute an expressive framework for multiagent planning under uncertainty, but solving them is provably intractable. We demonstrate how their scalability can be improved by exploiting locality of interaction between agents in a factored representation. Factored Dec-POMDP representations have been proposed before, but only for Dec-POMDPs whose transition and observation models are fully independent. Such strong assumptions simplify the planning problem, but result in models with limited applicability. By contrast, we consider general factored Dec-POMDPs for which we analyze the model dependencies over space (locality of interaction) and time (horizon of the problem). We also present a formulation of decomposable value functions. Together, our results allow us to exploit the problem structure as well as heuristics in a single framework that is based on collaborative graphical Bayesian games (CGBGs). A preliminary experiment shows a speedup of two orders of magnitude. [less ▲]

Detailed reference viewed: 21 (0 UL)