Reference : Learning model-free robot control by a Monte Carlo EM algorithm
Scientific journals : Article
Engineering, computing & technology : Computer science
Learning model-free robot control by a Monte Carlo EM algorithm
Vlassis, Nikos mailto [University of Luxembourg > Luxembourg Centre for Systems Biomedicine (LCSB) > >]
Toussaint, Marc [> >]
Kontes, Georgios [> >]
Piperidis, Savas [> >]
Autonomous Robots
Springer Science & Business Media B.V.
Yes (verified by ORBilu)
[en] Model-free robot control ; Reinforcement learning ; Probabilistic inference ; EM algorithm
[en] We address the problem of learning robot control by model-free reinforcement learning (RL). We adopt the probabilistic model for model-free RL of Vlassis and Toussaint (Proceedings of the international conference on machine learning, Montreal, Canada, 2009), and we propose a Monte Carlo EM algorithm (MCEM) for control learning that searches directly in the space of controller parameters using information obtained from randomly generated robot trajectories. MCEM is related to, and generalizes, the PoWER algorithm of Kober and Peters (Proceedings of the neural information processing systems, 2009). In the finite-horizon case MCEM reduces precisely to PoWER, but MCEM can also handle the discounted infinite-horizon case. An interesting result is that the infinite-horizon case can be viewed as a 'randomized' version of the finite-horizon case, in the sense that the length of each sampled trajectory is a random draw from an appropriately constructed geometric distribution. We provide some preliminary experiments demonstrating the effects of fixed (PoWER) vs randomized (MCEM) horizon length in two simulated and one real robot control tasks.

File(s) associated to this reference

Fulltext file(s):

Open access
09-vlassis-et-al-auro.pdfThe final publication is available at link.springer.comAuthor postprint751.76 kBView/Open

Bookmark and Share SFX Query

All documents in ORBilu are protected by a user license.