Reference : Learning navigation attractors for mobile robots with reinforcement learning and rese...
Scientific congresses, symposiums and conference proceedings : Paper published in a book
Engineering, computing & technology : Computer science
Learning navigation attractors for mobile robots with reinforcement learning and reservoir computing
Antonelo, Eric Aislan mailto [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) >]
Depeweg, Stefan [> >]
Schrauwen, Benjamin [> >]
Proceedings of the X Brazilian Congress on Computational Intelligence (CBIC)
Fortaleza, Brazil
X Brazilian Congress on Computational Intelligence (CBIC)
08-11-2011 to 10-11-2011
[en] reservoir computing ; reinforcement learning ; robot navigation
[en] Autonomous robot navigation in partially observable environments is a complex task because the state of the environment can not be completely determined only by the current sensory readings of a robot. This work uses the recently introduced paradigm for training recurrent neural networks (RNNs), called reservoir computing (RC), to model multiple navigation attractors in partially observable environments. In RC, the RNN with randomly generated fixed weights, called reservoir, projects the input into a high-dimensional dynamic space. Only the readout output layer is trained using standard linear regression techniques, and in this work, is used to approximate the state-action value function. By using a policy iteration framework, where an alternating sequence of policy improvement (samples generation from environment interaction) and policy evaluation (network training) steps are performed, the system is able to shape navigation attractors so that, after convergence, the robot follows the correct trajectory towards the goal. The experiments are accomplished using an e-puck robot extended with 8 distance sensors in a rectangular environment with an obstacle between the robot and the target region. The task is to reach the goal through the correct side of the environment, which is indicated by a temporary stimulus previously observed at the beginning of the episode. We show that the reservoir-based system (with short-term memory) can model these navigation attractors, whereas a feedforward network without memory fails to do so.

File(s) associated to this reference

Fulltext file(s):

Open access
2011_eric_cbic.pdfAuthor postprint2.03 MBView/Open

Bookmark and Share SFX Query

All documents in ORBilu are protected by a user license.