Hu-bot: promoting the cooperation between humans and mobile robots

Human-in-the-loop; Human–robot cooperation; Mobile robots; Reinforcement learning; Empirical validation; Hardware experiment; Human assistance; Physical robots; Robot behavior; Artificial Intelligence

Abstract :

[en] This paper investigates human–robot collaboration in a novel setup: a human helps a mobile robot that can move and navigate freely in an environment. Specifically, the human helps by remotely taking over control during the learning of a task. The task is to find and collect several items in a walled arena, and Reinforcement Learning is used to seek a suitable controller. If the human observes undesired robot behavior, they can directly issue commands for the wheels through a game joystick. Experiments in a simulator showed that human assistance improved robot behavior efficacy by 30% and efficiency by 12%. The best policies were also tested in real life, using physical robots. Hardware experiments showed no significant difference concerning the simulations, providing empirical validation of our approach in practice.

Disciplines :

Computer science

Author, co-author :

Miras, Karine ; Department of Computer Science, Vrije Universiteit Amsterdam, Amsterdam, Netherlands

MOCANU, Decebal Constantin ; University of Luxembourg > Faculty of Science, Technology and Medicine (FSTM) > Department of Computer Science (DCS) ; Faculty of Electrical Engineering, Mathematics and Computer Science, Universiteit Twente, Enschede, Netherlands ; Department of Mathematics and Computer Science, Eindhoven University of Technology, Eindhoven, Netherlands

Eiben, A.E.; Department of Computer Science, Vrije Universiteit Amsterdam, Amsterdam, Netherlands

External co-authors :

yes

Language :

English

Title :

Hu-bot: promoting the cooperation between humans and mobile robots

Publication date :

13 January 2023

Journal title :

Neural Computing and Applications

ISSN :

0941-0643

eISSN :

1433-3058

Publisher :

Springer Science and Business Media Deutschland GmbH

Volume :

Issue :

Pages :

16841 - 16852

Peer reviewed :

Peer Reviewed verified by ORBi

Focus Area :

Computational Sciences

Additional URL :

https://link.springer.com/content/pdf/10.1007/s00521-022-08061-z.pdf

Funders :

Ministerie van Onderwijs, Cultuur en Wetenschap

Funding text :

This research was funded by the Hybrid Intelligence Center, a 10-year program funded by the Dutch Ministry of Education, Culture and Science through the Netherlands Organization for Scientific Research ( https://www.hybrid-intelligence-centre.nl ), Grant No. 024.004.022.

Available on ORBilu :

since 09 May 2024

Statistics

Number of views

65 (6 by Unilu)

Number of downloads

17 (2 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenAlex citations

WoS citations^™

Bibliography

Park J, Nakamura KA (2015) Labor-saving and employment in a plant factory. Environ Control Biol 53:89–92 DOI: 10.2525/ecb.53.89
Connell JK (1993) The rise of labour migration to Japan. University of Sydney, Sydney
Takayama L, Ju W, Nass CBd (2008) Dangerous and dull: what everyday people think robots should do. In: ACM/IEEE international conference on human–robot interaction (HRI), pp 25–32
Jämsä-Jounela Sirkka-Liisa (2007) Future trends in process automation. Annu Rev Control 31:211–220 DOI: 10.1016/j.arcontrol.2007.08.003
Repperger D, Phillips C (2009) The human role in automation. In: Handbook of automation, pp 295–304
Zhang R, Torabi F, Guan L, Ballard D, Stone P (2019) Leveraging human guidance for deep reinforcement learning tasks. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence, IJCAI-19, p 7
Ho M, Littman M, MacGlashan J, Cushman F, Austerweil J (2016) Showing versus doing: teaching by demonstration. Adv Neural Inf Process Syst 29:3027–3035
Argall BD, Chernova S, Veloso M, Browning B (2009) A survey of robot learning from demonstration. Robot Auton Syst 57(5):469–483 DOI: 10.1016/j.robot.2008.10.024
Brys T, Harutyunyan A, Suay HB, Chernova S, Taylor ME, Nowé A (2015) Reinforcement learning from demonstration through shaping. In: Twenty-fourth international joint conference on artificial intelligence
Hester T, Vecerik M, Pietquin O, Lanctot M, Schaul T, Piot B, Horgan D, Quan J, Sendonaris A, Osband I et al (2018) Deep q-learning from demonstrations. In: Proceedings of the AAAI conference on artificial intelligence, vol 32
Nair A, McGrew B, Andrychowicz M, Zaremba W, Abbeel P (2018) Overcoming exploration in reinforcement learning with demonstrations. In: 2018 IEEE international conference on robotics and automation (ICRA), pp 6292–6299. IEEE
Fournier P, Sigaud O, Colas C, Chetouani MC (2021) Curriculum learning and imitation for object control in nonrewarding environments. IEEE Trans Cogn Dev Syst 13:239–248 DOI: 10.1109/TCDS.2019.2933371
Hussein A, Gaber M, Elyan E, Jayne C (2017) Imitation learning: a survey of learning methods. ACM Comput Surv (CSUR) 50:1–35 DOI: 10.1145/3054912
Kelly M, Sidrane C, Driggs-Campbell K, Kochenderfer MJ (2019) Hg-dagger: interactive imitation learning with human experts. In: International conference on robotics and automation (ICRA), pp 8077–8083. IEEE
Rosenstein M, Barto A, Si J, Barto A, Powell W, Wunsch D (2004) Supervised actor-critic reinforcement learning. In: Learning and approximate dynamic programming: scaling up to the real world, pp 359–380
Goecks VG, Gremillion GM, Lawhern VJ, Valasek J, Waytowich NR (2019) Efficiently combining human demonstrations and interventions for safe training of autonomous systems in real-time. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 2462–2470
Saunders W, Sastry G, Stuhlm”uller A, Evans OTwE (2018) Towards safe reinforcement learning via human intervention. In: Proceedings of the 17th international conference on autonomous agents and multiagent systems, pp 2067–2069
Najar A, Sigaud O, Chetouani M (2020) Interactively shaping robot behaviour with unlabeled human instructions. Auton Agents Multi-Agent Syst 34:1–35 DOI: 10.1007/s10458-020-09459-6
Pérez-Dattari R, Celemin C, Ruiz-del-Solar J, Kober J (2018) Interactive learning with corrective feedback for policies based on deep neural networks. In: International symposium on experimental robotics, pp 353–363. Springer
Najar A, Sigaud O, Chetouani M (2016) Training a robot with evaluative feedback and unlabeled guidance signals. In: 2016 25th IEEE international symposium on robot and human interactive communication (RO-MAN), pp 261–266
Rutard F, Sigaud O (2020) Chetouani: Tirl: enriching actor-critic rl with non-expert human teachers and a trust model. In: 2020 29th IEEE international conference on robot and human interactive communication (RO-MAN), pp 604–611
Knox WB, Stone P (2009) Interactively shaping agents via human reinforcement: the tamer framework. In: Proceedings of the fifth international conference on knowledge capture, pp 9–16
Sutton R, Barto A (2018) Reinforcement learning: an introduction
Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA (2017) Deep reinforcement learning: a brief survey. IEEE Signal Process Mag 34(6):26–38 DOI: 10.1109/MSP.2017.2743240
Haarnoja T, Zhou A, Abbeel P, Levine S (2018) Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: International conference on machine learning, pp 1861–1870. PMLR
Fujimoto S, Hoof H, Meger D. Addressing function approximation error in actor-critic methods. In: Proceedings of the 35th international conference on machine learning, vol 80, pp 1587–1596
Bischl B, Binder M, Lang M, Pielok T, Richter J, Coors S, Thomas J, Ullmann T, Becker M, Boulesteix A-L et al (2021) Hyperparameter optimization: foundations, algorithms, best practices and open challenges. arXiv preprint arXiv:2107.05847
Laud AD (2004) Theory and application of reward shaping in reinforcement learning
Acemoglu D, Restrepo P (2020) Robots and jobs: evidence from us labor markets. J Polit Econ 128(6):2188–2244 DOI: 10.1086/705716