Towards incremental autonomy framework for on-orbit vision-based grasping

Barad, Kuldeep Rambhai; Martinez Luna, Carol; Dentler, Jan; Olivares Mendez, Miguel Angel

Download

Paper published in a book (Scientific congresses, symposiums and conference proceedings)

Towards incremental autonomy framework for on-orbit vision-based grasping

Barad, Kuldeep Rambhai; Martinez Luna, Carol; Dentler, Jan et al.

2021 • In Proceedings of the International Astronautical Congress, IAC-2021

Permalink
https://hdl.handle.net/10993/49355

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

IAC-21,D1,6,7,x65061.pdf

Publisher postprint (806.12 kB)

Download

All documents in ORBilu are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

OSAM; Autonomy, On-orbit Robotics, Perception; , Manipulation, Learning

Abstract :

[en] This work presents a software-oriented autonomy framework that enables the incremental development of high robotic autonomy. The autonomy infrastructure in space applications is often cost-driven and built for a narrow time/complexity domain. In domains like On-orbit Servicing Assembly and Manufacturing (OSAM), this prevents scalability and generalizability, motivating a more consistent approach for the incremental development of robotic autonomy. For this purpose, the problem of vision-based grasping is described as a building block for high autonomy of dexterous space robots. Subsequently, the need for a framework is highlighted to enable bottom-up development of general autonomy with vision-based grasping as the starting point. The preliminary framework presented here comprises three components. First, an autonomy level classification provides a clear description of the autonomous behavior of the system. The stack abstraction provides a general classification of the development layers. Finally, the generic execution architecture condenses the flow of translating a high-level task description into real-world sense-planact routines. Overall, this work lays down foundational elements towards development of general robotic autonomy for scalablity in space application domains like OSAM.

Disciplines :

Aerospace & aeronautics engineering

Author, co-author :

Barad, Kuldeep Rambhai ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > Space Robotics

Martinez Luna, Carol ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > Space Robotics

Dentler, Jan; Redwire Space Europe > Robotics

Olivares Mendez, Miguel Angel ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > Space Robotics

External co-authors :

Language :

English

Title :

Towards incremental autonomy framework for on-orbit vision-based grasping

Publication date :

29 October 2021

Event name :

International Astronautical Congress

Event organizer :

International Astronautical Federation

Event place :

Dubai, United Arab Emirates

Event date :

24-10-2021 to 29-10-2021

Audience :

International

Main work title :

Proceedings of the International Astronautical Congress, IAC-2021

FnR Project :

FNR15799985 - Modular Vision For Dynamic Grasping Of Unknown Resident Space Objects, 2021 (01/04/2021-15/01/2025) - Kuldeep Rambhai Barad

Available on ORBilu :

since 07 January 2022

Statistics

Number of views

288 (58 by Unilu)

Number of downloads

121 (8 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

Bibliography

R Ambrose, IAD Nesnas, F Chandler, BD Allen, T Fong, L Matthies, and R Mueller. Nasa technology roadmaps: Ta 4: Robotics and autonomous systems. NASA, Washington DC, 2015.
Sara A. Carioscia, Benjamin A Corbin, and Bhavya Lal. Ways forward for on-orbit servicing, assembly, and manufacturing (osam) of spacecraft. IDA Science Technology and Policy Institue Report, 2018.
Andrew Ogilvie, Justin Allport, Michael Hannah, and John Lymer. Autonomous robotic operations for on-orbit satellite servicing. In Sensors and Systems for Space Applications II, volume 6958, page 695809. International Society for Optics and Photonics, 2008.
Andrew Ogilvie, Justin Allport, Michael Hannah, and John Lymer. Autonomous satellite servicing using the orbital express demonstration manipulator system. In Proc. of the 9th International Symposium on Artificial Intelligence, Robotics and Automation in Space (i-SAIRAS’08), pages 25–29, 2008.
Jürgen Telaar, Ingo Ahrns, Stéphane Estable, Wolfgang Rackl, Marco De Stefano, Roberto Lampariello, Nuno Santos, Pedro Serra, Marco Canetri, Finn Ankersen, et al. Gnc architecture for the e. deorbit mission. In 7th European Conference for Aeronautics and Space Sciences (EUCASS), 2017.
Eve Coste-Maniere and Reid Simmons. Architecture, the backbone of robotic systems. In Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), volume 1, pages 67–72. IEEE, 2000.
Rodney Brooks. A robust layered control system for a mobile robot. IEEE journal on robotics and automation, 2(1):14–23, 1986.
R Peter Bonasso, David Kortenkamp, David P Miller, and Marc Slack. Experiences with an architecture for intelligent, reactive agents. In International Workshop on Agent Theories, Architectures, and Languages, pages 187–202. Springer, 1995.
Richard Volpe, Issa Nesnas, Tara Estlin, Darren Mutz, Richard Petras, and Hari Das. The claraty architecture for robotic autonomy. In 2001 IEEE Aerospace Conference Proceedings (Cat. No. 01TH8542), volume 1, pages 1–121. IEEE, 2001.
Guillaume Brat, Ewen Denney, Kimberley Farrell, Dimitra Giannakopoulou, Ari Jónsson, Jeremy Frank, Mark Boddy, Todd Carpenter, Tara Estlin, and Mihail Pivtoraiko. A robust compositional architecture for autonomous systems. In 2006 IEEE Aerospace Conference, pages 8–pp. IEEE, 2006.
Jeremy Frank, Ari K Jónsson, and Paul Morris. On reformulating planning as dynamic constraint satisfaction. In International Symposium on Abstraction, Reformulation, and Approximation, pages 271–280. Springer, 2000.
Vandi Verma, Tara Estlin, Ari Jónsson, Corina Pasareanu, Reid Simmons, and Kam Tso. Plan execution interchange language (plexil) for executable plans and command sequences. In International symposium on artificial intelligence, robotics and automation in space (iSAIRAS), 2005.
Jorge Ocón, Juan Manuel Delfa, Alberto Medina, Daisy Lachat, Robert Marc, Mark Woods, Iain Wallace, Andrew Ian Coles, Amanda Jane Coles, Derek Long, et al. Ergo: A framework for the development of autonomous robots. In 14th Symposium on Advanced Space Technologies in Robotics and Automation, 2017.
Conor McGann, Frederic Py, K Rajan, H Thomas, R Henthorn, and R McEwen. T-rex: A model-based architecture for auv control. In 3rd Workshop on Planning and Plan Execution for Real-World Systems, volume 2007, 2007.
Herman Bruyninckx. Open robot control software: the orocos project. In Proceedings 2001 ICRA. IEEE international conference on robotics and automation (Cat. No. 01CH37164), volume 3, pages 2523–2528. IEEE, 2001.
M Muñoz Arancón, Giuseppe Montano, Malte Wirkus, Kilian Hoeflinger, Daniel Silveira, Nikolaos Tsiogkas, Jérôme Hugues, Herman Bruyninckx, Iulia Dragomir, and Ali Muhammad. Esrocos: a robotic operating system for space and terrestrial applications. In 14th Symposium on Advanced Space Technologies in Robotics and Automation (ASTRA 2017), page pp. 1, 2017.
Aviv Tamar, Yi Wu, Garrett Thomas, Sergey Levine, and Pieter Abbeel. Value iteration networks. In Advances in Neural Information Processing Systems (NeurIPS), page pp. 2154–2162, 2016.
Aravind Srinivas, Allan Jabri, Pieter Abbeel, Sergey Levine, and Chelsea Finn. Universal planning networks: Learning generalizable representations for visuomotor control. In International Conference on Machine Learning, pages 4732–4741. PMLR, 2018.
Bruce A Aikenhead, Robert G Daniell, and Frederick M Davis. Canadarm and the space shuttle. Journal of Vacuum Science & Technology A: Vacuum, Surfaces, and Films, 1(2):126–132, 1983.
Roel Boumans and Cock Heemskerk. The european robotic arm for the international space station. Robotics and Autonomous systems, 23(1-2):17–27, 1998.
Elliott Coleshill, Layi Oshinowo, Richard Rembala, Bardia Bina, Daniel Rey, and Shelley Sindelar. Dextre: Improving maintenance operations on the international space station. Acta Astronautica, 64(9-10):869–874, 2009.
Jeannette Bohg, Antonio Morales, Tamim Asfour, and Danica Kragic. Data-driven grasp synthesis—a survey. IEEE Transactions on Robotics, 30(2):289–309, 2013.
Anis Sahbani, Sahar El-Khoury, and Philippe Bidaud. An overview of 3d object grasp synthesis algorithms. Robotics and Autonomous Systems, 60(3):326–336, 2012.
Jeffrey Mahler, Florian T Pokorny, Brian Hou, Mel-rose Roderick, Michael Laskey, Mathieu Aubry, Kai Kohlhoff, Torsten Kröger, James Kuffner, and Ken Goldberg. Dex-net 1.0: A cloud-based network of 3d objects for robust grasp planning using a multiarmed bandit model with correlated rewards. In 2016 IEEE international conference on robotics and automation (ICRA), pages 1957–1964. IEEE, 2016.
Yun Jiang, Stephen Moseson, and Ashutosh Saxena. Efficient grasping from rgbd images: Learning using a new rectangle representation. In 2011 IEEE International conference on robotics and automation, pages 3304–3311. IEEE, 2011.
Antonio Morales, Tamim Asfour, Pedram Azad, Steffen Knoop, and Rudiger Dillmann. Integrated grasp planning and visual object localization for a humanoid robot with five-fingered hands. In 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5663–5668. IEEE, 2006.
Carlos Rubert, Beatriz León, Antonio Morales, and Joaquín Sancho-Bru. Characterisation of grasp quality metrics. Journal of Intelligent & Robotic Systems, 89(3):319–342, 2018.
Bhubaneswar Mishra. Grasp metrics: Optimality and complexity. In Algorithmic Foundations of Robotics, pages 137–166. AK Peters, 1995.
Ravi Balasubramanian, Ling Xu, Peter D Brook, Joshua R Smith, and Yoky Matsuoka. Physical human interactive guidance: Identifying grasping principles from human-planned grasps. IEEE Transactions on Robotics, 28(4):899–910, 2012.
Johan Tegin, Staffan Ekvall, Danica Kragic, Jan Wikander, and Boyko Iliev. based learning and control for automatic grasping. Intelligent Service Robotics, 2(1):23–30, 2009.
Rosen Diankov. Automated construction of robotic manipulation programs. PhD thesis, Carnegie Mellon University, 2010.
Andrew T Miller, Steffen Knoop, Henrik I Christensen, and Peter K Allen. Automatic grasp planning using shape primitives. In 2003 IEEE International Conference on Robotics and Automation (Cat. No. 03CH37422), volume 2, pages 1824–1829. IEEE, 2003.
Corey Goldfeder, Peter K Allen, Claire Lackner, and Raphael Pelossof. Grasp planning via decomposition trees. In Proceedings 2007 IEEE International Conference on Robotics and Automation, pages 4679–4684. IEEE, 2007.
Renaud Detry, Nicolas Pugeault, and Justus H Piater. A probabilistic framework for 3d visual object representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(10):1790–1803, 2009.
Lorenzo Pasqualetto Cassinis, Robert Fonod, and Eberhard Gill. Review of the robustness and applicability of monocular pose estimation systems for relative navigation with an uncooperative spacecraft. Progress in Aerospace Sciences, 110:100548, 2019.
Sumant Sharma and Simone D’Amico. Neural network-based pose estimation for noncooperative spacecraft rendezvous. IEEE Transactions on Aerospace and Electronic Systems, 56(6):4638–4658, 2020.
Tae Ha Park, Sumant Sharma, and Simone D’Amico. Towards robust learning-based pose estimation of noncooperative spacecraft. In 2019 AAS/AIAA Astrodynamics Specialist Conference, Portland, Maine. AIAA, 2019.
Mate Kisantal, Sumant Sharma, Tae Ha Park, Dario Izzo, Marcus Märtens, and Simone D’Amico. Satellite pose estimation challenge: Dataset, competition design, and results. IEEE Transactions on Aerospace and Electronic Systems, 56(5):4083–4098, 2020.
Mihai Andries, Ricardo Omar Chavez-Garcia, Raja Chatila, Alessandro Giusti, and Luca Maria Gambardella. Affordance equivalences in robotics: a formalism. Frontiers in neurorobotics, 12:26, 2018.
Jeffrey Mahler, Jacky Liang, Sherdil Niyaz, Michael Laskey, Richard Doan, Xinyu Liu, Juan Aparicio Ojea, and Ken Goldberg. Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. arXiv preprint arXiv:1703.09312, 2017.
Sergey Levine, Peter Pastor, Alex Krizhevsky, Julian Ibarz, and Deirdre Quillen. Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. The International Journal of Robotics Research, 37(4-5):421–436, 2018.
Arsalan Mousavian, Clemens Eppner, and Dieter Fox. 6-dof graspnet: Variational grasp generation for object manipulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2901–2910, 2019.
Douglas Morrison, Peter Corke, and Jürgen Leitner. Learning robust, real-time, reactive robotic grasping. The International journal of robotics research, 39(2-3):183–201, 2020.
Joseph Redmon and Anelia Angelova. Real-time grasp detection using convolutional neural networks. In 2015 IEEE International Conference on Robotics and Automation (ICRA), pages 1316–1322. IEEE, 2015.
Claire Dune, Anthony Remazeilles, Eric Marchand, and Cédric Leroux. Vision-based grasping of unknown objects to improve disabled people autonomy. In Robotics: Science and Systems Manipulation Workshop: Intelligence in Human Environments., 2008.
Gary M Bone, Andrew Lambert, and Mark Edwards. Automated modeling and robotic grasping of unknown three-dimensional objects. In 2008 IEEE International Conference on Robotics and Automation, pages 292–298. IEEE, 2008.
Dirk Kraft, Nicolas Pugeault, Emre Başeski, MILA POPOVIĆ, Danica Kragić, Sinan Kalkan, Florentin Wörgötter, and Norbert Krüger. Birth of the object: Detection of objectness and extraction of object shape through object–action complexes. International Journal of Humanoid Robotics, 5(02):247–265, 2008.
Ulrich Viereck, Andreas Pas, Kate Saenko, and Robert Platt. Learning a visuomotor controller for real world robotic grasping using simulated depth images. In Conference on Robot Learning, pages 291–300. PMLR, 2017.
Dmitry Kalashnikov, Alex Irpan, Peter Pastor, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, et al. Scalable deep reinforcement learning for vision-based robotic manipulation. In Conference on Robot Learning, pages 651–673. PMLR, 2018.
Shuran Song, Andy Zeng, Johnny Lee, and Thomas Funkhouser. Grasping in the wild: Learning 6dof closed-loop grasping from low-cost demonstrations. IEEE Robotics and Automation Letters, 5(3):4978–4985, 2020.
Leslie Pack Kaelbling. Foundations of learning in autonomous agents. Robotics and Autonomous Systems, 6(2), 1991. Also published in Toward Learning Robots, W. Van de Velde, Ed., MIT Press, 1991.
Li Liu, Wanli Ouyang, Xiaogang Wang, Paul Fieguth, Jie Chen, Xinwang Liu, and Matti Pietikäinen. Deep learning for generic object detection: A survey. International journal of computer vision, 128(2):261–318, 2020.
Andrej Karpathy and Li Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3128–3137, 2015.
Roberto Martín-Martín, Michelle A Lee, Rachel Gardner, Silvio Savarese, Jeannette Bohg, and Animesh Garg. Variable impedance control in end-effector space: An action space for reinforcement learning in contact-rich tasks. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1010–1017. IEEE, 2019.
Arthur Allshire, Roberto Martín-Martín, Charles Lin, Shawn Manuel, Silvio Savarese, and Animesh Garg. Laser: Learning a latent action space for efficient reinforcement learning. In ICRA 2021, 2021.
Kuan Fang, Yuke Zhu, Animesh Garg, Silvio Savarese, and Li Fei-Fei. Dynamics learning with cascaded variational inference for multi-step manipulation. In Conference on Robot Learning, 2019, 2019.
Ajay Mandlekar, Fabio Ramos, Byron Boots, Silvio Savarese, Li Fei-Fei, Animesh Garg, and Dieter Fox. Iris: Implicit reinforcement without interaction at scale for learning control from offline robot manipulation data. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 4414–4420. IEEE, 2020.
Yunzhu Li, Antonio Torralba, Anima Anandkumar, Dieter Fox, and Animesh Garg. Causal discovery in physical systems from videos. Advances in Neural Information Processing Systems, 33, 2020.
Simon Patane, Eric R Joyce, Michael P Snyder, and Paul Shestople. Archinaut: In-space manufacturing and assembly for next-generation space habitats. In AIAA SPACE and astronautics forum and exposition, page 5227, 2017.