Visual Servoing for Robotic On-Orbit Servicing: A Survey

On-orbit-servicing; Robotics capabilities; Servicing missions; Space manipulators; Visual-servoing; Robotics; Autonomous robotics; Vision-based systems; Robotic manipulation; Control and optimization; Computer science; Space industry

Abstract :

[en] On-orbit servicing (OOS) activities will power the next big step for sustainable exploration and commercialization of space. Developing robotic capabilities for autonomous OOS operations is a priority for the space industry. Visual Servoing (VS) enables robots to achieve the precise manoeuvres needed for critical OOS missions by utilizing visual information for motion control. This article presents an overview of existing VS approaches for autonomous OOS operations with space manipulator systems (SMS). We divide the approaches according to their contribution to the typical phases of a robotic OOS mission: a) Recognition, b) Approach, and c) Contact. We also present a discussion on the reviewed VS approaches, identifying current trends. Finally, we highlight the challenges and areas for future research on VS techniques for robotic OOS.

Research center :

Interdisciplinary Centre for Security, Reliability and Trust (SnT) > SpaceR – Space Robotics

Disciplines :

Computer science

Author, co-author :

AMAYA MEJIA, Lina Maria ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > Space Robotics

Ghita, Mohamed; Redwire Space Europe, Luxembourg

DENTLER, Jan Eric ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust > Automation ; Redwire Space Europe, Luxembourg

OLIVARES MENDEZ, Miguel Angel ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > Space Robotics

MARTINEZ LUNA, Carol ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > Space Robotics

External co-authors :

yes

Language :

English

Title :

Visual Servoing for Robotic On-Orbit Servicing: A Survey

Publication date :

September 2024

Event name :

2024 International Conference on Space Robotics (iSpaRo)

Event place :

Luxembourg, Lux

Event date :

24-06-2024 => 27-06-2024

Main work title :

2024 International Conference on Space Robotics, iSpaRo 2024

Publisher :

Institute of Electrical and Electronics Engineers Inc.

ISBN/EAN :

9798350367232

Peer reviewed :

Peer reviewed

Focus Area :

Computational Sciences

Development Goals :

9. Industry, innovation and infrastructure

Additional URL :

http://xplorestaging.ieee.org/ielx8/10685845/10685848/10687516.pdf?arnumber=10687516

FnR Project :

FNR18075131 - Enhancing Autonomous Space Robotic Manipulation trhough Efficient Visuomotor Skills (15/11/23 - 14/11/26) - Lina Maria Amaya Mejia

Name of the research project :

Enhancing Autonomous Space Robotic Manipulation trhough Efficient Visuomotor Skills

Funders :

Fonds National de la Recherche (FNR) Industrial Fellowship under Redwire Space Europe

Funding number :

18075131

Funding text :

This work is supported by the Luxembourg National Research Fund Industrial Fellowship grant (N.18075131) and Redwire Space Europe.

Available on ORBilu :

since 10 January 2025

Statistics

Number of views

153 (2 by Unilu)

Number of downloads

162 (0 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

OpenAlex citations

WoS citations^™

Bibliography

R. S. Jakhu and J. N. Pelton, "On-orbit servicing, active debris removal, and related activities," Global Space Governance: An International Study, pp. 331-356, 2017.
B. Ma et al., "Advances in space robots for on-orbit servicing: A comprehensive review," Advanced Intelligent Systems, p. 2200397, 2023.
G. Gibbs and S. Sachdev, "Canada and the international space station program: overview and status," Acta Astronautica, vol. 51, no. 1-9, pp. 591-600, 2002.
A. Flores-Abad et al., "A review of space robotics technologies for on-orbit servicing," Progress in aerospace sciences, vol. 68, pp. 1-26, 2014.
Y. Wang et al., "Review of research on the chinese space station robots," in Intelligent Robotics and Applications: 12th International Conference, ICIRA 2019, Shenyang, China, August 8-11, 2019, Proceedings, Part IV 12. Springer, 2019, pp. 423-430.
S. Nishida et al., "Engineering test satellite vii robot experiment subsystem," Journal of the Robotics Society of Japan, vol. 17, no. 8, pp. 1062-1066, 1999.
A. Ogilvie et al., "Autonomous robotic operations for on-orbit satellite servicing," in Sensors and Systems for Space Applications II, vol. 6958. SPIE, 2008, pp. 50-61.
The National Aeronautics and Space Administration. On-orbit servicing, assembly, and manufacturing 1 (osam-1). [Online]. Available: https://www.nasa.gov/mission/ on-orbit-servicing-assembly-and-manufacturing-1/
Canadian Space agency. Power data grapple fixture. [Online]. Available: https://www.asc-csa.gc.ca/eng/multimedia/search/image/8413
B. M. Moghaddam and R. Chhabra, "On the guidance, navigation and control of in-orbit space robotic missions: A survey and prospective vision," Acta Astronautica, vol. 184, pp. 70-100, 2021.
J. L. Ramon et al., "On-orbit free-floating manipulation using a twoarm robotic system," in ROBOVIS, 2021, pp. 57-63.
E. Papadopoulos et al., "Robotic manipulation and capture in space: A survey," Frontiers in Robotics and AI, p. 228, 2021.
B. Siciliano et al., Springer handbook of robotics. Springer, 2008, vol. 200.
F. Sellmaier et al., "On-orbit servicing missions: Challenges and solutions for spacecraft operations," in SpaceOps 2010 Conference Delivering on the Dream Hosted by NASA Marshall Space Flight Center and Organized by AIAA, 2010, p. 2159.
A. Alabdo et al., "Fpga-based architecture for direct visual control robotic systems," Mechatronics, vol. 39, pp. 204-216, 2016.
F. Chaumette and S. Hutchinson, "Visual servo control. i. basic approaches," IEEE Robotics & Automation Magazine, vol. 13, no. 4, pp. 82-90, 2006.
J. Hansen, "Integrated eye-in-hand/eye-to-hand visual servoing," 2012.
V. D. Cong and L. D. Hanh, "A new decoupled control law for imagebased visual servoing control of robot manipulators," International Journal of Intelligent Robotics and Applications, vol. 6, no. 3, pp. 576-585, 2022.
G. Palmieri et al., "A comparison between position-based and imagebased dynamic visual servoings in the control of a translating parallel manipulator," Journal of Robotics, vol. 2012, 2012.
Z. Machkour et al., "Classical and deep learning based visual servoing systems: a survey on state of the art," Journal of Intelligent & Robotic Systems, vol. 104, no. 1, p. 11, 2022.
T. Li et al., "Hybrid uncalibrated visual servoing control of harvesting robots with rgb-d cameras," IEEE Transactions on Industrial Electronics, vol. 70, no. 3, pp. 2729-2738, 2022.
H.-Y. Lin et al., "Semantic segmentation and 6dof pose estimation using rgb-d images and deep neural networks," in 2021 IEEE 30th International Symposium on Industrial Electronics (ISIE). IEEE, 2021, pp. 1-6.
R. Opromolla et al., "A review of cooperative and uncooperative spacecraft pose determination techniques for close-proximity operations," Progress in Aerospace Sciences, vol. 93, pp. 53-72, 2017.
M. Kalaitzakis et al., "Fiducial markers for pose estimation: Overview, applications and experimental comparison of the artag, apriltag, aruco and stag markers," Journal of Intelligent & Robotic Systems, vol. 101, pp. 1-26, 2021.
N. Inaba and M. Oda, "Autonomous satellite capture by a space robot: world first on-orbit experiment on a japanese robot satellite ets-vii," in Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), vol. 2. IEEE, 2000, pp. 1169-1174.
R. T. Howard et al., "The advanced video guidance sensor: Orbital express and the next generation," in AIP Conference Proceedings, vol. 969, no. 1. American Institute of Physics, 2008, pp. 717-724.
D. Wenberg et al., "Development of on-orbit assembly demonstrator in 3u cubesat form factor," in 2020 IEEE Aerospace Conference. IEEE, 2020, pp. 1-11.
J. L. Ramon et al., "Task space control for on-orbit space robotics using a new ros-based framework," Simulation Modelling Practice and Theory, p. 102790, 2023.
G. J. Arantes, "Rendezvous with a non-cooperating target," Ph.D. dissertation, Universitat Bremen, 2011.
W. Xu et al., "Non-holonomic path planning of space robot based on genetic algorithm," in 2006 IEEE International Conference on Robotics and Biomimetics. IEEE, 2006, pp. 1471-1476.
A. Shademan et al., "Robust uncalibrated visual servoing for autonomous on-orbit-servicing," in Proceedings of the i-SAIRAS, 2010.
F. Aghili et al., "Fault-tolerant position/attitude estimation of freefloating space objects using a laser range sensor," IEEE Sensors Journal, vol. 11, no. 1, pp. 176-185, 2010.
F. Aghili and C.-Y. Su, "Robust relative navigation by integration of icp and adaptive kalman filter using laser scanner and imu," IEEE/ASME Transactions on Mechatronics, vol. 21, no. 4, pp. 2015- 2026, 2016.
P. Mithun et al., "Image based visual servoing for tumbling objects," in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018, pp. 2901-2908.
R. Wang et al., "Research on a visual servo method of a manipulator based on velocity feedforward," Space: Science & Technology, 2021.
S. S. Lal, "Visual servo based space robotic docking for active space debris removal," 2021.
F. Aghili, "A prediction and motion-planning scheme for visually guided robotic capturing of free-floating tumbling objects with uncertain dynamics," IEEE Transactions on Robotics, vol. 28, no. 3, pp. 634-649, 2012.
-, "Fault-tolerant and adaptive visual servoing for capturing moving objects," IEEE/ASME Transactions on Mechatronics, vol. 27, no. 3, pp. 1773-1783, 2021.
R. Lampariello and G. Hirzinger, "Generating feasible trajectories for autonomous on-orbit grasping of spinning debris in a useful time," in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2013, pp. 5652-5659.
A. A. Hafez et al., "Reactionless visual servoing of a multi-arm space robot combined with other manipulation tasks," Robotics and Autonomous Systems, vol. 91, pp. 1-10, 2017.
J. L. Ramon et al., "Direct visual servoing and interaction control for a two-arms on-orbit servicing spacecraft," Acta Astronautica, vol. 192, pp. 368-378, 2022.
F. Aghili, "Autonomous sequential sub-manoeuvres in pre-and postcapturing space objects using obstructed 3-d vision data," IEEE Transactions on Aerospace and Electronic Systems, 2023.
G. Ma et al., "Hand-eye servo and impedance control for manipulator arm to capture target satellite safely," Robotica, vol. 33, no. 4, pp. 848-864, 2015.
C. Liang et al., "A visual-based servo control method for space manipulator assisted docking," in 2022 IEEE 6th Information Technology and Mechatronics Engineering Conference (ITOEC), vol. 6. IEEE, 2022, pp. 72-76.
A. Mahmood et al., "Visual monitoring and servoing of a cutting blade during telerobotic satellite servicing," in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 1903-1908.
P. Palmieri et al., "Inflatable robotic manipulator for space debris mitigation by visual servoing," in 2023 9th International Conference on Automation, Robotics and Applications (ICARA). IEEE, 2023, pp. 175-179.