Paper published in a book (Scientific congresses, symposiums and conference proceedings)
Reinforcement Learning for Attitude Control of a Spacecraft with Flexible Appendages
Mahfouz, Ahmed; Valiullin, Ayrat; Lukashevichus, Alexey et al.
2022In IAC 2022 congress proceedings, 73rd International Astronautical Congress (IAC)


Full Text
Publisher postprint (1.65 MB)

All documents in ORBilu are protected by a user license.

Send to


Keywords :
satellite; flexible appendages; attitude control; reinforcement learning; proximal policy optimization
Abstract :
[en] This study explores the reinforcement learning (RL) approach to constructing attitude control strategies for a LEOsatellite with flexible appendages. Attitude control system actuated by a set of three reaction wheels is considered.The satellite is assumed to move in a circular low Earth orbit under the action of gravity-gradient torque, randomdisturbance torque, and oscillations excited in flexible appendages. The control policy for rest-to-rest slew maneuversis learned via the Proximal Policy Optimization (PPO) technique. The robustness of the obtained control policy isanalyzed and compared to that of conventional controllers. The first part of the study is focused on problem formulationin terms of Markov Decision Processes, analysis of different reward-shaping techniques, and finally training the RL-agent and comparing the obtained results with the state-of-the-art RL-controllers as well as with the performance ofa commonly used quaternion feedback regulator (Lyapunov-based PD controller). We then proceed to consider thesame spacecraft with flexible appendages added to its structure. Equations of excitable oscillations are appended tothe system and coupling terms are added describing the interactions between the main rigid body and the flexiblestructures. The dynamics of the rigid spacecraft thus becomes coupled with that of its flexible appendages and thecontrol strategy should change accordingly in order to prevent actions that entail excitation of oscillation modes.Again PPO is used to learn the control policy for rest-to-rest slew maneuvers in the extended system. All in all,the proposed reinforcement learning strategy is shown to converge to a policy that matches the performance of thequaternion feedback regulator for a rigid spacecraft. It is also shown that a policy can be trained to take into accountthe highly nonlinear dynamics caused by the presence of flexible elements that need to be brought to rest in the requiredattitude. We also discuss the advantages of the reinforcement learning approach such as robustness and ability of onlinelearning pertaining to the systems that require a high level of autonomy
Disciplines :
Aerospace & aeronautics engineering
Author, co-author :
Mahfouz, Ahmed  ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > Automation
Valiullin, Ayrat
Lukashevichus, Alexey
Pritykin, Dmitry
External co-authors :
Language :
Title :
Reinforcement Learning for Attitude Control of a Spacecraft with Flexible Appendages
Publication date :
September 2022
Event name :
73rd International Astronautical Congress
Event organizer :
International Astronautical Federation
Event place :
Paris, France
Event date :
from 18-09-2022 to 22-09-2022
Main work title :
IAC 2022 congress proceedings, 73rd International Astronautical Congress (IAC)
Publisher :
International Astronautical Federation, Paris, France
Edition :
FnR Project :
FNR14302465 - Development Tool For Autonomous Constellation And Formation Control Of Microsatellites, 2019 (01/09/2020-31/08/2023) - Holger Voos
Available on ORBilu :
since 11 January 2023


Number of views
46 (0 by Unilu)
Number of downloads
72 (4 by Unilu)

Scopus citations®
Scopus citations®
without self-citations


Similar publications

Contact ORBilu