Learning of Control Behaviours in Flying Manipulation

Doctoral thesis (2020)

Machine learning is an ever-expanding field of research with a wide range of potential applications. It has been increasingly used in different robotics tasks enhancing their autonomy and intelligent ... [more ▼]

Machine learning is an ever-expanding field of research with a wide range of potential applications. It has been increasingly used in different robotics tasks enhancing their autonomy and intelligent behaviour. This thesis presents how machine learning techniques can enhance the decision-making ability for control tasks in aerial robots as well as amplify the safety, thus broadly improving their autonomy levels. The work starts with the development of a lightweight approach for identifying degradations of UAV hardware-related components, using traditional machine learning methods. By analysing the flight data stream from a UAV following a predefined mission, it predicts the level of degradation of components at early stages. In that context, real-world experiments have been conducted, showing that such approach can be used as a safety system during different experiments, where the flight path of the vehicle is defined a priori. The next objective of this thesis is to design intelligent control policies for flying robots with highly nonlinear dynamics, operating in continuous state-action setting, using model-free reinforcement learning methods. To achieve this objective, first, the nuances and potentials of reinforcement learning have been investigated. As a result, numerous insights and strategies have been pointed out for crafting efficient reward functions that lead to successful learning performance. Finally, a learning-based controller is provided for controlling a hexacopter UAV with 6-DoF, to perform stable navigation and hovering actions by directly mapping observations to low-level motor commands. To increase the complexity of the given objective, the degrees of freedom of the robotic platform is upgraded to 7-DoF, using a flying manipulation as learning agent. In this case, the agent learns to perform a mission composed of take-off, navigation, hovering and end-effector positioning tasks. Later, to demonstrate the effectiveness of the proposed controller and its ability to handle higher number of degrees of freedom, the flying manipulation has been extended to a robotic platform with 8-DoF. To overcome several challenges of reinforcement learning, the RotorS Gym experimental framework has been developed, providing a safe and close to real simulated environment for training multirotor systems. To handle the increasingly growing complexity of learning tasks, the Cyber Gym Robotics platform has been designed, which extends the RotorS Gym framework by several core functionalities. For instance, it offers an additional mission controller that allows to decompose complex missions into several subtasks, thus accelerating and facilitating the learning process. Yet another advantage of the Cyber Gym Robotics platform is its modularity which allows to seamlessly switch both, learning algorithms as well as agents. To validate these claims, real-world experiments have been conducted, demonstrating that the model trained in the simulation can be transferred onto a real physical robot with only minor adaptations. [less ▲]

Real time degradation identification of UAV using machine learning techniques

in International Conference on Unmanned Aircraft Systems ICUAS. Miami, USA, 2017 (2017, June 13)

The usages and functionalities of Unmanned Aerial Vehicles (UAV) have grown rapidly during the last years. They are being engaged in many types of missions, ranging from military to agriculture passing by ... [more ▼]

The usages and functionalities of Unmanned Aerial Vehicles (UAV) have grown rapidly during the last years. They are being engaged in many types of missions, ranging from military to agriculture passing by entertainment and rescue or even delivery. Nonetheless, for being able to perform such tasks, UAVs have to navigate safely in an often dynamic and partly unknown environment. This brings many challenges to overcome, some of which can lead to damages or degradations of different body parts. Thus, new tools and methods are required to allow the successful analysis and identification of the different threats that UAVs have to manage during their missions or flights. Various approaches, addressing this domain, have been proposed. However, most of them typically identify the changes in the UAVs behavior rather than the issue. This work presents an approach, which focuses not only on identifying degradations of UAVs during flights, but estimate the source of the failure as well. [less ▲]