Cooperative UAVs with Asynchronous Multi-agent Learning for Remote Data Collection

This paper addresses the optimization of trajectories for multiple Unmanned Aerial Vehicles (UAVs) and bandwidth allocation to enhance energy efficiency in a general cooperative data collection problem. We focus on a practical scenario where sensor nodes (SNs) are distributed over a remote area without terrestrial infrastructures and UAVs have to make their decisions asynchronously based on local information, with inter-UAV information exchange only possible when they are within communication range. While bandwidth allocation can be solved based on local observation at each hovering point, trajectory planning is not as straightforward due to imperfect information and asynchronous decisions. To tackle these challenges, we formulate the trajectory planning problem as a Decentralized Partially Observable Semi-Markov Decision Process (Dec-POSMDP), for which we introduce an asynchronous version of a well-known algorithm called QMIX to learn UAVs' policies. We also provide empirical evidences to demonstrate the learning performance of the proposed method, as well as the robustness of the learned policies in response to varying UAV configurations.