Cooperative Multi-Agent Deep Reinforcement Learning for Resource Management in Full Flexible VHTS Systems

in IEEE Transactions on Cognitive Communications and Networking (2022), 8(1),

Very high throughput satellite (VHTS) systems are expected to have a huge increase in traffic demand in the near future. Nevertheless, this increase will not be uniform over the entire service area due to ... [more ▼]

Very high throughput satellite (VHTS) systems are expected to have a huge increase in traffic demand in the near future. Nevertheless, this increase will not be uniform over the entire service area due to the non-uniform distribution of users and changes in traffic demand during the day. This problem is addressed by using flexible payload architectures, which allow the allocation of payload resources flexibly to meet the traffic demand of each beam, leading to dynamic resource management (DRM) approaches. However, DRM adds significant complexity to VHTS systems, so in this paper we discuss the use of one reinforcement learning (RL) algorithm and two deep reinforcement learning (DRL) algorithms to manage the resources available in flexible payload architectures for DRM. These algorithms are Q-Learning (QL), Deep Q-Learning (DQL) and Double Deep Q-Learning (DDQL) which are compared based on their performance, complexity and added latency. On the other hand, this work demonstrates the superiority a cooperative multiagent (CMA) decentralized distribution has over a single agent (SA). [less ▲]

Convolutional Neural Networks for Flexible Payload Management in VHTS Systems

in IEEE Systems Journal (2021), 15(3), 4675-4686

Very high throughput satellite (VHTS) systems are expected to have a large increase in traffic demand in the near future. However, this increase will not be uniform throughout the service area due to the ... [more ▼]

Very high throughput satellite (VHTS) systems are expected to have a large increase in traffic demand in the near future. However, this increase will not be uniform throughout the service area due to the nonuniform user distribution, and the changing traffic demand during the day. This problem is addressed using flexible payload architectures, enabling the allocation of the payload resources in a flexible manner to meet traffic demand of each beam, leading to dynamic resource management (DRM) approaches. However, DRM adds significant complexity to the VHTS systems, which is why in this article, we are analyzing the use of convolutional neural networks (CNNs) to manage the resources available in flexible payload architectures for DRM. The VHTS system model is first outlined, for introducing the DRM problem statement and the CNN-based solution. A comparison between different payload architectures is performed in terms of DRM response, and the CNN algorithm performance is compared by three other algorithms, previously suggested in the literature to demonstrate the effectiveness of the suggested approach and to examine all the challenges involved. [less ▲]

Optimization in VHTS Satellite System Design with Irregular Beam Coverage for Non-Uniform Traffic Distribution

in Remote Sensing (2021), 13(13),

Very High Throughput Satellites (VHTS) have a pivotal role in complementing terrestrial networks to increase traffic demand. VHTS systems currently assume a uniform distribution of traffic in the service ... [more ▼]

Very High Throughput Satellites (VHTS) have a pivotal role in complementing terrestrial networks to increase traffic demand. VHTS systems currently assume a uniform distribution of traffic in the service area, but in a real system, traffic demands are not uniform and are dynamic. A possible solution is to use flexible payloads, but the cost of the design increases considerably. On the other hand, a fixed payload that uses irregular beam coverage depending on traffic demand allows maintaining the cost of a fixed payload while minimizing the error between offered and required capacity. This paper presents a proposal for optimizing irregular beams coverage and beam pattern, minimizing the costs per Gbps in orbit, the Normalized Coverage Error, and Offered Capacity Error per beam. We present the analysis and performance for the case study and compare it with a previous algorithm for a uniform coverage area. [less ▲]

Forward Link Optimization for the Design of VHTS Satellite Networks

in Electronics (2020), 9(3),

The concept of geostationary VHTS (Very High Throughput Satellites) is based on multibeam coverage with intensive frequency and polarization reuse, in addition to the use of larger bandwidths in the ... [more ▼]

The concept of geostationary VHTS (Very High Throughput Satellites) is based on multibeam coverage with intensive frequency and polarization reuse, in addition to the use of larger bandwidths in the feeder links, in order to provide high capacity satellite links at a reduced cost per Gbps in orbit. The dimensioning and design of satellite networks based on VHTS imposes the analysis of multiple trade-offs to achieve an optimal solution in terms of cost, capacity, and the figure of merit of the user terminal. In this paper, we propose a new method for sizing VHTS satellite networks based on an analytical expression of the forward link CINR (Carrier-to-Interference-plus-Noise Ratio) that is used to evaluate the trade-off of different combinations of system parameters. The proposed method considers both technical and commercial requirements as inputs, including the constraints to achieve the optimum solution in terms of the user G/T, the number of beams, and the system cost. The cost model includes both satellite and ground segments. Exemplary results are presented with feeder links using Q/V bands, DVB-S2X and transmission methods based on CCM and VCM (Constant and Variable Coding and Modulation, respectively) in two scenarios with different service areas. [less ▲]