ReSQ: Reinforcement Learning-Based Queue Allocation in Software-Defined Queuing Framework

in Journal of Networking and Network Applications (2022), 2(4), 143152

With the evolution of 5G networks, the demand for Ultra-Reliable Low Latency Communications (URLLC) services is increasing. Software-Defined Networking (SDN) offers flexible network management to ... [more ▼]

With the evolution of 5G networks, the demand for Ultra-Reliable Low Latency Communications (URLLC) services is increasing. Software-Defined Networking (SDN) offers flexible network management to prioritize URLLC services coexisting with best-effort traffic. Utilizing the network programmability feature of SDN, Software-Defined Queueing (SDQ) framework selects the optimal output port queue on forwarding devices and routing path for incoming traffic flows to provide deterministic Quality of Service (QoS) support required for URLLC traffic. However, in the existing SDQ framework, the selections of optimal queue and path are done manually by observing the traffic type of each incoming flow, the available bandwidth of each potential routing path, and the status of output port queues of each forwarding device on each potential routing path. The static allocations of path and queue for each flow are inefficient to provide a deterministic QoS guarantee for a high volume of incoming traffic which is typical in 5G networks. The limited buffer availability on the forwarding devices is another constraint regarding optimal queue allocation that ensures an end-to-end (E2E) delay guarantee. To address these challenges, in this paper, we extend the SDQ framework by automating queue management with a reinforcement learning (RL)-based approach. The proposed queue management approach considers diverse QoS demands as well as a limited buffer on the forwarding devices and performs prioritized queue allocation. Our approach also includes a hash-based flow grouping to handle a high volume of traffic having diverse latency demands and a path selection mechanism based on available bandwidth and hop count. The simulation result shows that the proposed scheme ReSQ reduces the QoS violation ratio by 10.45% as compared to the baseline scheme that selects queues randomly. [less ▲]

D-ViNE: Dynamic Virtual Network Embedding in Non-Terrestrial Networks

in IEEE Wireless Communications and Networking Conference (WCNC) (2022, April 10)

In this paper, we address the virtual network embedding (VNE) problem in non-terrestrial networks (NTNs) enabling dynamic changes in the virtual network function (VNF) deployment to maximize the service ... [more ▼]

In this paper, we address the virtual network embedding (VNE) problem in non-terrestrial networks (NTNs) enabling dynamic changes in the virtual network function (VNF) deployment to maximize the service acceptance rate and service revenue. NTNs such as satellite networks involve highly dynamic topology and limited resources in terms of rate and power. VNE in NTNs is a challenge because a static strategy under-performs when new service requests arrive or the network topology changes unexpectedly due to failures or other events. Existing solutions do not consider the power constraint of satellites and rate limitation of inter-satellite links (ISLs) which are essential parameters for dynamic adjustment of existing VNE strategy in NTNs. In this work, we propose a dynamic VNE algorithm that selects a suitable VNE strategy for new and existing services considering the time-varying network topology. The proposed scheme, D-ViNE, increases the service acceptance ratio by 8.51% compared to the benchmark scheme TS-MAPSCH. [less ▲]