Deep Reinforcement Learning for Radio Resource Management in Future AI-Driven HetNets

Abstract

Future wireless networks are expected to be extremely complex due to their massive heterogeneity in terms of the types of network architectures they integrate, types of emerging applications and services they support, and types and numbers of smart IoT devices they serve. Future wireless networks will integrate different radio access technologies (RATs) in the licensed and unlicensed bands across the land, sea, air, and deepspace into a single wireless network in order to ensure high hyper-connectivity, reliability, and coverage. Such networks are expected to support vertical disruptive applications and services in different sectors, including extended reality, holographic projection, autonomous systems, connected healthcare diagnostics, and wireless brain-machine interaction. Meanwhile, emerging user devices, such as smartphones/tablets, IoT devices, wearables, and implants, are expected to exceed 30 billion by 2023 and are increasingly being equipped with advanced signal processing capabilities that enable them to support simultaneous multi-connectivity to these multi-RATs in order to support the future applications and services. Unfortunately, radio resource allocation and management (RRAM) in such large-scale and heterogeneous wireless networks (HetNets) becomes one of the major challenges encountered during system design and deployment. Obtaining accurate and real-time information about network statistics and channel state information (CSI) becomes quite challenging or even impossible due to the rapid variations of the wireless channels and the massive heterogeneity of the network. On the other hand, it is expected that state-of-the-art approaches, such as optimization-based, game-theoretic-based, and heuristic-based methods, will encounter severe limitations when used to address RRAM problems in future HetNets due to their low scalability, poor adaptivity, high computation complexity, prohibitive information exchange, and requirements for perfect CSI. Hence, experts expect that future HetNets will be mainly AI-driven, where AI tools will be utilized to create self-organizing networks (SONs) that autonomously manage network resources and functions. In this context, deep reinforcement learning (DRL) approaches have emerged lately as promising and effective AI-based tools to address various RRAM issues in wireless networks. DRL-based RRAM methods have shown efficient results that outperform or are comparable to existing conventional methods that require full knowledge of network statistics and CSI. This research addresses practical solutions that achieve reliable and enhanced connectivity for user devices while ensuring an optimized RRAM in the future AI-driven HetNets. In particular, we look into developing efficient and low-complexity schemes based on emerging DRL techniques, such as deep Q network (DQN) and deep deterministic policy gradient (DDPG) algorithms, that jointly assign the optimum RATs and the optimum radio resources to the multi-mode- and multi-homing-based user devices while achieving the quality of services (QoSs) requirements of both network and user devices. In addition, we propose a novel multi-task DRL (MTDRL)-based scheme to optimize the radio resources of reconfigurable intelligent surface (RIS)-assisted HetNets. In particular, we study the problem of downlink network rate control in smallcells underlying macrocell multiple-input single-output (MISO)-based HetNet, where user devices can connect to both mmWave-operating smallcells and a sub-6GHz-operating RIS-assisted macrocell.

Date of Award	2022
Original language	American English
Awarding Institution	HBKU College of Science and Engineering