DRL-based Federated Uncertainty-guided Semi-Supervised Learning for Network Traffic Selection and Threshold Determination in ZSM

The ever-expanding landscape of advanced applications and services, as well as the associated emerging attacks in the zero-touch network and service management (ZSM) paradigm, necessitates novel approaches to manage complex network infrastructures while addressing the security requirements of beyond 5G networks. To address this issue, we present a cutting-edge, novel semi-supervised federated learning approach that incorporates a Deep Reinforcement Learning (DRL) agent for real-time defense system updates. Specifically, we propose the DRL-FedUSS framework, which stands for DRL-based Federated Uncertainty-guided Semi-Supervised learning. DRL-FedUSS is designed explicitly for Label-at-Client scenarios to accelerate the training convergence when clients hold a scarcity of labeled and an abundance of unlabeled network traffic samples. The DRL-FedUSS framework integrates a DRL agent that intelligently selects the most informative samples with a real-time adaptive threshold for data annotation, considering uncertainty, time, budget constraints, and, most importantly, the convergence rate and confidence level constraints. Our extensive simulations on realistic non-independent and identically distributed (non-IID) datasets prove that the DRL-FedUSS framework outperforms baseline approaches, achieving superior intrusion detection accuracy, reducing the associated cost, and accelerating the convergence rate with minimal network traffic labeled data.