Reinforcement Learning with Attention for Dynamic Spectrum Access in Optical Networks

In this paper, we propose a novel deep Q-reinforcement learning framework employing self-attention models to address dynamic resource allocation problems in optical wireless networks. The proposed framework overcomes the limitations of traditional recurrent neural networks, which struggle with sequential data execution and capturing dependencies within sequences. By leveraging a self-attention mechanism, the model enables users to learn channel access policies in an online, fully distributed manner without requiring coordination or global network information. The key innovation of this work is the integration of a single-agent DQL with a low-complexity self-attention mechanism. Unlike multi-agent systems, which often require complex coordination, the proposed single-agent framework simplifies the learning process while maintaining high performance. Numerical results demonstrate the superiority of the proposed framework. The model achieves a 153% improvement in cumulative rewards compared to state-of-the-art GRU and Long short-term memory based models. Additionally, it reduces collisions to less than 0.5% and achieves approximately twice the channel throughput of the slotted-Aloha protocol with optimal attempt probability.