Robust Distributed UAV Location and Beamforming Optimization via Multi-Agent RL and Convex Design in Multi-UAV Networks

This paper addresses the joint problem of distributed UAV placement and multi-antenna beamforming for downlink Rate-Splitting Multiple Access (RSMA) in multi-UAV networks with uncertain user locations. We propose a practical two-stage hybrid solution that (i) learns robust, decentralized UAV placements via a QMIX-based multi-agent deep Q-learning framework that encodes the environment as fixed-resolution multi-channel spatial maps and employs CoordConv to preserve absolute spatial information, and (ii) optimizes per-UAV multi-antenna precoding via a tractable semidefinite-programming (SDP) relaxation augmented with successive convex approximation and difference-of-convex programming to handle nonconvex log-terms and rank-1 constraints. The learning stage provides centralized training with fully decentralized execution and yields a single trained agent that generalizes across varying user counts and cluster configurations without retraining. The convex precoding stage enforces user rate, power and RSMA SIC constraints and is solved efficiently using an interior-point SDP solver. Extensive simulations (varying user counts, UAV counts, grid resolutions and cluster patterns) demonstrate fast convergence of the proposed QMIX placement (convergence in 4×10⁴ training rounds in our setup), substantial sum rate gains over two competitive benchmarks, and favorable scalability and computational profiles.