A Model-Free Multi-Objective Deep Reinforcement Learning based Controller for Modular Multilevel Converters

The increasing deployment of renewable energy systems, electric vehicles, and high-voltage direct current transmission infrastructures has intensified interest in Modular Multilevel Converters (MMCs), which offer superior scalability, waveform quality, and fault tolerance. However, MMC control remains challenging due to its nonlinear dynamics and multi-objective requirements, including output current tracking, capacitor-voltage balancing, and circulating current suppression. This paper presents a model-free control strategy based on Deep Reinforcement Learning (DRL), employing the Proximal Policy Optimization algorithm to achieve these control objectives in a 3-level single-phase MMC. The proposed DRL-based controller learns an optimal switching policy directly from interaction data, eliminating the need for an accurate system model or manual tuning. Simulation results from MATLAB/Simulink confirm that the trained agent achieves low total harmonic distortion, maintains capacitor voltages around the desired values, and minimizes the circulating current, while demonstrating robustness under load variations and dynamic transients. These findings highlight the effectiveness of the proposed DRL approach as a scalable and adaptive solution for complex multilevel inverter control problems.