Reinforcement Learning-Based Controller with Reduced Switching Frequency for Grid-Tied 9-Level Crossover Switches Cell Multilevel Inverter

This paper presents a novel reinforcement learning-based controller (RL-C) for a single-phase, grid-connected, 9-level crossover switches cell (CSC9) multilevel inverter (MLI). The proposed RL-C is designed to achieve three control objectives: regulating the capacitor voltage around its reference value, delivering a pure sinusoidal current to the grid with the capability to provide both active and reactive power, and reducing the average switching frequency. An actor-critic architecture is employed in the design of the RL-C, utilizing the on-policy Proximal Policy Optimization (PPO) learning algorithm for its stability and efficiency. The RL-C in this paper is developed, trained, and tested in the MATLAB/Simulink environment. The trained agent demonstrated its effectiveness in fulfilling the control objectives under various dynamic operating conditions and CSC9 capacitor and filtering inductor configurations, without the need for retraining. Furthermore, a comparative analysis is conducted against finite-control set model predictive control (FCS-MPC) and a variant of RL-C without the switching frequency reduction objective. Compared with FCS-MPC, the proposed RL-C yields lower total harmonic distortion in the current and lower capacitor voltage error for a compact CSC9 design over a range of 4 A to 15 A operating conditions.