Lyapunov-Based Reward Function Design for Reinforcement Learning Control of Grid-Connected Multilevel Inverters: A PUC5 Case Study

This paper proposes a Lyapunov-based reward function design for Reinforcement Learning Control (RL-C) of a grid-connected 5-level Packed U Cell (PUC5) inverter. Unlike existing RL approaches that rely solely on tracking-error-based rewards, the proposed method integrates the Lyapunov stability condition directly into the reward formulation, ensuring stable learning and robust closed-loop performance. The control objectives are to regulate the flying capacitor voltage and inject a low total harmonic distortion (THD) grid current. The Proximal Policy Optimization (PPO) algorithm is employed and trained in a MATLAB/Simulink environment under randomized operating conditions to enhance generalization. Simulation results demonstrate that the Lyapunov-based RL-C achieves stable operation with THD as low as 1.9 % and capacitor voltage error below 1 V across different current levels. Moreover, the trained agent exhibits strong adaptability to parameter variations and untrained operating points, confirming the proposed framework's robustness and suitability for real-time power electronics applications.