Advanced Proximal Policy Optimization Strategy for Resilient Cyber-Physical Power Grid Stability Against Hostile Electrical Disruptions

The efficient and secure operation of power grids is essential for ensuring reliable electricity supply and supporting the integration of renewable energy sources. Yet, the landscape is marred by burgeoning adversarial attacks, particularly targeting power systems employing cutting-edge deep reinforcement learning (DRL) methodologies. This study proposes a proximal policy optimization (PPO) agent against a randomized adversarial opponent aiming to disrupt grid operations. The performance of the PPO agent is assessed across various power grid environments alongside several baseline agents, including the do-nothing agent, the random agent, the topology greedy agent, and the power line switch agent with adversarial training. Over multiple epochs of adversarial training, the average rewards, number of steps to resolution, and computational time are recorded. The simulation results on the IEEE 14-bus system and the reduced IEEE 118-bus system demonstrate a nuanced supremacy and applicability of the PPO algorithm compared to heuristic and randomized approaches. The main contributions of this paper include 1) Introducing an optimized PPO algorithm assessed using two IEEE bus system environments; and 2) Applying an adversarial-training-based DRL to improve the robustness of PPO alorthim’s policies in the electrical grid environment.