Reinforcement Learning for Sustainable Closed-Loop Reservoir Management: A Focused Review

Closed-loop reservoir management (CLRM) requires sequential decision making under geological and operational uncertainty, traditionally relying on repeated fullphysics simulation and static optimization. While effective, such workflows become computationally expensive when models and control strategies must be updated frequently as new data become available. Reinforcement learning (RL) offers a decisioncentric alternative by directly learning adaptive control policies that optimize long-term objectives through interaction with a simulated environment. This paper presents a focused review of reinforcement learning as the decision-making engine for sustainable closed-loop reservoir management. Rather than providing an exhaustive survey of artificial intelligence techniques, the review synthesizes key design choices that govern the effectiveness, robustness, and scalability of RL-based CLRM workflows. Specifically, the paper reviews RL problem formulation for reservoir management, including state representation, action parameterization under operational constraints, and reward design aligned with long-term economic objectives. The role of surrogate and proxy models in enabling computationally efficient RL training is also examined. Sustainability considerations are highlighted throughout the review, including computational efficiency, sample-efficient learning, robustness under geological uncertainty, and long-term operational stability. In addition, the paper summarizes commonly reported computational sustainability indicators (e.g., fullphysics simulation usage, proxy-to-simulator interaction ratios, and GPU-hour/ CO₂ e reporting) to support reproducible benchmarking across RL-based CLRM workflows.