Pellet Injection-based Model Predictive Control of the Density Profile in Tokamaks by Leveraging Deep Reinforcement Learning

Z. Wang, S. T. Paruchuri, H. Al Khawaldeh, T. Rafiq, E. Schuster

31st IEEE Symposium on Fusion Energy (SOFE)

Cambridge, MA, USA, June 23-26, 2025

Effective regulation of the plasma-density profile can play a critical role in achieving magnetohydrodynamic stability, sustaining optimal fusion conditions, and maintaining confinement in tokamaks. Traditional methods, such as gas puffing, face substantial time delays, especially in large-scale reactors like ITER. Moreover, gas penetration into the core may be limited by pedestal conditions in high-confinement operation. As an alternative, pellet injection offers faster response times with the capability of a deeper penetration into the plasma core but presents at the same time significant challenges due to its discrete, on/off nature and complex control timing. This work introduces a novel model predictive control (MPC) framework enhanced by deep reinforcement learning to address these challenges. The proposed MPC scheme employs a linear control law capable of managing discrete control inputs and handling the nonlinear plasma dynamics by leveraging real-time learning. The plasma density profile evolution model is discretized in space by using the finite difference method (FDM), enabling the application of MPC. The dynamic pellet-injection process is modeled in discrete-time to accurately simulate fueling effects. Reinforcement learning (RL) optimizes control policies amid the inherent variability of pellet injection dynamics. Utilizing a deep Q-network (DQN) to handle the discrete control actions, the RL component adapts online to changes, improving control accuracy and convergence. Simulation studies validate the effectiveness of the proposed control strategy, demonstrating its capability to maintain the desired plasma density profile with high accuracy and reduced control effort.

*Supported by the US DOE under DE-SC0010537.