Current Profile Control in EAST via Reinforcement-Learning-based Model Predictive Control

Z. Wang, S. Morosohk, S.T. Paruchuri, E. Schuster

65th Division of Plasma Physics (DPP) Annual Meeting of the American Physical Society (APS)

Denver, CO, USA, October 30 – November 3, 2023

The spatial distribution of the toroidal plasma current density, simply referred to as current profile, is crucial for the realization of advanced modes of operation in tokamaks. A control-oriented model, governed by the Magnetic Diffusion Equation (MDE), is discretized and linearized to make it suitable for Model Predictive Control (MPC) design while simultaneously reducing the associated computational burden. This MPC algorithm generates optimal control strategies that satisfy the physical constraints imposed by available actuators within the Experimental Advanced Superconducting Tokamak (EAST), such as neutral beam injection and lower hybrid wave sources. Furthermore, this work introduces an innovative methodological enhancement by proposing a Reinforcement-Learning-based Model Predictive Control (RLMPC) approach. The role of reinforcement learning is to counterbalance the model uncertainties in the simplified control-oriented model employed within the MPC strategy. Swift convergence of the tunable parameters within the RLMPC is facilitated through the use of a second-order Least Square Temporal Difference Q-learning (LSTDQ) algorithm. Simulation studies based on COTSIM show that the proposed RLMPC adeptly tracks a desired profile evolution, demonstrating robust resilience in the face of disturbances.

*Supported by the US DOE under DE-SC0010537.