Dynamic Actuator Allocation via Reinforcement Learning for Concurrent Regulation of Plasma Properties

S.T. Paruchuri, E. Schuster

30th IEEE Symposium on Fusion Energy

Oxford, UK, July 9-13, 2023

Designing a single controller to simultaneously regulate all kinetic and magnetic plasma properties in a tokamak can be difficult, if not impossible, due to the system’s complex coupled dynamics. A more viable solution is developing individual algorithms to control one or more plasma properties and integrating them in the plasma control system (PCS) to regulate the target scenario. However, such integration requires an actuator allocation algorithm to convert virtual inputs from individual controllers into physical actuator commands (e.g., neutral beam powers) and to arbitrate the competition for available actuators by the different controllers. Existing actuator allocation algorithms rely on solving a static optimization problem at each time instant. Real-time static optimization can be computationally expensive in some instances. Furthermore, static optimization ignores the temporal evolution of actuator variables and constraints. Therefore, dynamic actuator allocation algorithms have been proposed recently as an alternative. These algorithms use ordinary differential equations to describe the relation between virtual inputs and physical actuators. A dynamic actuator allocation problem is formulated for a certain class of plasma control algorithms. A reinforcement learning (RL)-based algorithm is proposed to solve the dynamic allocation problem in real time. The RL-based algorithm relies on data collected in real-time to learn the allocation policy that prescribes the optimal combination of physical actuator commands for a given set of virtual inputs. The proposed algorithm is tailored to the DIII-D tokamak and tested in nonlinear simulations based on the Control Oriented Transport SIMulator (COTSIM).

*Supported by the US DOE (DE-SC0010661, DE-SC0021385).