Dynamic Actuator Allocation via Reinforcement Learning for Concurrent Plasma Control Objectives

S. T. Paruchuri, V. Graber, H. Al Khawaldeh, E. Schuster

IEEE Transactions on Plasma Science, vol. 52, no. 9, pp. 4140-4146, 2024

Abstract

Designing a single controller to simultaneously reg- ulate all kinetic and magnetic plasma properties in a tokamak can be difficult, if not impossible, due to the system’s complex coupled dynamics. A more viable solution is developing individual algorithms to control one or more plasma properties and integrating them in the plasma control system (PCS) to regulate the target scenario. However, such integration requires an actuator allocation algorithm to convert virtual commands from individual controllers into physical actuator requests (e.g., neutral beam powers) and to arbitrate the competition for available actuators by the different controllers. Existing actuator allocation algorithms rely on solving a static optimization problem at each time instant. Real-time static optimization can be computationally expensive in some instances. Furthermore, static optimization ignores the history of the actuator outputs and the temporal evolution of the actuator constraints. Therefore, dynamic actuator allocation algorithms have been proposed recently as an alternative. These algorithms use ordinary differential equations to describe the relation between virtual commands and physical actuator requests. In this work, a minimax optimization-based dynamic actuator allocation problem is formulated for a certain class of plasma control algorithms. A reinforcement learning (RL)-based algorithm is proposed to solve the optimization and hence the allocation problem. The proposed algorithm is tested using nonlinear simulations.