Advanced Process Control and Real-Time Decision-Making via Reinforcement Learning

Reinforcement Learning (RL) has significantly expanded the field of process control and real-time decision-making in the past decade, enabling the optimal operation of highly complex dynamic systems via the use of neural networks. RL has been exceptionally useful in situations where it is challenging to create a mechanistic model or develop a high-accuracy data-driven surrogate representation. RL has proven to be transformative in the areas of robotics, game-playing, and hierarchical decision-making, but has been challenging to implement for the direct control of chemical process systems. In this talk, we will first provide an overview on the basics of neural networks and state-of-the-art RL algorithms. Then we will focus on two key challenges in RL-based chemical process control: (i) how to improve the learning efficiency which typically requires a very large amount of training data until making reliable decisions? and (ii) how to ensure the safety and stability of the learned control policy? To address these challenges, we will introduce a novel transfer-learned RL algorithm that leverages Y-wise Affine Neural Networks. This specialized neural network architecture can exactly represent the explicit control policy from multi-parametric model predictive control (mp-MPC). The Y-wise Affine Neural Networks can thus serve as a hot start and transfer the mp-MPC knowledge to RL training. This contributes to fully eliminate the unsafe and time-consuming exploration stage during RL training, while providing control actions with confidence. We will also discuss the implementation of continuous policy improvement to heuristically guarantee that the mp-MPC solution serves as an effective lower bound to the transfer-learned RL. The computational and practical advantages of this algorithm will be demonstrated on the control and real-time decision-making of multiple representative chemical process systems.