A Robust Reinforcement Learning Control Method for Uncertain Process Industry Based on Knowledge-Constrained Adversarial Perturbation
针对过程工业中观测状态受不确定扰动影响的问题，提出一种基于知识约束对抗扰动的鲁棒强化学习控制方法，通过构建反应气氛代理模型和动态扰动集，提升控制性能，案例验证于锌电积过程。
The process industry is a continuous manufacturing system that comprises intricate physical and chemical reactions. Given the increasing constraints on resources and energy, it is urgent to optimize process indicators by maintaining an efficient reaction atmosphere. Reinforcement learning (RL), using trial and error to learn control strategies, has become a topic of interest in the control community. However, practical implementation reveals that the mapping between observed state variables and the reaction atmosphere is subject to uncertain disturbances, which seriously affect the reliability of process indicator control. To address these issues, a robust RL (RRL) control method based on knowledge-constrained adversarial perturbation is proposed. It applies the adversary to perturb the observed state to characterize the uncertain disturbance. First, the insight of composite modeling for the process industry is presented to factorize the inherent and external uncertainties. Based on this insight, a reaction atmosphere indicator surrogate model is built to quantify the inherent uncertainty. Second, by leveraging the variation boundary information of the surrogate model, a dynamic state perturbation set and its update policy are proposed to ensure the rationality of the state perturbation. Last, an external uncertain time series generation method with continuity constraints is proposed to incorporate reasonable external uncertainty in the training process. Case validation in zinc electrowinning demonstrates that the proposed method effectively enhances control performance in uncertain scenarios.