Continual Reinforcement Learning Formulation for Zero-Sum Game-Based Constrained Optimal Tracking
针对部分不确定的非线性离散时间系统，提出一种基于零和博弈的强化学习最优跟踪控制方法，通过障碍函数处理状态约束，并采用权重巩固机制实现持续学习，避免灾难性遗忘。
This study provides a novel reinforcement learning-based optimal tracking control of partially uncertain nonlinear discrete-time (DT) systems with state constraints using zero-sum game (ZSG) formulation. To address optimal tracking, a novel augmented system consisting of tracking error and its integral value, along with an uncertain desired trajectory, is constructed. A barrier function (BF) with a tradeoff factor is incorporated into the cost function to keep the state trajectories to remain within a compact set and to balance safety with optimality. Next, by using the modified value functional, the ZSG formulation is introduced wherein an actor-critic neural network (NN) framework is employed to approximate the value functional, optimal control input, and worst disturbance. The critic NN weights are tuned once at the sample instants and then iteratively within sampling instants. Using control input errors, the actor NN weights are adjusted once a sampling instant. The concurrent learning term in the critic weight tuning law overcomes the need for the persistency excitation (PE) condition. Further, a weight consolidation scheme is incorporated into the critic update law to attain lifelong learning by overcoming catastrophic forgetting. Finally, a numerical example supports the analytical claims.