Optimal Stochastic Containment Control of Discrete-Time Multiagent Systems With Process Disturbances
研究了有向图下受过程扰动的离散时间多智能体系统的最优随机包含控制问题，通过模型变换将问题转化为包含误差系统的随机最优控制，并设计了基于策略迭代和Q学习的模型无关算法，使系统以最小能量输入实现均方有界包含。
This article explores the optimal containment control of discrete-time multiagent systems (MASs) with the digraph and unknown dynamics under process disturbances. We first demonstrate, through a model transformation, that the mean square bounded containment of MASs can be achieved by guaranteeing the mean square boundedness of the containment error systems. Hence, we can transform the optimal stochastic containment control problem of MASs into a stochastic optimal control problem for containment error systems. Subsequently, utilizing the Bellman optimality principle and the stochastic Lyapunov equation (SLE), we design a model-based policy iteration (PI) algorithm for the optimal stochastic containment control of MASs. This model-based algorithm, by minimizing the cost function in linear quadratic form, enables MASs to achieve mean square bounded containment with the least possible energy input. To circumvent the dependency on the model information, we introduce an online model-free algorithm for the stochastic optimal control problem. The model-free algorithm is developed based on the Q-learning algorithm. Specifically, it uses a historical MAS trajectory to estimate the kernel matrix <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H</i> of the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i> function, enabling the resolution of the optimal stochastic containment control problem without model information. To realize the model-free algorithm, the LSTD estimator with bounded bias is employed in the policy evaluation step. We prove the equivalence between the model-free algorithm and the model-based algorithm. Finally, a numerical case is presented to demonstrate the efficacy of the proposed algorithms in achieving the optimal stochastic containment control of MASs.