Dynamic Leader–Follower Output Containment Control of Heterogeneous Multiagent Systems Using Reinforcement Learning
针对动态领导者异构多智能体系统，提出一种无模型强化学习算法，使跟随者在不了解系统动态的情况下，通过在线学习实现输出轨迹保持在领导者形成的凸包内。
This article addresses the optimal containment problem of heterogeneous multiagent systems (MASs) with dynamic leaders via reinforcement learning (RL), where the dynamics of all agents are all completely unknown. A distributed model-free observer is constructed for each follower to estimate the leaders’ dynamics and the output trajectories inside the convex hull formed by the leaders. Based on the designed observers, the optimal containment problem is formulated as an optimal tracking control issue. Then the discounted performance functions are introduced to obtain algebraic Riccati equations (AREs). And a model-free RL algorithm is developed to learn the AREs online. To implement this algorithm, we design a single critic neural network structure for each follower to approximate Q-function, and estimate optimal control policy and worst-case adversarial input policy. Finally, a numerical simulation is provided to demonstrate the effectiveness of the proposed algorithm.