Distributed Optimal Consensus Problem of Input Constrained Nonlinear Discrete-Time MASs: A Mode-Free Reinforcement Learning Approach
提出一种无模型强化学习方法，解决输入受限的非线性离散时间多智能体系统的最优一致控制问题，通过演员-评论家框架和渐进过渡控制方法处理耦合方程和物理约束，并设计同步阻塞方法实现分布式同步。
In this article, a model-free reinforcement learning (RL) approach is proposed for solving the optimal consensus control issue of nonlinear discrete-time multiagent systems with input constraint. To address the challenge of solving the coupled discrete Hamilton-Jacobi-Bellman (HJB) equation, a RL approach based on actor-critic framework is proposed for optimal consensus control. A well-defined cost function is designed, and the actor and critic networks are updated through online learning to obtain the optimal controllers. Furthermore, the actuator's performance is often limited due to physical constraints. To address such actuator constraints, a gradual transition control (GTC) method is proposed, and update-free and update-weak policies are introduced to further optimize network performance. Additionally, in real-world distributed systems, the actor-critic networks deployed in each agent rely on data from neighboring agents, which necessitates addressing the issue of distributed synchronization. To address this challenge, the synchronization blocking method is designed, which designs additional control signals for each agent to handle these issues. Finally, two simulations under different scenarios are presented to verify the effectiveness of the proposed approach.