Multistep Q -Learning-Based Optimal Consensus Control of Linear Discrete-Time Multiagent Systems
针对多智能体系统的最优一致性问题，提出多步Q学习方法，通过将一致性转化为求解最优Q函数，并设计Actor-Critic网络实现，仿真验证了其相比单步Q学习的优越性。
This article considers the optimal consensus control for the multiagent systems problem. By developing the multiagent multistep Q-learning (MaMsQL), the methodology achieves enhanced efficiency while addressing the issue of the complex interaction dynamics between agents, environmental uncertainty, thus ultimately meeting demand of balancing exploration and exploitation. First, associated with the performance index, the Q-function is established to prove that all optimal Q-functions form a Nash equilibrium outcome, thereby the consensus problem is converted to finding the optimal Q-functions. Then, the MaMsQL method is developed with theoretical proof of its convergence. Finally, the method is implemented through a specially designed Actor-Critic network. By virtue of the comparison with multiagent single step Q-learning, the effectiveness and superiority of this method are verified through simulation examples.