MARL Sim2real Transfer: Merging Physical Reality With Digital Virtuality in Metaverse
本文用多智能体深度强化学习解决元宇宙中数字与物理实体间的仿真到现实迁移问题，提出循环MARL方法并成功应用于多无人机系统，对元宇宙和机器人领域研究者有参考价值。
Metaverse is an artificial virtual world mapped from and interacting with the real world. In metaverse, digital entities coexist with their physical counterparts. Powered by deep learning, metaverse is inevitably becoming more intelligent in the interactions between reality and virtuality. However, it is confronted with a nontrivial problem known as sim2real transfer when deep learning techniques try to bridge the reality gap between the physical world and simulations. In this article, we use multiagent deep reinforcement learning (MARL) to implement collective intelligence for digital entities as well as their physical counterparts. To model the immersive environments in metaverse, we define a nonstationary variant of Markov games and propose a recurrent MARL solution to it. Based on the solution, MARL sim2real transfer that bridges real and virtual multiple unmanned aerial vehicle (multi-UAV) systems is successfully conducted by employing recurrent multiagent deep deterministic policy gradient (R-MADDPG) with the domain randomization technique. Additionally, we use perception-control modularization to improve the generalization performance of MARL policies and make training more efficient.