Safe Reinforcement Learning: Optimal Formation Control With Collision Avoidance of Multiple Satellite Systems
针对多卫星系统的避碰与编队控制问题，提出一种基于自适应动态规划的安全强化学习算法，通过引入障碍函数和距离变权重方法，实现最优编队控制策略的自适应学习。
This article addresses the collision avoidance and formation control problem for multisatellite systems. A novel safe reinforcement learning (RL) algorithm based on an adaptive dynamic programming framework is proposed. The highlights of the algorithm are the adaptive distance-varying learning method to integrate online data with historical data and the usage of the barrier function (BF) to achieve collision avoidance. First, the BF is introduced into the designed cost function such that the multisatellite formation system can achieve obstacle avoidance and guarantee the safety. Next, a safe RL algorithm is developed through the critic network structure. A distance-varying weight is introduced, which combines experience replay samples with extrapolation samples. By minimizing the cost function, the optimal formation control policy can be obtained with an adaptive formation and self-learning ability. Then, the stability and safety of the proposed algorithm are analyzed. Finally, the effectiveness and superiority of the proposed algorithm are verified by numerical simulations.