Local 2-D Path Planning of Unmanned Underwater Vehicles in Continuous Action Space Based on the Twin-Delayed Deep Deterministic Policy Gradient
针对无人水下航行器在连续动作空间中的局部二维路径规划问题，提出一种改进的TD3算法，通过添加均值函数和动作存储机制减少边界动作输出，并引入实时声纳变量使模型更符合实际水下导航情况。
In this article, the local two-dimensional (2-D) path planning problem is studied for an unmanned underwater vehicle (UUV) under continuous action space, and an improved algorithm is proposed based on the twin-delayed deep deterministic policy gradient (TD3). The mean function is added to the policy gradient to bring the output action of the algorithm closer to the mean of the action space. Hence, it suppresses the trend of a large number of boundary actions output by the TD3 algorithm. Based on the experience replay buffer, action storage is constructed to realize the automatic adjustment of the weight coefficient. Therefore, it reduces the additional hyperparameter tuning work caused by the change in the structure of the algorithm. In the setting of environmental variables and reward functions, real-time sonar variables are added to make the algorithm model more consistent with the actual underwater navigation situation. Based on ROS, a simulation environment is built and used to verify the path planning performance of the proposed algorithm.