Advanced autonomous collision avoidance for maritime navigation: A reinforcement learning approach with ship dynamics and environmental awareness
提出一种基于强化学习的自主避碰框架，通过离散动作空间、MMG模型和多维奖励函数，实现符合COLREGs的动态可行轨迹，适用于多船和非合作环境。
Autonomous collision avoidance is critical for ensuring the safety and efficiency of maritime navigation. However, existing approaches often struggle to achieve realistic manoeuvrability, robust generalisation, and compliance with the Convention on the International Regulations for Preventing Collisions at Sea (COLREGs). To address these challenges, this study proposes a Reinforcement Learning (RL)-based collision avoidance framework, integrating three key innovations. Firstly, a discrete action space is designed to accurately capture the rudder control characteristics commonly used in real maritime operations. This is integrated with a Manoeuvring Modelling Group (MMG) model, ensuring that the generated trajectories are dynamically feasible and operationally realistic. Secondly, a multi-dimensional reward function is developed, incorporating collision risk, distance to target, navigational efficiency, operational comfort, and compliance with COLREGs. This is further supported by a line-of-sight (LOS) tracking mechanism, which stabilises heading corrections based on dynamic path requirements, significantly improving the agent’s course-keeping ability. Finally, the framework includes a robust generalisation strategy, using polygonal obstacle modelling to represent complex, irregular hazards more accurately. This is combined with real-world bathymetric data and multi-ship encounters for rigorous validation, ensuring the system can operate effectively in uncertain, multi-agent, and non-cooperative environments. The proposed model is trained using the Phasic Policy Gradient (PPG) algorithm within an Actor-Critic (AC) architecture, enabling robust policy learning under uncertainty. Simulation results demonstrate that the framework effectively reduces collision risk, maintains stable trajectories, and adheres to COLREGs, making it a practical and scalable solution for next-generation autonomous ship navigation.