A hierarchical reinforcement learning approach for real-time berth allocation and quay crane scheduling
提出分层强化学习框架，通过三个协作智能体实时处理泊位分配和岸桥调度，在考虑碳排放和不确定性下接近最优解，为港口管理者提供决策支持。
Efficient and low-carbon berth allocation and quay crane scheduling are crucial for enhancing the competitiveness and sustainability of container ports. This study addresses the berth allocation and quay crane assignment and scheduling problem (BACASP) by incorporating carbon emission costs and uncertainties in vessel arrival times and quay crane processing times. A novel hierarchical reinforcement learning (HRL)-based scheduling framework is proposed, employing three cooperative agents to support real-time berth allocation and quay crane scheduling in dynamic environments. The upper-level agent determines whether to release waiting vessels, while two lower-level agents allocate berth locations and assign quay cranes. Numerical experiments demonstrate the effectiveness of the HRL framework compared to a mixed integer programming (MIP) approach with perfect information, highlighting its capability to achieve near-optimal solutions under sequentially observed information. The study also investigates the impact of uncertainty on operational and carbon emission costs, providing practical managerial insights for port operators. These findings underscore the potential of leveraging well-structured HRL frameworks to address complex and dynamic port operation problems.