A collaborative evolutionary reinforcement learning approach to dynamic pickup and delivery challenges
提出Evo-RL框架，将深度强化学习与遗传算法协同，解决动态取送货问题中严格LIFO和时间窗约束下的奖励稀疏和局部最优难题，在ICAPS 2021基准上显著优于现有方法。
• Evo-RL synergizes Deep RL and GA for dynamic pickup and delivery problems. • RL handles strategic allocation while GA refines routes to mitigate reward sparsity. • Effectively handles strict LIFO and time window constraints in side-loading fleets. • Models stochastic delivery times and distances to guide robust policy learning. • Significantly outperforms heuristics and RL baselines on large-scale benchmarks. In the contemporary landscape of global logistics, the efficient delivery of commodities at scale has become imperative. Central to this challenge is the Dynamic Pickup and Delivery Problem (DPDP). While DPDP is inherently complex due to stochastic orders and real-time requirements, it becomes particularly formidable in specific supply chain variants constrained by strict loading protocols, such as the Last-In-First-Out (LIFO) requirement found in air cargo and side-loading fleets. Traditional heuristics often struggle to escape local optima under such rigid constraints, while standard Reinforcement Learning (RL) faces difficulties with reward sparsity and high-dimensional action spaces. To address these challenges, we propose Evo-RL, a holistic evolutionary reinforcement learning framework. Unlike disjointed hybrid methods, Evo-RL establishes a synergistic closed loop: the RL agent acts as a strategic planner to capture latent spatiotemporal dependencies and generate feasible initial assignments, while a population-based Genetic Algorithm (GA) serves as an optimization engine to refine sequences and provide dense feedback signals. This collaborative mechanism allows the RL agent to overcome reward sparsity by learning from the diverse evolutionary population. We evaluate our approach on the real-world, large-scale DPDP benchmark from the ICAPS 2021 competition. Empirical results demonstrate that Evo-RL significantly outperforms state-of-the-art heuristics and deep RL baselines in terms of solution quality and convergence speed, proving its efficacy in handling dynamic constraints and large-scale complexities.