Four-dimensional spatial-temporal packet encapsulation in the Cyber-Physical Internet: A reinforcement learning approach
针对信息物理互联网中箱子在三维几何约束和近实时时间要求下封装进集装箱的问题，提出一种深度强化学习算法，同时处理箱子分配和装箱两个目标，并通过GPU加速满足实时性，实验证明优于传统启发式方法。
• Reveal gaps in Physical Internet through a systematic literature review. • The insights for cyber capability enable the emerging Cyber-Physical Internet. • The CPI five-layer model plays a role as the OSI model for Internet in logistics. • Future research directions for Cyber-Physical Internet are identified and defined. This paper investigates packet encapsulation in the Cyber-Physical Internet (CPI), where boxes must be encapsulated into containers under joint three-dimensional (3D) geometric constraints and near real-time temporal requirements, governed by physical handling and transshipment tolerances. Even though the Heterogeneous Container Loading Problem (HCLP) is prevalent in logistics, it often overlooks the real-time encapsulation needs of CPI. Thus, this paper proposes a novel approach for four-dimensional (4D) spatial–temporal encapsulation in CPI. We focus on two sequential objectives: box allocation (assigning standardized boxes to appropriate containers) and box packing (efficiently loading boxes into allocated containers). To address these objectives, the 4D spatial–temporal encapsulation process in CPI is abstracted as a near real-time, large-scale HCLP (TL-HCLP), where stringent near real‑time operational constraints and massive scale requirements render traditional exact algorithms and heuristic methods insufficient. Therefore, we develop a deep reinforcement learning (DRL) algorithm. While most existing DRL algorithms are designed for single-container loading scenarios, addressing only the box packing objective, our method introduces an original reward mechanism that simultaneously handles both box allocation and packing tasks. During the implementation, we encountered computational bottlenecks in meeting the near real-time requirements. To overcome this, we further optimized the solution by incorporating GPU acceleration, significantly reducing the computation time. To validate the efficiency of the proposed algorithm, multiple computational experiments were conducted. The results demonstrate that the DRL algorithm outperforms traditional heuristics, confirming its effectiveness and applicability in real-world CPI scenarios.