A multi-objective reinforcement learning approach for resequencing scheduling problems in automotive manufacturing systems
针对汽车涂装车间颜色分批和装配车间顺序要求，提出多目标深度Q网络算法，在单机流水车间环境下优化颜色更换成本与顺序延迟两个矛盾目标，实验表明该方法优于元启发式和包络Q学习算法。
This study investigated a multi-objective resequencing scheduling problem in the automotive manufacturing systems due to operational requirements on the color-batching of the paint shop and sequential requirements on the sequence adherence of an assembly shop. Resequencing cars as color-oriented batches reduced the costs of color changes and operational costs for paint shops. Also, assembly shops required paint shops to complete cars with fewer delays so that high sequence adherence with its demand was assured. Based on real-world applications, we investigated two contradictory objectives-color change costs and sequence tardiness-in a single-machine flowshop scheduling environment. A multi-objective-deep-Q-network algorithm was developed to determine the Pareto frontier. Reward shaping was designed to improve the convergence of the neural network. The 2D-folded-normal distribution was designed to sample the preference, which made the exploration and exploitation of the neural network more comprehensive and improved the training efficiency. Two experiments were conducted and showed that the proposed approach outperformed the meta-heuristic algorithm and the envelope Q-learning algorithm in solving time, performance, the convergence of the neural network, and the diversity of the Pareto frontier. Therefore, the proposed approach can be used in automotive paint shops to improve scheduling efficiency and reduce operational costs.