Supplier Selection and Material Sourcing With Multiuncertainties in Cloud Manufacturing Using Reinforcement Learning
针对云制造中物流多不确定性（交货时间随机、中断和损失），提出一种结合近端策略优化、循环神经网络和期望模型的随机优化方法，以最小化总成本并智能决策自制或外购。
Compared to traditional manufacturing, there are several unique characteristics in cloud manufacturing (CMfg): more candidate suppliers, more material types, more supply modes, and broader geographically distributed suppliers. These characteristics lead to a huge set of candidate supply plans with several uncertainties in logistics. It is a great challenge to effectively and efficiently select appropriate suppliers for each type of material. In this article, we consider a SSMS problem with multiuncertainties in logistics to minimize the total cost of a CMfg manufacturing enterprise. The delivery time is stochastic along with the consideration of stochastic disruption and loss in logistics. Based on state-and-transition modeling, a stochastic dynamic programming model is developed for the problem under study. By integrating proximal policy optimization (PPO) with recurrent neural network (RNN) and expectation model (EM), a stochastic optimization method PPO-REM is proposed to minimize the cost by effectively selecting suppliers and intelligently making make-or-buy decisions under uncertainties. The proposed method is evaluated by comparing to other reinforcement learning (RL) methods and existing methods for similar problems over a comprehensive set of numerical experiments with some real-world data. Experimental results show that the proposed PPO-REM converges faster with a higher reward than other RL methods, and it costs the least as compared to existing methods for similar problems.