A soft actor-critic reinforcement learning-based method for remaining useful life prediction
提出一种基于软演员-评论家算法的强化学习方法，利用自编码器提取传感器关键特征并构建状态变量，结合误差梯度的奖励函数和渐进式早停法，在CMAPSS和XJTU-SY数据集上实现更高精度的设备剩余寿命预测。
Remaining useful life (RUL) prediction techniques play a crucial role in manufacturing equipment condition management and maintenance planning. Currently, data-driven deep learning methods have made significant advancements in this field. However, traditional approaches have not adequately considered the temporal correlations in both sensor data and RUL prediction values during the degradation process of equipment. The existing reinforcement learning (RL) methods face challenges such as lacking of sufficient lifespan variation information in the state variables, ignorance of dynamic changes in prediction error in the reward function design, and adoption of fixed interaction termination conditions that can't effectively promote the agent's learning of device degradation information. Therefore, this paper proposes a RL model based on the soft actor-critic (SAC) algorithm. Firstly, an autoencoder is employed to extract key features from the data collected by sensors. Subsequently, these key features, along with multi-dimensional lifespan features containing information from multiple historical time steps, are utilized to construct the state variables in RL. Next, a reward function is formulated taking into account error gradients. Finally, a progressive early stopping method is proposed to train the model. Extensive experiments are conducted on the CMAPSS dataset and XJTU-SY bearing dataset, and the proposed method demonstrates higher prediction accuracy compared to mainstream approaches.