Full automation or just a copilot? A study of large language models on spare parts demand forecasting with OpenAI o1-mini
研究了大型语言模型在备件需求预测中的能力，发现其响应高度可变，但通过多次平均和详细提示可提升准确性，决策树方法能进一步降低预测误差。
Large language models (LLMs) promise process automation but entail reliability concerns. We investigate the capability of LLMs in the context of operations management, specifically demand forecasting of spare parts. For this, we develop a self-reflective simulation framework with an LLM producing automatic demand forecasts, using simulated and real-world data. This includes generating and debugging a Python script, verifying the results, and iteratively improving the outcomes. Our results demonstrate a high variability of responses, especially for stock-keeping units with irregular demand. Averaging over several repetitions can improve performance and result in forecasts that are comparable in accuracy to common benchmarks. Providing more detailed instructions in the prompt can improve forecasting accuracy. Training a decision tree to determine which method to choose for which kind of stock keeping unit results in lower forecasting errors compared to always using the same method or deciding on the method based on classifications from previous research.