Quantifying uncertainty in deep learning binary classification with discrete noise in inputs for risk-based decision making
提出一种基于变异比率的框架，量化深度神经网络二分类模型在输入含离散噪声时的预测不确定性，帮助识别风险敏感案例，适用于医疗、金融等领域。
The use of Deep Neural Network (DNN) models in risk-based decision-making has garnered significant attention across fields such as medicine, finance, manufacturing, and quality control. To mitigate risks, it is crucial to assess the confidence or uncertainty of the prediction alongside the overall performance of the algorithm. Transitioning from deterministic and probabilistic DNNs to Bayesian deep learning models enables the quantification of prediction uncertainty by leveraging second-order predictive distributions. Shannon entropy and mutual information serve as key metrics for assessing and decomposing this uncertainty into epistemic and aleatoric components, which are crucial for developing effective uncertainty reduction strategies. However, entropy-based measures can be inconsistent, and mutual information may constrain the assessment of multiple sources of epistemic uncertainty. Additionally, assuming errors follow normal distributions, while simple, is inadequate for modeling discrete errors. We propose a framework for quantifying the prediction uncertainty in DNN binary classification models using the variation-ratio measure. This approach facilitates risk-based decision-making in presence of discrete errors in predictors. Our model accounts for two sources of epistemic uncertainty: model parameters and predictor errors with a known finite discrete distribution. The variation-ratio measure effectively identifies significant contributions of predictor errors to total uncertainty. Applied to a case study on tuberculosis treatment outcome prediction and 10 simulated datasets, our framework detects risk-sensitive cases where predictor errors may alter predictive uncertainty. Even when epistemic uncertainty is attributed solely to model parameters, our approach exhibits superior uncertainty awareness compared to the Monte Carlo dropout method.