高维通用学习过程的拟合优度评估

A Goodness-of-Fit Assessment for General Learning Procedures in High Dimensions

Journal of the American Statistical Association · 2025
被引 0
ABS 4

中文导读

提出一种适用于高维预测变量的拟合优度检验方法,可评估从线性回归到神经网络等不同学习过程是否达到最优性能,通过数据拆分和残差累积协方差实现,模拟和真实数据验证有效。

Abstract

Black-box learners have demonstrated remarkable success across various fields due to their high predictive accuracy. However, the complexity of their learning procedures poses significant challenges in evaluating whether a given learner has achieved optimal performance on datasets with unknown data-generating mechanisms. We propose a general goodness-of-fit test for assessing different learning procedures involving high-dimensional predictors, encompassing methods from classical linear regression to advanced neural networks. Our goodness-of-fit test leverages data-splitting, utilizing the test set to evaluate the black-box learner trained on the training set. By examining the cumulative covariance of the residuals, our method can effectively handle high-dimensional predictors. Extensive simulations and three real data analyses validate the effectiveness of our method.

统计学计量经济学机器学习高维数据分析