Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low
研究了最佳子集选择在低信噪比环境下的过拟合问题，提出带正则化的子集选择方法，结合整数规划和一阶方法提升预测性能，适用于高维噪声数据建模。
Learning Compact High-Dimensional Models in Noisy Environments Building compact, interpretable statistical models where the output depends upon a small number of input features is a well-known problem in modern analytics applications. A fundamental tool used in this context is the prominent best subset selection (BSS) procedure, which seeks to obtain the best linear fit to data subject to a constraint on the number of nonzero features. Whereas the BSS procedure works exceptionally well in some regimes, it performs pretty poorly in out-of-sample predictive performance when the underlying data are noisy, which is quite common in practice. In this paper, we explore this relatively less-understood overfitting behavior of BSS in low-signal noisy environments and propose alternatives that appear to mitigate such shortcomings. We study the theoretical statistical properties of our proposed regularized BSS procedure and show promising computational results on various data sets, using tools from integer programming and first-order methods.