Probabilistic iterative hard thresholding for sparse learning
针对高维小样本数据，提出一种概率迭代硬阈值算法，解决带基数约束的期望目标优化问题，证明随机过程收敛性，并在两个机器学习问题上验证效果。
Abstract For statistical modeling wherein the data regime is unfavorable in terms of dimensionality relative to the sample size, finding hidden sparsity in the relationship structure between variables can be critical in formulating an accurate statistical model. The so-called “ $$\ell _0$$ norm”, which counts the number of non-zero components in a vector, is a strong reliable mechanism of enforcing sparsity when incorporated into an optimization problem for minimizing the fit of a given model to a set of observations. However, in big data settings wherein noisy estimates of the gradient must be evaluated out of computational necessity, the literature is scant on methods that reliably converge. In this paper, we present an approach towards solving expectation objective optimization problems with cardinality constraints. We prove convergence of the underlying stochastic process and demonstrate the performance on two Machine Learning problems.