Binary kernel logistic regression: A sparsity-inducing formulation and a convergent decomposition training algorithm
针对核逻辑回归模型不稀疏的问题，提出一种能诱导稀疏性的训练公式，并设计收敛的分解算法，在保持精度的同时提升稀疏性，对分类任务有实用价值。
Kernel logistic regression (KLR) is a widely used supervised learning method for binary and multi-class classification, which provides estimates of the conditional probabilities of class membership for the data points. Unlike other kernel methods such as Support Vector Machines (SVMs), KLRs are generally not sparse. Previous attempts to deal with sparsity in KLR include a heuristic method referred to as the Import Vector Machine (IVM) and ad hoc regularizations such as the ℓ 1 / 2 -based one. Achieving a good trade-off between prediction accuracy and sparsity is still a challenging issue with a potential significant impact from the application point of view. In this work, we revisit binary KLR and propose an extension of the training formulation proposed by Keerthi et al., which is able to induce sparsity in the trained model, while maintaining good testing accuracy. To efficiently solve the dual of this formulation, we devise a decomposition algorithm of Sequential Minimal Optimization type which exploits second-order information, and for which we establish global convergence. Numerical experiments conducted on 12 datasets from the literature show that the proposed binary KLR approach achieves a competitive trade-off between accuracy and sparsity with respect to IVM, ℓ 1 / 2 -based regularization for KLR, and SVM while retaining the advantages of providing informative estimates of the class membership probabilities.