Embedded Feature Selection Based on Relevance Vector Machines With an Approximated Marginal Likelihood and its Industrial Application
提出一种基于近似边际似然的相关向量机嵌入式特征选择方法，通过高斯近似和吉布斯采样估计后验分布，在基准和工业数据集上比现有方法预测更准、特征选择更好。
Feature selection is of great importance to make prediction for process variables in industrial production. An embedded feature selection method, based on relevance vector machines with an approximated marginal likelihood function, is proposed in this study. By setting hierarchical prior distributions over the model weights and the parameters of the automatic relevance determination kernel function, respectively, a Gaussian approximation method is designed to approximate the intractable exact marginal likelihood by using the law of the total expectation and the total covariance. Furthermore, in this study, the joint posterior distribution over the model weights and the kernel parameters is estimated by combining the Gibbs sampling with a Laplace approximation. Thus, feature selection is performed by examining the posterior over the kernel parameters. To verify the performance of the proposed method, a series of benchmark datasets and two practical industrial datasets are employed. The experimental results demonstrate that the proposed method not only produces higher prediction accuracy than other methods but also performs better in feature selection, especially in industrial case.