Universum parametric $$\nu $$-support vector regression for binary classification problems with its applications
提出一种结合Universum数据的参数ν-支持向量回归方法，通过求解一个优化问题得到两个非平行超平面，并处理异方差噪声，在人工和真实数据集上提升了分类精度。
Abstract Universum data sets, a collection of data sets that do not belong to any specific class in a classification problem, give previous information about data in the mathematical problem under consideration to enhance the classifiers’ generalization performance. Recently, some researchers have integrated Universum data into the existing models to propose new models which result in improved classification performance. Inspired by these Universum models, an efficient parametric $$ \nu $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>ν</mml:mi></mml:math> -support vector regression with Universum data ( $$ {\mathfrak {U}} $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>U</mml:mi></mml:math> Par- $$ \nu $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>ν</mml:mi></mml:math> -SVR) is proposed in this work. This method, which finds two non-parallel hyperplanes by solving one optimization problem and considers heteroscedastic noise, overcomes some common disadvantages of the previous methods. The $$ {\mathfrak {U}} $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>U</mml:mi></mml:math> Par- $$ \nu $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>ν</mml:mi></mml:math> -SVR includes unlabeled samples that don’t belong to any class in the training process, which results in a quadratic programming problem. Two approaches are proposed to solve this problem. The first approach derives the dual formulation using the Lagrangian function and KKT conditions. Furthermore, a least squares parametric $$ \nu $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>ν</mml:mi></mml:math> -support vector regression with Universum data (named LS- $$ {\mathfrak {U}} $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>U</mml:mi></mml:math> Par- $$ \nu $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>ν</mml:mi></mml:math> -SVR) is suggested to further increase the generalization performance. The LS- $$ {\mathfrak {U}} $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>U</mml:mi></mml:math> Par- $$ \nu $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>ν</mml:mi></mml:math> -SVR solves a single system of linear equations, instead of addressing a quadratic programming problem in the dual formulation. Numerical experiments on artificial, UCI, credit card, NDC, and handwritten digit recognition data sets show that the suggested Universum model and its solving methodologies improve prediction accuracy.