Ensemble Deep Random Vector Functional Link Neural Network for Regression
本文研究了深度随机向量函数链接网络在回归问题上的表现，提出了新的正则化方法和多种带跳跃连接的变体，在31个UCI数据集上验证了有效性。
Inspired by the ensemble strategy of machine learning, deep random vector functional link (dRVFL), and ensemble dRVFL (edRVFL) has shown state-of-the-art results on different datasets. Our present work first fills the gap of dRVFL and edRVFL work in the field of regression. We test and evaluate the performances of the dRVFLs on regression problems. Subsequently, we propose a novel regularization method boosted factor (BF), two dRVFLs variants edRVFL with skip connection (edRVFL-SC) and edRVFL with random skip connections (edRVFL-RSC) and one strategy ensemble skip connection edRVFL (esc-edRVFL) which show significant improvement over the original dRVFL. The BF is a newly introduced hyperparameter to scale the values of the activated hidden neurons to accommodate the diversity of the data, and it is also able to filter the neurons. edRVFL-SC and edRVFL-RSC are the edRVFL variants with skip connections. In edRVFL-SC, we apply dense skip connections to the edRVFL, which is inspired by the residual architecture in the deep learning area. However, due to the specificity of randomized networks, the simple skip connections are probably leading to the reuse of useless features. To address this problem, we propose a random skip connection-based edRVFL, which can keep the diversity in the latent space. esc-RVFL is an ensemble scheme that utilizes several edRVFL-RSC models trained on the different folds of the training dataset. The esc-edRVFL is identified as the best-performing algorithm through a comprehensive evaluation of 31 UCI datasets.