Regularizing Multilayer Perceptron for Robustness
针对硬件实现中多层感知器易受连接故障、乘性噪声和加性噪声影响的问题，提出一种通过添加三个正则化项来惩罚系统误差的方法，在回归和分类任务上验证了有效性。
The weights of a multilayer perceptron (MLP) may be altered by multiplicative and/or additive noises if it is implemented in hardware. Moreover, if an MLP is implemented using analog circuits, it is prone to stuck-at 0 faults, i.e., link failures. In this paper, we have proposed a methodology for making an MLP robust with respect to link failures, multiplicative noise, and additive noise. This is achieved by penalizing the system error with three regularizing terms. To train the system we use a weighted sum of the following four terms: 1) mean squared error (MSE); 2) l <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> norm of the weight vector; 3) sum of squares of the first order derivatives of MSE with respect to weights; and 4) sum of squares of the second order derivatives of MSE with respect to weights. The proposed approach has been tested on ten regression and ten classification tasks with link failure, multiplicative noise, and additive noise scenarios. Our experimental results demonstrate the effectiveness of the proposed regularization to achieve robust training of an MLP.