A Group Regularization Framework of Convolutional Neural Networks Based on the Impact of Lp Regularizers on Magnitude
提出一个评估组正则化惩罚效果的新框架，通过分析Lp正则化对权重幅度的影响，发现L1,2正则化在结构化剪枝中表现良好，并据此设计了一种混合组正则化方法。
Group regularization is commonly employed in network pruning to achieve structured model compression. However, the rationale behind existing studies on group regularization predominantly hinges on the sparsity capabilities of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{p}$ </tex-math></inline-formula> regularizers. This singular focus may lead to erroneous interpretations. In response to these limitations, this article proposes a novel framework for evaluating the penalization efficacy of group regularization methods by analyzing the impact of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{p}$ </tex-math></inline-formula> regularizers on weight magnitudes and weight group magnitudes. Within this framework, we demonstrate that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{1,2}$ </tex-math></inline-formula> regularization, contrary to prevailing literature, indeed exhibits favorable performance in structured pruning tasks. Motivated by this insight, we introduce a hybrid group regularization approach that integrates <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{1,2}$ </tex-math></inline-formula> regularization and group <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{1/2}$ </tex-math></inline-formula> regularization (denoted as HGL1,2&<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{1/2}$ </tex-math></inline-formula>). This novel method addresses the challenge of selecting appropriate <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{p}$ </tex-math></inline-formula> regularizers for penalizing weight groups by leveraging <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{1,2}$ </tex-math></inline-formula> regularization for penalizing groups with magnitudes exceeding a critical threshold while employing group <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{1/2}$ </tex-math></inline-formula> regularization for other groups. Experimental evaluations are conducted to verify the efficiency of the proposed hybrid group regularization method and the viability of the introduced framework.