Knowledge Transfer-Based Sparse Deep Belief Network
提出一种基于知识迁移和网络结构优化的深度信念网络KT-SDBN，通过规则提取与融合生成稀疏网络，在仅用30%参数时仍保持良好学习性能，适合关注模型压缩与效率的研究者。
Deep learning has made remarkable achievements in various applications in recent years. With the increasing computing power and the "black box" problem of neural networks, however, the development of deep neural networks (DNNs) has entered a bottleneck period. This article proposes a novel deep belief network (DBN) based on knowledge transfer and optimization of the network structure. First, a neural-symbolic model is proposed to extract rules to describe the dynamic operation mechanism of the deep network. Second, knowledge fusion is proposed based on the merge and deletion of the extracted rules from the DBN model. Finally, a new DNN, knowledge transfer-based sparse DBN (KT-SDBN) is constructed to generate a sparse network without excessive information loss. In comparison with DBN, KT-SDBN has a more sparse network structure and better learning performance on the existing knowledge and data. The experimental results in the benchmark data indicate that KT-SDBN not only has effective feature learning performance with 30% of the original network parameters but also shows a large compression rate that is far larger than other structure optimization algorithms.