Graph Neural Network-Based Surrogate Model for Evolutionary Neural Architecture Search
提出一种基于图神经网络的双阶段代理模型DSGENAS，通过图嵌入提取架构特征并联合两个代理模型，在降低计算成本的同时实现准确稳定的架构性能预测，在NAS基准上取得最优结果。
Evolutionary neural architecture search (ENAS) is an approach to automating network architecture design. A primary challenge of ENAS is the demand for expensive computational resources, and many ENAS methods use surrogate models to reduce costs. However, existing architecture representation methods are not rich enough in capturing the information of architectures, making them inadequate for building effective surrogate models. Furthermore, existing research studies often rely on a single model or strategy to predict architecture performance, which is not always accurate or reliable. To alleviate these issues, this article proposes a dual-stage surrogate model based on graph neural networks (GNNs) for ENAS, namely dual-stage surrogate model for evolutionary neural architecture search (DSGENAS). First, to effectively represent the neural architectures for the surrogate model, we employ a GNN-based architecture embedding method to extract architecture features. Second, based on the architecture representation, a dual-stage surrogate model strategy is proposed and integrated into the ENAS framework. This strategy combines two surrogate models in the evolutionary search process, one for global performance-tier learning and the other for local performance relationship learning. Experimental results show that extracting architecture features through GNNs can achieve a more effective architecture representation. The two different surrogate models can jointly assist the search process in ENAS. Furthermore, DSGENAS can achieve accurate and stable prediction results, obtaining state-of-the-art results on neural architecture search (NAS) benchmarks.