Benchmarking Analysis of Evolutionary Neural Architecture Search
针对进化神经架构搜索缺乏专用基准函数的问题，基于NAS-Bench-101提出了三个基准函数，通过特征提取和函数工具包实现搜索空间划分和收敛性分析，实验验证了其正确性。
Evolutionary computation-based neural architecture search (ENAS) is a popular technique for automating the architecture design of deep neural networks. For any evolutionary computation-based algorithm, the runtime and convergence are the most important aspects concerned by theoretical analysis. However, because of the lacking of benchmarked fitness functions specialized for ENAS, the corresponding theoretical work is rarely available. To address this issue, we propose three different benchmark functions in this paper based on NAS-Bench-101. Specifically, we first propose a correlation-based feature extraction method, to capture the accuracy relationship between neural architectures and their fitness values. Furthermore, we propose a function toolkit, which allows combining different architecture features to specific benchmark functions. In addition, three benchmark functions are derived upon the toolkit by considering the features of neural net topologies, the features of neural operations, and their combinations. Based on these designs, the search space partition and transition probability calculation could be easily established, which in turn greatly promote the runtime and convergence analysis. We perform the experiments of ranking correlation, and the experimental results demonstrate the correctness of the proposed benchmark functions. To the best of our knowledge, this is the first work focusing on ENAS benchmark functions.