EvoREP: Evolving Reliable Ensemble of Proxies for Zero-Shot Neural Architecture Search
提出一种非线性集成方法EvoREP，通过多样性感知的自适应粒子群优化算法聚合九个零成本代理，实现对候选网络质量的可靠评估，在多个NAS基准上取得高相关性。
Zero-shot neural architecture search (NAS) offers a significant boost to the efficiency of automated network design, where zero-cost proxies play a key role in evaluating the quality of candidate networks without training. However, most existing proxies measure network capability by a single specific characteristic, which is insufficient to achieve a high correlation with the final performance. Although some studies attempt to use multiple proxies for better estimation performance, they often fall short in considering the inherently multi-dimensional properties of neural networks and/or rely on simple combinations of these proxies. This work presents an effective nonlinear ensemble method to aggregate nine representative zero-cost proxies aiming at various network characteristics for comprehensive network evaluations. To obtain a reliable aggregation of proxies on a given task and efficiently utilize candidate base proxies, a diversity-aware adaptive particle swarm optimization (DAA-PSO) approach is developed to evolve the ensemble proxy. DAA-PSO maintains a pool of best solutions with varying sparsity levels to enhance the diversity of particles and employs an adaptive solution sparsification strategy to effectively identify the most suitable base proxies for the ensemble. The proposed method, named EvoREP (evolving reliable ensemble of zero-cost proxies), can efficiently discover the best ensemble proxy for the target task. Extensive experiments on popular NAS benchmarks and open-domain search spaces demonstrate the effectiveness and promising transferability of EvoREP in evolving ensemble proxies, exhibiting remarkably high correlation with test accuracy.