One-Step Diffusion Distillation Evolutionary Algorithm with Score Implicit Matching for Neural Architecture Search
提出OSD2E-SIM-NAS方法，将迭代扩散进化过程蒸馏为单步推理，计算复杂度从O(T)降至O(1)，在CIFAR-10/100和ImageNet上取得低错误率，生成任务中搜索成本仅0.39 GPU天。
This paper addresses two persistent challenges in Evolutionary Neural Architecture Search (ENAS): high computational cost and premature convergence. We propose a novel method—One-Step Diffusion Distillation Evolutionary Algorithm with Score Implicit Matching for NAS (OSD<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>E-SIM-NAS)—which significantly reduces the complexity of the search process while maintaining performance. The key innovation is the distillation of the iterative diffusion-based evolutionary process into a single inference step via Score Implicit Matching (SIM), reducing computational complexity from <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i>(<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T</i>) to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i>(1). SIM leverages implicit score matching to enable dynamic variance modulation, naturally balancing exploration and exploitation—an essential characteristic inherited from diffusion models. To further mitigate premature convergence and improve search robustness, we introduce a diversity-preserving mechanism that combines simulated annealing, stochastic teacher sampling, and periodic injection of new individuals. We validate the portability and effectiveness of OSD<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>E-SIM-NAS across both image classification and generative tasks. The method achieves test error rates of 2.55% on CIFAR-10, 16.15% on CIFAR-100, and 24.61% on ImageNet. In generative tasks on CIFAR-100, it attains an Inception Score (IS) of 9.01± 0.07, a Fréchet Inception Distance (FID) of 15.06, and a remarkably low search cost of 0.39 GPU-days, demonstrating superior efficiency, diversity, and adaptability across tasks.