Beyond Distribution Shift: Investigating Generalisation in Automated Algorithm Selection for Single-Objective Black-Box Optimisation
该研究提出三层次评估框架，系统考察单目标黑箱优化中自动算法选择模型在分布偏移下的泛化能力，发现分布偏移、问题特征与算法性能关联等因素共同影响泛化，且少量目标域数据即可显著提升跨域泛化。
Automated algorithm selection enhances black-box optimisation by recommending best-performing algorithms tailored to problem features. However, key factors influencing the generalisation ability of automated algorithm selection remain poorly understood. This study proposes a triple-level evaluation framework to systematically investigate the generalisation ability of such models in single-objective black-box optimisation under progressively increasing distribution shift. Problem instances are generated from two distinct benchmarking suites, represented using Exploratory Landscape Analysis features. Density-based and distance-based metrics are employed to explicitly quantify the distribution shift between training and test problem sets. The complementarity of training and test problem sets is further assessed in both feature and algorithm performance spaces to uncover additional factors influencing generalisation. The results show that automated algorithm selection models consistently outperform a baseline, although performance degrades as the distribution shift between training and test problem sets increases. Distribution shift alone does not fully explain performance variations. Problem class–specific landscape features, feature–algorithm performance associations, and training set representativeness are also identified as critical determinants of generalisation. Furthermore, reallocating even a small fraction of target-domain data to training problem sets significantly enhances cross-domain generalisation.