Using the Empirical Attainment Function for Analyzing Single-Objective Black-Box Optimization Algorithms
本文提出用经验达到函数替代目标经验累积分布函数来评估黑箱优化算法，无需预设质量目标，能更精确捕捉性能差异，并集成到IOHanalyzer平台便于使用。
A widely accepted way to assess the performance of iterative black-box optimizers is to analyze their empirical cumulative distribution function (ECDF) of predefined quality targets achieved not later than a given runtime. In this work, we consider an alternative approach, based on the empirical attainment function (EAF) and we show that the target-based ECDF is an approximation of the EAF. We argue that the EAF has several advantages over the target-based ECDF. In particular, it does not require defining a priori quality targets per function, captures performance differences more precisely, and enables the use of additional summary statistics that enrich the analysis. We also show that the average area over the convergence curves is a simpler-to-calculate, but equivalent, measure of anytime performance. To facilitate the accessibility of the EAF, we integrate a module to compute it into the IOHanalyzer platform. Finally, we illustrate the use of the EAF via synthetic examples and via the data available for the black-box optimization benchmark suite.