EvolCAF: Automatic Cost-Aware Acquisition Function Design Using Large Language Models
提出EvolCAF框架，结合大型语言模型与进化计算自动设计成本感知的采集函数，减少人工试错，在多种任务上效率与泛化性优于人类专家设计的EIpu和EI-cool方法。
To address optimization problems that involve expensive evaluations with unknown and heterogeneous costs, cost-aware Bayesian optimization (BO) emerges as a prominent solution in many real-world scenarios. However, as a critical step in developing BO algorithms, the design of efficient cost-aware acquisition functions (AFs) remains a significant challenge. This paper introduces EvolCAF, a novel framework that integrates large language models (LLMs) with evolutionary computation (EC) to automatically design cost-aware AFs. Leveraging the crossover and mutation in the algorithmic space, EvolCAF offers a novel design fashion, significantly reducing the reliance on domain expertise and labor-intensive trial-and-error process in the traditional manual design paradigm. We find the best AF designed by EvolCAF effectively utilizes the available information, including historical data, surrogate models and budget details. It introduces novel ideas not previously explored in the existing literature on acquisition function design, allowing for clear interpretations to provide insights into its behavior and decision-making process. In comparison to the well-known EIpu and EI-cool methods designed by human experts, our approach showcases remarkable efficiency and generalization across various synthetic and real-world tasks.