A Multi-Fidelity Genetic Algorithm for Hyperparameter Optimization of Deep Neural Networks
提出一种名为GAMF2O的多保真遗传算法，通过结合低保真评估结果和学习能力来优化深度神经网络的超参数，降低计算成本，并在分类和回归问题上超越现有方法。
Hyperparameter optimization on Machine Learning models is crucial for their correct refinement. For complex big models (such as Deep Learning models), in which a single training model is supposed to have a very high computational cost, this optimization sometimes becomes unfeasible. Multi-fidelity optimization algorithms are a solution to alleviate this computational cost of optimizing hyperparameters of Deep Learning models. In this scope, we propose GAMF2O, a new multi-fidelity algorithm that relies on a genetic algorithm. This paper clearly defines how to adapt the evolutionary scenario to follow the multi-fidelity approach, and we propose a new scheme to evaluate each individual based on the use of two objectives: the result of the low-fidelity evaluation and the learning capacity, with the use of the latter being novel during the evaluation process. Our experimental section allows us to show how our proposal improves the state-of-the-art in different classification and regression problems.