Experience Evolution-Guided Multi-Objective Reinforcement Learning
提出经验演化引导的多目标强化学习方法，通过演化经验偏好权重提升训练效果，避免高维参数空间限制，在连续和离散任务上优于现有方法。
The combination of Evolutionary Algorithms (EAs) and Reinforcement Learning (RL) has been found to be effective in single-objective RL. However, in general, existing methods cannot be directly extended to Multi-Objective RL (MORL) with policies that take preferences as their input, because they utilize EAs to optimize policy parameters while the evaluation of such policies is prohibitively expensive. Thus, this article proposes Experience Evolution-guided MORL (E<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>MORL), which evolves the preference weight of the experiences to enhance their contributions to RL training. This experience-level evolution avoids the limitations of high-dimensional parameter spaces and can provide superior experiences to promote the agent’s learning. Meanwhile, the generated experiences are evaluated by their utilities under specific preferences, which obviates the need for massive interactions during the evaluation of the population. In addition, the population obtains well-performing candidates from the RL agent and maintains diversity through crowding distance to better cover the Pareto front. Experiments on continuous and discrete tasks validate the superiority of E<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>MORL over several state-of-the-art MORL methods.