An Online Evolving Method For a Safe and Fast Automated Vehicle Control System
提出一种在线演化有限状态机方法，自动构建最优马尔可夫驾驶模型，结合强化学习控制器，在模拟多车道场景中实现更安全快速的自动驾驶。
An online evolving method, named evolving finite state machine (e-FSM), is proposed to develop an optimal Markov driving model. The model has the same properties as a standard Markov model, but its states and transition dynamics evolve without human supervision. In this article, we introduce: 1) the principles of the e-FSM’s novel capabilities: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">online state determination</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">online transition-dynamics identification</i> for elaborating the Markov driving model and 2) an advanced online evolving framework (a-OEF) for supporting the reinforcement-learning-based controller’s decision making by using the evolved model. For the evaluation of the proposed methodology and framework, the ego vehicle is controlled by the double deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula> -network (DDQN) controller with and without the a-OEF in the multilane driving scenario where various naturalistic traffic situations are simulated. Simulation results show that better control performance in terms of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fast</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">safe</i> driving is achieved via the DDQN with the a-OEF, which demonstrates that the Markov driving models evolved by the e-FSMs effectively support detecting and revising the controller’s incorrect decision making.