High-Level Decision Making in a Hierarchical Control Framework: Integrating HMDP and MPC for Autonomous Systems
提出一个集成连续动力学与离散马尔可夫决策过程的混合框架，用于自主系统的高层决策，并应用于智能车辆自动变道，仿真验证其在动态复杂环境中的有效性。
This article addresses challenges of autonomous decisions making influenced by discrete system states, underlying continuous dynamics, and evolving operational environments. A comprehensive framework is proposed, encompassing new modeling, problem formulation, control design, and stability analysis. The framework integrates continuous system dynamics, used for low-level control, with discrete Markov decision processes (MDP) for high-level decision making. To capture the interactions between these domains, the decision-making system is modeled as a hybrid system consisting of a controlled MDP and autonomous (uncontrolled) continuous dynamics, collectively referred to as the hybrid Markov decision process (HMDP). The design focuses on ensuring safety and optimality by accounting for both discrete and continuous state variables across different levels. With the help of the model predictive control (MPC) concept, a decision-making scheme is developed for the hybrid model, with guarantees for recursive feasibility and stability. The proposed framework is applied to the autonomous lane changing system for intelligent vehicles, and simulation shows its capability to handle diverse behaviors in dynamic and complex environments.