Hierarchical Optimization Design for Autonomous Flight of Vision-Based Quadrotor Using Reinforcement Learning
将四旋翼自主飞行问题分为控制层和决策层，用强化学习分层优化，设计并行策略迭代算法和课程学习机制，提升窄空间穿越任务的安全性和效率。
Although quadrotor has been widely used in practical engineering, its autonomous flight ability needs to be improved in complex operating environment. The autonomous flight problem for quadrotor with monocular vision is investigated, which is divided into control layer and decision layer in this article. Reinforcement learning method is utilized for hierarchical optimization to ensure that quadrotor completes narrow space traversal tasks safely and efficiently. First, considering the dynamic characteristics of the quadrotor with the motor speed as the control input, a parallel policy iteration algorithm is designed for the nonaffine nonlinear system, and the proposed controller can be learned online to improve the fundamental control performance. On this basis, the autonomous decision problem with visual information as input is modeled as a Markov decision process, and a curriculum learning mechanism is introduced to overcome the difficulties caused by sparse reward. At the same time, the clipping function is optimized to improve the learning efficiency of proximal policy optimization (PPO) algorithm for autonomous flight capabilities. Finally, the effectiveness of the proposed intelligent control and decision methods are verified through simulation.