Hierarchical Motion Learning for Goal-Oriented Movements With Speed–Accuracy Tradeoff of a Musculoskeletal System
研究如何在肌肉骨骼系统中通过分层运动学习框架实现快速且准确的目标导向运动，引入菲茨定律和基底神经节回路模型来动态决策，并改进策略梯度算法生成肌肉激励，实验证明该方法在自适应运动生成上优于其他强化学习算法。
Generating various goal-oriented movements via the flexible muscle model of the musculoskeletal system as fast and accurately as possible is a pressing problem, which is also the basis of most human adaptive behaviors, such as reaching, catching, interception, and pointing. This article focuses on the adaptive motion generation of fast goal-oriented motion on the musculoskeletal system by implementing the speed-accuracy tradeoff (SAT) in a hierarchical motion learning framework. First, we introduce Fitts' Law into the modified basal ganglia circuit-inspired iterative decision-making model for achieving dynamic and adaptive decision making. Then, as a time constraint, the decision is decomposed into a series of supervised terms by the proposed striatal FSI-SPN interneuron circuit-inspired velocity modulator to implement the tradeoff smoothly on the musculoskeletal system. Finally, an improved policy gradient algorithm is suggested to generate the muscle excitations of the modulated motion via the proposed muscle co-contraction policy, which promotes general cooperation between flexor and extensor muscles. In experiments, a redundant musculoskeletal arm model is trained to perform the adaptive quick pointing movements. By combining the muscle co-contraction policy with SAT, our algorithm shows the most efficient training and the best performance in the adaptive motion generation among the other three popular reinforcement learning algorithms on the musculoskeletal model.