Self-Supervised Learning for Intuitive Control of Prosthetic Hand Movements via Sonomyography
提出一种自监督学习算法，仅利用无标签前臂超声信号同时估计手腕旋转和手抓握运动，实验表明其性能优于监督学习，尤其适用于截肢者，为肌声图人机交互开辟新途径。
As a primary effector of humans, the hand plays a crucial role in many aspects of daily life. Recognizing multidegree-of-freedom hand movements from muscle activity helps infer human motion intentions. Solving this problem has direct applications in prosthetic and exoskeleton control. Here, we propose a self-supervised learning algorithm inspired by muscle synergies to achieve simultaneous estimation of wrist rotation (supination/pronation) and hand grasp (open/close) from sonomyography-the muscle deformation detected by a wearable ultrasound array. Unlike conventional methods collecting both muscle activity and hand kinematics for supervised model calibration, this algorithm only uses unlabeled forearm ultrasound signals for self-supervised wrist and hand movement estimation, where movement labels are auto-generated. The performance of the proposed algorithm was experimentally evaluated with ten participants including an amputee. Offline analysis demonstrated that the proposed algorithm can accurately estimate simultaneous wrist rotation and hand grasp movements and were 0.98 and 0.94 for the able-bodied, and 0.98 and 0.90 for the amputee, respectively). Notably, the performance of the self-supervised learning was superior to the supervised learning for the amputee. Online experiments demonstrated that intended wrist and hand movements can be deciphered in real time, enabling accurate control of a virtual hand. This study will open up a new avenue for the sonomyographic human-machine interaction.