Bionic AR: Standalone Augmented Reality System Controlled by Multimodal Biosignals
提出一种名为Bionic AR的独立增强现实系统，通过嵌入头戴式显示器的电极采集脑电、眼电和面部肌电信号，实现脑机接口、眼动书写、面部表情和无声语音识别，各模块实时准确率均超过98%。
Augmented reality (AR) technology can elevate user experiences by facilitating intuitive and immersive communication. Commercial AR devices typically rely on hand-held controllers or cameras to generate control commands; however, neither of these approaches is feasible when hand usage is limited. To address this limitation, we introduce a novel stand-alone AR system“, Bionic AR.” This system integrates a head-mounted display (HMD) with biosignal-based human–computer interface (HCI) technology. By embedding passive dry electrodes into the proposed AR headset in contact areas such as the occipital area, brow ridge, and cheeks, the system can simultaneously record electroencephalography (EEG), electrooculography (EOG), and facial electromyography (fEMG) data. The acquired multimodal biosignals were utilized to implement a steady-state visual evoked potential-based brain–computer interface (BCI), an eye-writing pattern recognition system, a facial expression recognition system, and a silent speech recognition system. In real-time experiments, each HCI module achieved accuracies exceeding 98%, underscoring the feasibility and practical potential of the proposed Bionic AR system.