Switching between touch and voice: factors influencing modality selection in multimodal systems
研究了多模态系统中用户选择触摸或语音模态的关键因素，发现每次触摸的音节数和菜单结构显著影响选择，而触摸次数影响不大。
Users frequently encounter multimodal systems and make decisions regarding the most suitable modality for a given task. This study examines users' modality selection in multimodal systems, focusing on the features and menu structures of available modalities. Modality selection tasks that considered the number of touches, syllables per touch (S/T), and menu structure were presented to participants. Binomial logistic regression analysis revealed that S/T and menu structure significantly predicted modality selection. The voice modality was predominantly chosen when S/T ranged from 2 to 3 in non-hierarchical tasks and 4 to 5 in hierarchical tasks. In contrast, the number of touches did not significantly predict modality selection. Additionally, modality features and menu structure significantly influenced interaction effort, including both physical and mental dimensions. This study identified modality switching points by evaluating the interaction effort associated with voice and touch modalities. This study identifies key factors - syllables per touch and menu structure - affecting touch and voice modality preferences in multimodal systems. Hierarchical tasks significantly influence the switching point. The findings provide a framework for designing adaptive interfaces that enhance usability and reduce interaction effort.