Using LLMs as sentiment analyzers to predict review helpfulness: first insights to open the black box
研究用大语言模型分析在线评论情感，发现其对星级评分的判断准确度受商品情感性影响，且模型与原始评分的偏差能预测评论有用性，对营销研究和平台设计有参考价值。
Abstract This study examines the potential of large language models for sentiment analysis in marketing. Using the empirical setting of online customer reviews, we further explore implications for prediction of review helpfulness. Relying on a dataset of 28,900 product reviews from a consumer platform and an experiment with N = 1063 participants, we find that the LLM’s accuracy in assessing intended meaning (as in the star-rating) in customer-written text depends on the degree of emotionality, as in purchases of hedonic (vs. utilitarian) goods. We further demonstrate that deviations between LLM classification and original star rating predict lower review helpfulness. This effect is mediated by the deviation of human readers’ assumption on the intended star rating from the actual star rating and moderated by the degree of information asymmetry before the purchase; that is, a greater deviation between the LLM classification and the original star rating indicates lower review helpfulness for search goods than for experience goods.