ChatGPT for Textual Analysis? How to Use Generative LLMs in Accounting Research
探讨了ChatGPT等生成式大语言模型在会计研究文本分析中的应用，比较了与传统方法的优劣，并通过案例展示其高准确率，为研究者提供使用指南。
Generative large language models (GLLMs), such as ChatGPT and GPT-4 by OpenAI, are emerging as powerful tools for textual analysis tasks in accounting research. GLLMs can solve any textual analysis task solvable using nongenerative methods as well as tasks previously only solvable using human coding. Whereas GLLMs are new and powerful, they also come with limitations and present new challenges that require care and due diligence. This paper highlights the applications of GLLMs for accounting research and compares them with existing methods. It also provides a framework on how to effectively use GLLMs by addressing key considerations, such as model selection, prompt engineering, and ensuring construct validity. In a case study, I demonstrate the capabilities of GLLMs by detecting nonanswers in earnings conference calls, a traditionally challenging task to automate. The new GPT method achieves an accuracy of 96% and reduces the nonanswer error rate by 70% relative to the existing Gow et al. (2021) method. Finally, I discuss the importance of addressing bias, replicability, and data sharing concerns when using GLLMs. Taken together, this paper provides researchers, reviewers, and editors with the knowledge and tools to effectively use and evaluate GLLMs for academic research. This paper was accepted by Eric So, accounting. Funding: Supported by the Foster School of Business – University of Washington. Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2023.03253 .