Scaling Up With Integrity: Valid and Efficient Narrative Policy Framework Analyses Using Large Language Models
本研究展示了如何利用大语言模型(零样本、少样本和微调分类技术)自动识别叙事元素,以解决大规模政策文档分析中的效度和效率问题,为叙事政策框架研究者提供可复现的方法。
ABSTRACT Given vast quantities of digital and online data—such as news articles, congressional testimony, and social media posts—the potential for large scale narrative analyses has dramatically increased. Narrative Policy Framework (NPF) researchers can now access thousands or even millions of policy‐relevant documents. However, these large datasets also bring new challenges, including how to scale analyses while maintaining validity and reliability. Traditionally, NPF studies employ labor intensive human coding to identify narrative elements, limiting the number of documents that can be feasibly analyzed. Recognizing the potential of large language models (LLMs) to automate such analyses, this study demonstrates the use of zero‐shot, few‐shot, and fine‐tuned classification techniques to identify narrative elements. By prioritizing construct validity at each step in the process, we offer a rigorous and replicable approach to integrating LLMs into narrative policy research.