Urban traffic evaluation with social media data: A consensus-based LLM negotiation paradigm
本文提出一种基于大语言模型（LLM）的协商范式，利用社交媒体数据自动采集、处理并评价城市交通，通过两个LLM迭代协商达成共识，提升评价的可靠性和精准性，并在上海和北京的微博数据上验证了有效性。
Evaluating urban traffic systems from the perspective of travelers is critical for traffic planning and operations management, and social media data has been proven to be a rich and timely source for extracting travel experiences. However, it is challenging to evaluate urban traffic using social media data due to difficulties in obtaining a high-quality dataset that reveals travel experiences and conducting precise evaluations based on the collected posts. With recent advancements in large language models (LLMs), this paper establishes a consensus-based LLM negotiation paradigm to address these challenges. Firstly, we propose an LLM-driven automated data collection and processing pipeline for social media data. Specifically, the understanding and reasoning abilities of LLMs facilitate filtering the obtained raw data with high accuracy. More importantly, LLMs do not require any manual labeling of data or extensive pre-training compared to traditional machine learning methods. As for the evaluation system, we leverage the LLMs to cluster and summarize the key indicators for urban traffic evaluation. Furthermore, we propose a consensus-based LLM negotiation mechanism in which two distinct LLMs serve as the generator and discriminator, respectively. They iteratively evaluate and refine their assessments for each social media post until consensus is reached. In this way, the text data is classified into its associated indicators, with scores assigned based on sentiment analysis. Lastly, we propose the LLM-augmented analytic hierarchy process (AHP), by which the weights of each indicator are determined by LLMs with different role-playing prompts. The effectiveness of the proposed framework is validated in Shanghai and Beijing using social media posts collected from Weibo. The obtained dataset by LLMs is significantly enhanced in both quality and quantity. Moreover, the negotiation mechanism of LLMs can provide not only reliable and robust evaluation results, but also identify key issues behind the evaluation results and offer targeted and precise suggestions. This paper contributes to the use of LLMs in future traffic evaluation and operations.