Isotonic mechanism for exponential family estimation in machine learning peer review
针对机器学习会议中作者对多篇投稿的质量排序，提出等渗机制调整评审分数，在保持排序的同时提升估计精度，并证明作者有动机如实排序。
Abstract In 2023, the International Conference on Machine Learning (ICML) required authors with multiple submissions to rank their papers by perceived quality. In this paper, we leverage these author-specified rankings to enhance peer review in machine learning and artificial intelligence conferences by extending the isotonic mechanism to exponential family distributions. This mechanism produces adjusted scores closely aligned with the original scores while strictly adhering to the author-specified rankings. An appealing feature of the mechanism is its applicability to a broad class of exponential family distributions without requiring knowledge of the specific distribution form. We show an author is incentivized to provide accurate rankings if her utility is a convex additive function of the adjusted review scores. For a subclass of exponential family distributions, we prove that an author reports truthfully only if elicitation involves pairwise comparisons between her submissions, thus highlighting the optimality of rankings in truthful information elicitation. Moreover, the adjusted scores significantly enhance estimation accuracy compared to original scores and achieve near-minimax optimality when ground-truth scores have bounded total variation. We conclude with a numerical analysis using ICML 2023 ranking data, demonstrating substantial estimation improvements in approximating a proxy ground-truth quality of submissions via the isotonic mechanism.