具有错误发现率保证的自适应新颖性检测

Adaptive novelty detection with false discovery rate guarantee

Annals of Statistics · 2024
被引 6
ABS 4★

中文导读

提出AdaDetect方法,利用半监督学习检测新颖性,无需分布假设即可在有限样本中控制错误发现率,适用于经济学等领域的异常检测。

Abstract

This paper studies the semisupervised novelty detection problem where a set of “typical” measurements is available to the researcher. Motivated by recent advances in multiple testing and conformal inference, we propose AdaDetect, a flexible method that is able to wrap around any probabilistic classification algorithm and control the false discovery rate (FDR) on detected novelties in finite samples without any distributional assumption other than exchangeability. In contrast to classical FDR-controlling procedures that are often committed to a pre-specified p-value function, AdaDetect learns the transformation in a data-adaptive manner to focus the power on the directions that distinguish between inliers and outliers. Inspired by the multiple testing literature, we further propose variants of AdaDetect that are adaptive to the proportion of nulls while maintaining the finite-sample FDR control. The methods are illustrated on synthetic datasets and real-world datasets, including an application in astrophysics.

统计学计量经济学机器学习假设检验