Conformal prediction after data-dependent model selection
针对数据依赖模型选择后构造有效共形预测集的问题，提出无需额外数据分割的新方法，保证有限样本有效性并渐近最优宽度，适用于模型选择与预测集构建。
Given a family of pretrained models and a hold-out set, how can we construct a valid conformal prediction set while selecting a model that minimizes the width of the set? If we use the same hold-out data set both to select a model (the model that yields the smallest conformal prediction sets) and then to construct a conformal prediction set based on that selected model, we suffer a loss of coverage due to selection bias. Alternatively, we could further split the data to perform selection and calibration separately, but this comes at a steep cost if the size of the dataset is limited. In this paper, we address the challenge of constructing a valid prediction set after data-dependent model selection—commonly, selecting the model that minimizes the width of the resulting prediction sets. Our novel methods can be implemented efficiently and admit finite-sample validity guarantees without invoking additional sample-splitting. We show that our methods yield prediction sets with asymptotically optimal width under certain notions of regularity for the model class. The improvement in the width of the prediction sets constructed by our methods is further demonstrated through applications to synthetic datasets in various settings and a real data example.