Large-scale model comparison with fast model confidence sets
提出一种新算法,通过反向过程(从两个模型开始逐步添加)来构建模型置信集,将计算复杂度从O(M^3)降至O(M^2),内存成本从O(M^2)降至O(M),并支持后续添加模型,便于协作。
The paper proposes a new algorithm for finding the confidence set of a collection of forecasts or prediction models. Existing numerical implementations use an elimination approach, where one starts with the full collection of models and successively eliminates the worst performing until the null of equal predictive ability is no longer rejected at a given confidence level. The intuition behind the proposed implementation lies in reversing the process, i.e. starting with a collection of two models and updating both the model rankings and p-values as models are successively added to the collection. The first benefit of this approach is a reduction of one polynomial order in both the time complexity and memory cost of finding the confidence set of a collection of M models using the R rule, falling respectively from O(M^3) to O(M^2) and from O(M^2) to O(M). The second key benefit is that it allows for further models to be added at a later point in time, thus enabling collaborative efforts using the model confidence set procedure. The paper proves that this implementation is equivalent to the elimination approach, demonstrates the improved performance on a multivariate GARCH collection consisting of 4800 models, and discusses possible use-cases where this improved performance could prove useful.