海量数据的分布式统计推断

Distributed statistical inference for massive data

Annals of Statistics · 2021
被引 31
ABS 4★

中文导读

研究了海量数据下一般对称统计量的分布式推断,给出了估计效率和渐近分布,并提出了两种分布式自助法来近似分布,其中伪分布式自助法在数据子集数量大时尤其有效。

Abstract

This paper considers distributed statistical inference for general symmetric statistics in the context of massive data with efficient computation. Estimation efficiency and asymptotic distributions of the distributed statistics are provided, which reveal different results between the nondegenerate and degenerate cases, and show the number of the data subsets plays an important role. Two distributed bootstrap methods are proposed and analyzed to approximation the underlying distribution of the distributed statistics with improved computation efficiency over existing methods. The accuracy of the distributional approximation by the bootstrap are studied theoretically. One of the methods, the pseudo-distributed bootstrap, is particularly attractive if the number of datasets is large as it directly resamples the subset-based statistics, assumes less stringent conditions and its performance can be improved by studentization.

统计推断大数据计算统计分布式计算自助法