Powerful Significance Testing for Unbalanced Clusters
针对SigClust方法在候选簇规模不平衡时检验力不足的问题,提出一种基于k-means聚类推广的改进方法,在平衡与不平衡场景下均表现强效,并在肾癌基因表达数据中验证了其价值。
Clustering methods are popular for revealing structure in data, particularly in the high-dimensional setting common to contemporary data science. A central statistical question is “are the clusters really there?” One pioneering method in statistical cluster validation is SigClust, but it is severely underpowered in the important setting where the candidate clusters have unbalanced sizes, such as in rare subtypes of disease. We show why this is the case and propose a remedy that is powerful in both the unbalanced and balanced settings, using a novel generalization of k-means clustering. We illustrate the value of our method using a high-dimensional dataset of gene expression in kidney cancer patients. A Python implementation is available at https://github.com/thomaskeefe/sigclust.