Bing Cheng and Howell Tong’s contribution to the Discussion of ‘Statistical exploration of the Manifold Hypothesis’ by Whiteley et al.
本文讨论了流形假设在信息几何和生成统计中的局限性，并提出基于信息几何的检验方法，通过协变量Fisher信息矩阵的特征分解来检验流形假设，对统计学习和AI研究者有参考价值。
Due to space limitations, we highlight only two issues with the manifold hypothesis (MH) here: (1) information geometry speaking, MH represents a strong restriction on the data space; (2) in terms of generative statistics/AI, the MH reduces the capacity of a generative model to represent the real world. To address the above issues, among others, we briefly outline below an information geometrical approach. Let χ be a data space. Consider an infinite-dimensional manifold M={f|f(x),x∈χ} of smooth probability density functions. Define the tangent space where ft(x)=γ(t)(x), γ:(−δ,δ)→M is a smooth curve, t being the parameter such that γ(0)=f. Endow TfM with the Fisher–Rao metric gf at f∈M by It holds that and it is algebraically equivalent to the space of score functions, namely where s(x)=h(x)/f(x). Define the covariate tangent subspace S of TfM of dimension ≤n by Then, because S is a finite-dimensional closed subspace of TfM, TfM=S⊕S⊥,meaning to every h∈TfM, there exists uniquely the orthogonal decomposition where hs∈S, ε(x)∈S⊥, and w∈Rn. Define the covariate Fisher Information Matrix (cFIM) as Gf=((Gf)ij)n×n, where (Gf)ij=gf(∂f/∂xi,∂f/∂xj). Let D be a locally parameterized manifold via the smooth embedding x=r(y), where y∈Rd, the intrinsic coordinates with d<n. Let Ml=span{u1,…,ud} a subset of TfM, where the basis vectors uj’s are the intrinsic scores uj=∂lnf(x)/∂yj,j=1,…,d. Denote the cFIM on D (the intrinsic FIM) by Under general conditions, it holds that Mathematically speaking, the MH is essentially one of characterizing ID by analysing cFIM. The above information geometry setup facilitates testing the MH, namely vs An Eigen-decomposition of the empirical cFIM immediately furnishes a practical test. Further, note that trace(Gf) is an information measure called the G-entropy, which has been exploited to produce, among others, a fast model selection criterion (Bian et al. 2025, https://arxiv.org/abs/2503.06331). No funding received. No data used or generated.