Rates of Convergence in the Central Limit Theorem for Markov Chains, with an Application to TD Learning
用Stein方法和泊松方程证明了向量值鞅差序列和马尔可夫链函数的非渐近中心极限定理，并应用于带平均的时序差分学习算法。
We prove a nonasymptotic central limit theorem (CLT) for vector-valued martingale differences using Stein’s method, and we use Poisson’s equation to extend the result to functions of Markov chains. We then show that these results can be applied to establish a nonasymptotic CLT for temporal difference learning with averaging. Funding: This work was supported by National Science Foundation [Grants CNS 23-12714, CCF 22-07547, and CNS 21-06801] and Air Force Office of Scientific Research [Grant FA9550-24-1-0002].