Natural Gradient Variational Bayes Without Fisher Matrix Analytic Calculation and Its Inversion
提出一种迭代逼近Fisher信息矩阵逆的方法，避免解析计算和显式求逆，实现高效变分贝叶斯推断，适用于高斯近似和归一化流等场景。
This article introduces a method for efficiently approximating the inverse of the Fisher information matrix, a crucial step in achieving effective variational Bayes inference.A notable aspect of our approach is the avoidance of analytically computing the Fisher information matrix and its explicit inversion.Instead, we introduce an iterative procedure for generating a sequence of matrices that converge to the inverse of Fisher information.The natural gradient variational Bayes algorithm without analytic expression of the Fisher matrix and its inversion is provably convergent and achieves a convergence rate of order O(log s/s), with s the number of iterations.We also obtain a central limit theorem for the iterates.Implementation of our method does not require storage of large matrices, and achieves a linear complexity in the number of variational parameters.Our algorithm exhibits versatility, making it applicable across a diverse array of variational Bayes domains, including Gaussian approximation and normalizing flow Variational Bayes.We offer a range of numerical examples to demonstrate the efficiency and reliability of the proposed variational Bayes method.Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.