Analytic natural gradient updates for Cholesky factor in Gaussian variational approximation
推导了高斯变分近似中协方差或精度矩阵乔列斯基因子的解析自然梯度更新，解决了正定性问题，并提出了带动量的随机归一化自然梯度上升法，适用于广义线性混合模型和深度神经网络。
Abstract Natural gradients can improve convergence in stochastic variational inference significantly but inverting the Fisher information matrix is daunting in high dimensions. Moreover, in Gaussian variational approximation, natural gradient updates of the precision matrix do not ensure positive definiteness. To tackle this issue, we derive analytic natural gradient updates of the Cholesky factor of the covariance or precision matrix and consider sparsity constraints representing different posterior correlation structures. Stochastic normalized natural gradient ascent with momentum is proposed for implementation in generalized linear mixed models and deep neural networks.