Bayes Reading Group: Sifan Liu (Duke)

Friday Sep 19, 2025, 11:00 AM → 12:00 PM America/New_York

3rd Floor Conference Room (162 Fifth Avenue )

Discussion Lead: Sifan Liu (Duke)

Topics: Mean-Field Variational Inference along Principal Component Axes

Abstract: Mean-field variational inference (MFVI) approximates a target distribution with a product distribution in the standard coordinate system, offering a scalable approach to Bayesian inference but often underestimating uncertainty. We show that MFVI can be greatly improved when performed along carefully chosen principal component axes rather than the standard coordinates. The principal components are obtained from a cross-covariance matrix of the target’s score function and identify orthogonal directions that explain the most coordinatewise discrepancies between the target and a Gaussian reference. Performing MFVI in these axes directly targets the largest deviations from Gaussian, yielding a more accurate approximation

MFVI in a rotated coordinate system defines a rotation and a coordinatewise map that together brings the target closer to Gaussian. Iterating this procedure yields a sequence of transformations that progressively transforms the target to Gaussian. The resulting algorithm provides a computationally efficient construction of normalizing flows, requiring only MFVI sub-problems and avoiding large-scale optimization. In posterior sampling tasks, we demonstrate that the proposed method achieves higher accuracy than standard MFVI, while maintaining much lower computational cost than normalizing flows. This is joint work with Yifan Chen (UCLA).