Bayes Reading Group: Kamélia Daudel (ESSEC Business School)

Friday Oct 25, 2024, 11:00 AM → 12:00 PM America/New_York

3rd Floor Conference Room (162 Fifth Avenue)

Discussion Lead: Kamélia Daudel

Topic: Learning with Importance Weighted Variational Inference

Link: 

1. https://www.jmlr.org/papers/volume24/22-1160/22-1160.pdf

2. https://arxiv.org/pdf/2410.12035 

Abstract: Several popular variational bounds involving importance weighting ideas have been proposed to generalize and improve on the Evidence Lower BOund (ELBO) in the context of maximum likelihood optimization, such as the Importance Weighted Auto-Encoder (IWAE) and the Variational Rényi (VR) bounds. The methodology to learn the parameters of interest using these bounds typically amounts to running gradient-based variational inference algorithms that incorporate the reparameterization trick. However, the way the choice of the variational bound impacts the outcome of variational inference algorithms can be unclear. 

 

In this talk, we will present and motivate the VR-IWAE bound, a novel variational bound that unifies the ELBO, IWAE and VR bounds methodologies. In particular, we will provide asymptotic analyses for the VR-IWAE bound and its reparameterized gradient estimator, which reveal the advantages and limitations of the VR-IWAE bound methodology while enabling us to compare of the ELBO, IWAE and VR bounds methodologies. Our work advances the understanding of importance weighted variational inference methods and we will illustrate our theoretical findings empirically.