Bayes Reading Group: Bob Carpenter

Friday Apr 15, 2022, 11:00 AM → 12:00 PM America/New_York

Virtual/--Flatiron Institute (162 5th Avenue)

Discussion Lead: Bob Carpenter 

Topic: Fu, Luo, and Zhang. 2016. Quasi-Newton Hamiltonian Monte Carlo.  UAI.

Abstract: The Hamiltonian Monte Carlo (HMC) method has become significantly popular in recent years.It is the state-of-the-art MCMC sampler due to its more efficient exploration to the parameter space than the standard random-walk based proposal. The key idea behind HMC is that it makes use of first-order gradient information about the target distribution. In this paper, we propose a novel dynamics using second-order geometric information about the desired distribution. The second-order information is estimated by using a quasi-Newton method (say, the BFGS method), so it does not bring heavy computational burden. Moreover, our theoretical analysis guarantees that this dynamics remains the target distribution invariant. As a result, the proposed quasiNewton Hamiltonian Monte Carlo (QNHMC) algorithm traverses the parameter space more efficiently than the standard HMC and produces a less correlated series of samples. Finally, empirical evaluation on simulated data verifies the effectiveness and efficiency of our approach.We also conduct applications of QNHMC in Bayesian logistic regression and d online Bayesian matrix factorization problems.

Link: https://auai.org/uai2016/proceedings/papers/102.pdf