Bayes Reading Group: Hugh Dance (UCL)

Friday Dec 19, 2025, 11:00 AM → 12:00 PM America/New_York

3rd Floor Conference Room (162 Fifth Avenue )

Discussion lead:  Hugh Dance (UCL)

Title: Efficiently Vectorized MCMC on Modern Accelerators

Abstract: With the advent of automatic vectorization tools  writing multi-chain MCMC algorithms is often now as simple as invoking those tools on single-chain code. Whilst convenient, for various MCMC algorithms this results in a synchronization problem -- loosely speaking, at each iteration all chains running in parallel must wait until the last chain has finished drawing its sample. In this work, we show how to design single-chain MCMC algorithms in a way that avoids synchronization overheads when deploying automatic vectorization tools, via the framework of finite state machines (FSMs). Using a simplified model, we derive an exact theoretical form of the obtainable speed-ups using our approach, and use it to make principled recommendations for optimal algorithm design. We implement several popular MCMC algorithms as FSMs, including Elliptical Slice Sampling, HMC-NUTS, and Delayed Rejection, demonstrating speed-ups of up to an order of magnitude in experiment