I have been long wanting to compare diffusion models to MCMC empirically. Running Langevin using the annealing scheme from diffusion models results in good samples even for hard targets. Details at jakiw.com/diffusion_as_annea… 1/n

However, one can try to approximate such an annealing scheme by choosing p_t = p(alpha * x). A very short first attempt already improves upon the above annealing scheme! 6/n

For a Gaussian mixture, one can calculate the solution to the forward SDE in diffusion models/score-based generative models in closed form. Therefore, we can use this path as smoothed versions and plug it into a Langevin SDE. Turns out, this works nearly perfect! 4/6

The Jupyter notebook here contains an introduction to SGMs using JAX: jakiw.com/sgm_intro It sets a focus on understanding some theoretical aspects as well as their generalization capabilities.