week8.md

Week 8

Lecture

• Lecture #14: Hamiltonian Monte Carlo
  • Lecture Video
  • Lecture Notes
  • Lecture Summary
• Lecture #15: Parallel Tempering and Stochastic HMC
  • Lecture Video
  • Lecture Notes
  • Lecture Summary

Activities

• In-Class Exercise #14: Introduction to HMC
• In-Class Exercise #14 Solutions
• In-Class Exercise #15: Applying HMC to Bayesian Logistic Regression Models
• In-Class Exercise #15: Solutions
• Homework #7
• Homework #7 Solutions

Reading

Applications and Broader Impact

1. The Cloud is a Factory (Chapter 1 from Your Computer is on Fire, available through Hollis)
2. The carbon impact of artificial intelligence
3. Green AI
4. The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Research

Hamiltonian Monte Carlo:

1. (Introductory) A Conceptual Introduction to Hamiltonian Monte Carlo
2. (In-Depth) MCMC using Hamiltonian dynamics

For Nice Visualizations of HMC Samplers:

1. (Introductory) Markov Chains: Why Walk When You Can Flow?
2. (Introductory) Hamiltonian Monte Carlo explained

For those really interested in rigor:

1. (Advanced) Geometric integrators and the Hamiltonian Monte Carlo method
2. (Advanced) On the convergence of Hamiltonian Monte Carlo

Parallel Tempering:

1. (In-Depth) Parallel tempering: Theory, applications, and new perspectives

Improvements to HMC and MCMC in General:

1. (Research Paper) Stochastic Gradient Hamiltonian Monte Carlo
2. (Research Paper) The No-U-Turn Sampler
3. (Research Paper) Riemannian Manifold Hamiltonian Monte Carlo
4. (Research Paper) Does Hamiltonian Monte Carlo mix faster than a random walk on multimodal densities?