CSCI 699: Probabilistic and Generative Models, Spring 2025

This course focuses on probabilistic models and their central role within modern machine learning and generative modeling. With probabilistic methods increasingly driving advancements in AI, this course will explore its applications across a range of topics, including , approximate inference algorithms (MCMC, variational inference), deep generative models (autoregressive, score-matching, diffusion, and flow-based models), and model-based sequential decision making.

Course Staff

Willie Neiswanger
Instructor
Office hours: Friday 5:30-6:30pm
Location: KAP 144

Oliver Liu
Teaching Assistant
Office hours: Wednesday 3:30-4:30pm
Location: GCS 4th Floor

Logistics

Assignments: Submit all written assignments, including all project-related write-ups, on Brightspace. Grades and feedback will also be provided on Brightspace.
General discussion: Please use the official course Slack channel for general questions.
Other discussion: Email Willie and Oliver (neiswang@ and zliu2898@) or come to office hours to discuss individual matters, such as project ideas or grading.

Prerequisites

This course is designed for students currently pursuing research, or who wish to pursue research, in probabilistic machine learning or deep generative models. It will be beneficial to have familiarity with machine learning (at the level of CSCI 567), algorithms (at the level of CSCI 570), and probability (at the level of MATH 505a). Students are expected to be comfortable with reading and presenting modern machine learning conference papers.

Schedule

Date	Topic	Notes	Due in Class
Fri Jan 17	Class Overview	Overview on probabilistic and generative models, class outline, probability review.
Fri Jan 24	PGM 1	Intro to probabilistic graphical models (PGMs).	Student introductions
Fri Jan 31	PGM 2	Classic algorithms in PGMs for exact and approximate inference.
Fri Feb 7	Approximate Bayes 1	Markov chain Monte Carlo (MCMC), Metropolis-Hastings, Gibbs sampling.	Project Pitches 1
Fri Feb 14	Approximate Bayes 2	Variational inference (VI) and gradient-based MCMC (LMC, HMC).	Project Pitches 2
Fri Feb 21	VAE & GAN	Variational autoencoders (VAEs) and generative adversarial networks (GANs).
Fri Feb 28	Autoregressive Models	Autoregressive models, transformer architecture, LLMs.	Paper presentations.
Fri Mar 7	Diffusion Models	Score-based generative models and diffusion models.	Paper presentations.
Fri Mar 14	Flow-based Models	Normalizing flows, continuous normalizing flows, flow matching.	Paper presentations.
Fri Mar 21		Spring Break!
Fri Mar 28	Predictive UQ	Predictive uncertainty quantification (UQ), classic/neural models.	Midway report (all). Paper presentations.
Fri Apr 4	Active Learning	Active Learning, Bayesian optimization, sequential decision making.	Paper presentations.
Fri Apr 11	Generative Decisions	Using generative models within decision making procedures.	Paper presentations.
Fri Apr 18	TBA	TBA	Paper presentations.
Fri Apr 25		Final Presentations 1.	Final presentations.
May 2		Final Presentations 2	Final presentations.