Gem Alert

Here is a list of manuscripts (papers/articles/monographs/tutorials etc.) that I find myself referring to in an unexpectedly diverse variety of contexts.

Probabilistic Modelling and Variational Inference

  1. Albergo, Boffi, Vanden-Eijden: Stochastic Interpolants: A Unifying Framework for Flows and Diffusions. (official impl) rigorous, unifying analysis and generalization of continous-time stochastic interpolation
  2. McAllester, Stratos: Formal Limitations on the Measurement of Mutual Information (official impl) fundamental KL and MI estimation bounds, interesting proof technique
  3. Krueger et al.: Bayesian Hypernetworks very flexible and underexplored Bayesian model
  4. Touchette: The large deviation approach to statistical mechanics digestible bridge to statistical mechanics for people with ml bg
  5. Luo: Understanding Diļ¬€usion Models: A Unified Perspective have only skimmed through this, but recommended reading imo
  6. Zhi-Han Training Latent Variable Models with Auto-encoding Variational Bayes: A Tutorial intuitive exposition of AEVB as approximate EM
  7. Marion et al.: Implicit Diffusion: Efficient Optimization through Stochastic Sampling (official impl) optimize implicit models from samples/without going through likelihood estimation

Nonparametric and Semiparametric Inference

  1. Chernozhukov et al.: Automatic Debiased Machine Learning via Riesz Regression (official impl for a follow-up work) foundational for AutoDML
  2. Foster and Syrgkanis: Orthogonal Statistical Learning (official impl) analysis like AutoDML outside the standard M/Z-estimand form
  3. Tibshirani: Conformal Prediction gentle and balanced introduction to conformal prediction
  4. Han et al.: Optimal rates of entropy estimation over Lipschitz balls synthesis of various ways of using simple polynomial approximations for estimating functionals

Representation Learning

  1. Zhai et al.: Contextures: Representations from Contexts fundamental work in our understanding of representation learning + scaling laws, math is a little terse though
  2. Tsai, Yeh, Ravikumar: Faith-Shap: The Faithful Shapley Interaction Index (official impl) feature importance w/o unhealthy assumptions

Misc.

  1. Lin: Bayesian Epistemology scratches a certain itch