Welcome

About Me:

I'm Avi Amalanshu, a 5th year undergraduate student (B.Tech + M.Tech dual degree) at IIT-KGP. These days, I'm doing research on neurosymbolic and distributed learning-- broadly, the synthesis of hardcore statistical ML and systems (PL & OS/metrics). Check out my website to learn more.

About This:

This is going to contain two things:

My recommendations
My own personal reading list

across anything you can read:

Textbooks (organized by broad area)
Novels (including non-academic works)
- I have lost the habit of reading over the past few years and I'm trying to start again. So most of this is a to-read for now.
Conference proceedings (mainly intended as a recording of my reading notes/summaries)
Individual Research Papers
Monographs

Use the sidebar to navigate. Happy reading!

Inference and Learning

I think people just getting into ML often try to get away with the bare minimum of probability. I am a staunch believer in studying in-context from statistics and information theory, even if your personal interests don't require them (if they do, I suggest a more rigorous treatment than what I prescribe here). Besides gaining a more fundamental and usable understanding, the bigger picture-view makes it easier to focus on the right things. And, extra insight never hurt anyone.

Probability & Statistics Basics

Though quite brief, worth a skim for context: Pishro-Nik's online probability text. (H. Pishro-Nik, "Introduction to probability, statistics, and random processes", available at probabilitycourse.com, Kappa Research LLC, 2014).
Bertsekas and Tsitsiklis' book is not only a great introduction to the tools you'll need but is also covered through free lecture notes and a great MIT OCW course w/ lecture videos. (D.P. Bertsekas, J.N. Tsitsiklis "Introduction to probability" 2nd ed, Athena Scientific, 2008).
The first few chapters of Steven Kay's parametric estimation book will give you an idea of theoretical constraints such as bias-variance tradeoff. Plus, it's a new (sigproc) angle to look at the same thing; always useful. If you find yourself interested in Bayesian methods the second half of the book may also be useful. (S.M. Kay "Fundamentals of statistical signal processing: estimation theory", Prentice-Hall signal processing series, 1993)

Information Theory and Inference

David MacKay's book is a fantastic and very readable introduction to the math behind inference. The book mentions which chapters are useful for which readers. Could skip the stuff on coding theory but it's fun + useful for some emerging ML topics. (D.J. MacKay "Information Theory, Inference, and Learning Algorithms" v7.2, Cambridge University Press, 2005)
If you're interested in insight on issues with sample complexity/data efficiency, Chapter 11 of Cover & Thomas might be useful. (T.M. Cover, J.A. Thomas "Elements of Information Theory" 2nd ed, Wiley, 1991)

Alternatively...

I've heard ISLR/ISLP is great and covers pretty much all the important parts of what I've described above. I haven't read it myself (it's rather new), but it looks great if you're into less math rigor and more English-heavy explanations. The authors are folks who have developed some pretty fundamental works in statistics in their own research. (G. James, D. Witten, T. Hastie, R. Tibshirani, J. Taylor "An introduction to statistical learning" w/ Applications in Python, Springer, 2023).

State-of-the-Art Learning

Though some think it's long in the tooth, at least the first two sections of the OG Deep Learning book are excellent background. (I. Goodfellow, Y. Bengio, A. Courville "Deep learning", MIT Press, 2016).
The Bishops' Deep Learning book is a more modern (as of '24) book. I haven't read this myself either but the author's OG ML book, Pattern Recognition & Machine Intelligence, was great. (C.M. Bishop, H. Bishop "Deep Learning Foundations and Concepts", Springer, 2024).

Special Topics

WIP

Monographs

I love monographs because you can basically inhale one in a single evening and be ready to put in work the next morning. Here are some monographs + other short texts that I found useful in various categories:

Awesome Machine Learning Monographs

...and book chapters

Neurosymbolic and Neurocausal Learning

Neuro-Causal Models (Aragam and Ravikumar 2024)
Causal Fairness Analysis - A Causal Toolkit for Fair Machine Learning (Plečko and Bareinboim 2024)
Neurosymbolic AI for Reasoning Over Knowledge Graphs: A Survey (DeLong et al. 2023)
Neurosymbolic Programming (Chaudhuri et al. 2021)

Conformal Prediction

Conformal Prediction: A Gentle Introduction (Angelopoulos et al. 2022)
A Tutorial on Conformal Prediction (Shafer and Vovk 2008)

Distributed Learning

Advances and Problems in Federated Learning (Kairouz et al. 2019)

Misc.

Foundation Models for Natural Language Processing (Paaß and Giesselbach 2023) -- technically a book but pick any chapter 🤪
An Introduction to Variational Autoencoders (Kingma and Welling 2019)
Algorithms for Reinforcement Learning (Szepesvàri 2009)
Learning Deep Architectures for AI (Bengio 2009)

Awesome Systems Monographs

Messing around with systems is the right way to do things. Blogs are a more intimate and accessible way for people who have messed around with systems to communicate their findings. I plan to make a blog post section here, which will probably be more useful than this.