Welcome to Avi's reading-list!

I'm Avi Amalanshu, an incoming MS in ML student at CMU and former 5 year B.Tech + M.Tech student at IIT-KGP. I'm interested in representaiton learning, particularly pertaining to higher-order reasoning and symbolic execution. Check out my website to learn more.

About This:

Right now, this thing is documenting my summer readings. For the past few years, I've saved a bunch of blogs, papers and texts that I found intriguing but didn't immediataely have the time to do justice to. This is probably the last time in the forseeable future that I can read whatever I want out of my own curiosity, so I plan on clearing out that backlog.

I also want to hit the ground running with my coursework at CMU this Fall. So I will read as close to that material as my whimsies permit. And revise some basics.

Use the sidebar to navigate. Happy reading!

Summer Syllabus

Analysis

Introduction to Real Analysis

  1. Abbott chapters 1-4
  2. Carothers chapters 1, 3-4
  3. Big Rudin chapters 2-5

Measure Theory

  1. PMATH450
  2. Big Rudin chapters 6-8

Functional Analysis Basics

  1. Inf Desc chapters 1-5
  2. Carothers chapters 10-12

Bibliography

Category Theory

  1. WWC chapters 1.0-1.2
  2. Chapter 0 chapters 1-3
  3. nLab wikies Category, Category Theory
  4. ...
    • Work chapters 1-2, 5-7
    • CTC chapters 1-5
  5. Categories OCW chapters 2, 4-7

Bibliography

Combinatorics

  1. MAT307 lectures 1-6
  2. ...
    • PTC
    • CTGD

Enumerative Basics

  1. Inf Desc chapters 6-8
  2. Ardila lectures 1-7
  3. Counting

Symbolic Methods

  1. Ardila lectures 10-19
  2. Analytic chapters 1-3

Linear Algebraic Methods

  1. LinAlg

Extremal Basics

  1. Extremal chapters 6-8

Bibliography

Probability & Statistics

Probability Theory

  1. Wasserman chapters 1-5
  2. Probability Course

Statistical Learning Theory

  1. ESL chapter 1, 5
  2. MacKay chapter 3
  3. Wasserman chapter 6, 8-10

Supervised Learning

  1. ESL chapter 2

Regression

  1. ESL chapter 3
  2. Wasserman chapter 13

Classification

  1. ESL chapter 4
  2. MacKay chapter 20-22

Model Selection and Complexity

  1. ESL chapter 7
  2. MacKay chapter 28

Additive Models and Trees

  1. ESL chapters 9-10

Information and Coding Theory

  1. MacKay chapters 4-6, 8-17, 24

Probabilistic Graphical Models

Graphical Models

  1. Murphy chapter 4

Undirected Models

  1. MacKay chapter 31
  2. Wasserman chapter 18

Directed Models

  1. MacKay chapter 37
  2. Wasserman chapter 17

Variational Methods

  1. MacKay chapter 33

Sampling and Monte Carlo Methods

  1. MacKay chapters 29-30, 32
  2. Bishop-Bishop chapter 14

Deep Generative Models

  1. MacKay chapter 43
  2. VAEs
  3. Bishop-Bishop chapters 17-18, 20

Causality and GNNs

  1. Bishop-Bishop chapter 13
  2. Wasserman chapter 16
  3. Bayesian Epistemology
  4. Elements of Causal Inference chapters 1, 3-4
  5. Pearl chapters 7-11

Bibliography

Programming Languages

Lambda Calculus

  1. Programming w/ Math
  2. Lambda Tutorial

Language Design

C

  1. Brief Intro to C
  2. DevC

C++

  1. History of Cpp
  2. Brief Intro to Cpp
  3. Evolution of Cpp
  4. ThrivingCpp

Declarative and Functional Languages

Lisp & co.

  1. Evolution of Lisp
  2. Evolution of EmacsLisp
  3. History of Clojure

Haskell

  1. History of Haskell
  2. GreatGood

ML

  1. History of StandardML

Prolog

  1. Brief Intro to Prolog
  2. BirthOfProlog

Bibliography

Computer Systems

Books/Monographs

Papers

  1. Matt Kline; What every systems programmer should know about concurrency
  2. Ulrich Drepper; What Every Programmer Should Know About Memory
  3. Alexander Stephan and Lars Wustrich; The Path of a Packet through the Linux Kernel

Blogs

  1. Matthew Plant; Writing a Simple Garbage Collector in C
  2. Tal Hoffman; Software Transactional Memory: a stairway to lock-free programming heaven?
  3. Sidharth M.; How Nginx Handles Thousands of Concurrent Requests
  4. Toby (from Wales); PicoCTF 2021 - Binary Exploitation Challenge Writeups

Other

  1. The Missing Semester

Algorithms

Novels

See my Goodreads: click here

Old

Inference and Learning

I think people just getting into ML often try to get away with the bare minimum of probability. I am a staunch believer in studying in-context from statistics and information theory, even if your personal interests don't require them (if they do, I suggest a more rigorous treatment than what I prescribe here). Besides gaining a more fundamental and usable understanding, the bigger picture-view makes it easier to focus on the right things. And, extra insight never hurt anyone.

Probability & Statistics Basics

  1. Though quite brief, worth a skim for context: Pishro-Nik's online probability text. (H. Pishro-Nik, "Introduction to probability, statistics, and random processes", available at probabilitycourse.com, Kappa Research LLC, 2014).
  2. Bertsekas and Tsitsiklis' book is not only a great introduction to the tools you'll need but is also covered through free lecture notes and a great MIT OCW course w/ lecture videos. (D.P. Bertsekas, J.N. Tsitsiklis "Introduction to probability" 2nd ed, Athena Scientific, 2008).
  3. The first few chapters of Steven Kay's parametric estimation book will give you an idea of theoretical constraints such as bias-variance tradeoff. Plus, it's a new (sigproc) angle to look at the same thing; always useful. If you find yourself interested in Bayesian methods the second half of the book may also be useful. (S.M. Kay "Fundamentals of statistical signal processing: estimation theory", Prentice-Hall signal processing series, 1993)

Information Theory and Inference

  1. David MacKay's book is a fantastic and very readable introduction to the math behind inference. The book mentions which chapters are useful for which readers. Could skip the stuff on coding theory but it's fun + useful for some emerging ML topics. (D.J. MacKay "Information Theory, Inference, and Learning Algorithms" v7.2, Cambridge University Press, 2005)
  2. If you're interested in insight on issues with sample complexity/data efficiency, Chapter 11 of Cover & Thomas might be useful. (T.M. Cover, J.A. Thomas "Elements of Information Theory" 2nd ed, Wiley, 1991)

Alternatively...

I've heard ISLR/ISLP is great and covers pretty much all the important parts of what I've described above. I haven't read it myself (it's rather new), but it looks great if you're into less math rigor and more English-heavy explanations. The authors are folks who have developed some pretty fundamental works in statistics in their own research. (G. James, D. Witten, T. Hastie, R. Tibshirani, J. Taylor "An introduction to statistical learning" w/ Applications in Python, Springer, 2023).

State-of-the-Art Learning

  1. Though some think it's long in the tooth, at least the first two sections of the OG Deep Learning book are excellent background. (I. Goodfellow, Y. Bengio, A. Courville "Deep learning", MIT Press, 2016).
  2. The Bishops' Deep Learning book is a more modern (as of '24) book. I haven't read this myself either but the author's OG ML book, Pattern Recognition & Machine Intelligence, was great. (C.M. Bishop, H. Bishop "Deep Learning Foundations and Concepts", Springer, 2024).

Special Topics

WIP

Monographs

I love monographs because you can basically inhale one in a single evening and be ready to put in work the next morning. Here are some monographs + other short texts that I found useful in various categories:

Awesome Machine Learning Monographs

...and book chapters

Neurosymbolic and Neurocausal Learning

Conformal Prediction

Distributed Learning

Misc.

Awesome Systems Monographs

Messing around with systems is the right way to do things. Blogs are a more intimate and accessible way for people who have messed around with systems to communicate their findings. I plan to make a blog post section here, which will probably be more useful than this.

PL

OS

Hardware/Arch

Awesome Programming Monographs

(...and whitepapers/What Every Programmer Should Know About X)

Papers

WIP. For now, enjoy Ilya Sutskever(?)'s 27 papers.

Conferences

Coming soon. (Think something like Awesome Papers)