A field-first mastery book on modern ML systems

The Holy Grail

130 chapters and 5 appendices covering ML foundations, training, inference internals, production serving, CUDA programming, model optimization, retrieval, agents, distributed systems, observability, build/deploy, and ML system design interviews. The ML track ramps Beginner → Expert → PhD → Production; every chapter ends with curated references and practice problems.

Start reading Interview prep path (56 chapters) Preface
Chapters
130
Parts
10
Appendices
5
Questions
530+

The interview-prep path

56 chapters

The focused track for ML systems interviews. Covers the concepts interviewers actually test, skipping material that's important for production but rarely asked about. Start here if you have 2–4 weeks. Read front to back if you have more time.

Part III — Inference
21 Prefill vs decode: the two-phase nature of LLM inference 22 The KV cache: the single most important optimization in LLM inference 23 Batching: static, dynamic, continuous (Orca) 24 PagedAttention and vLLM as a virtual-memory system for KV cache 25 FlashAttention and the GPU memory hierarchy 26 Quantization: INT8, INT4, FP8, AWQ, GPTQ, SmoothQuant 30 Cost modeling for inference: tokens, GPUs, dollars 31 Latency budgets, tail latency, and the p99 problem 33 The attention compression family: MHA, MQA, GQA, MLA 34 Mixture of Experts: routing, balancing, and the inference cost story 36 Disaggregated prefill/decode: production reality with workload-dependent payoff 44 The serving framework landscape: vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, MLC, Triton 45 Inference servers and orchestration: KServe, BentoML, Seldon, Ray Serve, Triton Inference Server
Part X — Interview Playbook
114 The interview framework: clarify, estimate, design, drill, ops 115 Capacity planning math: the back-of-envelope kit 116 Design a chatbot for 1M users 117 Design a RAG system over 10TB of documents 118 Design a content moderation pipeline 119 Design a real-time recommendation system 120 Design a multi-tenant model serving platform 121 Design a multi-tenant fine-tuning service 122 The vocabulary that interviewers respect 123 Common mistakes and how to recover mid-interview 124 The coding interview: twenty ML systems algorithms 125 Design: the upstream platforms 126 Design: the production infrastructure 127 Design: the frontier scenarios 128 Behavioral interviews and the levels ladder 129 The day of the interview 130 Company-specific prep and mock interview transcripts

Also essential: Appendix E — 530+ interview questions organized by topic, difficulty, and role with "what they want to hear" rubrics.

Contents

Ten parts plus five appendices. Click any chapter to start reading.

Part I

ML Foundations

10 chapters · Chapters 1–10

Part II

Training, Fine-Tuning, Alignment

10 chapters · Chapters 11–20

Part III

Inference Internals & Production Serving

36 chapters · Chapters 21–56

Part IV

Information Retrieval & RAG

9 chapters · Chapters 57–65

Part V

Agents, Tool Use, Workflow Orchestration

7 chapters · Chapters 66–72

Part VI

Distributed Systems & Request Lifecycle

12 chapters · Chapters 73–84

Part VII

The Data Plane

7 chapters · Chapters 85–91

Part VIII

Observability, Reliability, Incidents

9 chapters · Chapters 92–100

Part IX

Build, Deploy, Operate

13 chapters · Chapters 101–113

Part X

ML System Design Interview Playbook

17 chapters · Chapters 114–130

Appendices