Reading List
Papers, books, essays, courses, and engineering blogs that have shaped how I think about systems.
This is the exact map I share when someone asks me where to start with AI and systems engineering. No filler; everything here earned its place. Jump to the Essential Reads or search the whole collection below.
🎥
Videoshelf
Iconic lectures, tutorials, and talks that every tech nerd should watch.
Must-Watch Series
Neural Networks: Zero to Hero
Andrej Karpathy
Neural Networks Visualized
3Blue1Brown
Stanford CS231n
Fei-Fei Li & Karpathy
Practical Deep Learning
fast.ai / Jeremy Howard
Essence of Linear Algebra
3Blue1Brown
Essence of Calculus
3Blue1Brown
MIT 18.06 Linear Algebra
Gilbert Strang / MIT OCW
Machine Learning, Clearly Explained
StatQuest / Josh Starmer
▶
Watch on YouTube
Stanford CS229: Machine Learning
Andrew Ng
Build a Large Language Model (From Scratch)
Sebastian Raschka
▶
Watch on YouTube
Diffusion & Large Vision Models
Stanford CME
Channels Worth Subscribing To
Legendary Talks & Lectures
Let's Build GPT from Scratch
Andrej Karpathy
Attention Is All You Need
Yannic Kilcher
Turing Award Lecture: Deep Learning
Yann LeCun, Yoshua Bengio, Geoffrey Hinton
Geoffrey Hinton on AI Risks
The Godfather of AI
Demis Hassabis: DeepMind
Lex Fridman Podcast
Sam Altman: OpenAI CEO
Lex Fridman Podcast
AlphaGo & AlphaZero
David Silver / DeepMind
John Carmack: Doom, Quake, VR, AGI
Lex Fridman Podcast
The Character of Physical Law
Richard Feynman
Jim Keller: Moore's Law & Future of Computing
Lex Fridman Podcast
The Computer Revolution Hasn't Happened Yet
Alan Kay / OOPSLA 1997
Dario Amodei: Anthropic CEO on AI Safety
Lex Fridman Podcast
George Hotz: Hacking, AI, Self-Driving
Lex Fridman Podcast
How We Teach Computers to Understand Pictures
Fei-Fei Li / TED
Historic Moments in Tech
The Mother of All Demos
Doug Engelbart, 1968
Steve Jobs Introduces Macintosh
Apple, 1984
Windows 95 Launch
Bill Gates & Jay Leno, 1995
Steve Jobs Introduces iPhone
Apple, 2007
First Android Demo
Google, 2007
AlphaGo - The Movie
DeepMind, 2016
First Falcon 9 Landing
SpaceX, 2015
GPT-4 Developer Livestream
OpenAI, 2023
Google Duplex AI Assistant
Google I/O, 2018
📚
Bookshelf
Technical
Fundamentals & Algorithms
📖
Structure and Interpretation of Computer Programs
Abelson & Sussman
📖
The Art of Computer Programming
Donald Knuth
📖
Introduction to Algorithms (CLRS)
Cormen, Leiserson, Rivest, Stein
📖
The Algorithm Design Manual
Steven Skiena
📖
Programming Pearls
Jon Bentley
📖
Compilers: Principles, Techniques, and Tools
Aho, Lam, Sethi, Ullman
Systems & Infrastructure
📖
Computer Systems: A Programmer's Perspective
Bryant & O'Hallaron
📖
Operating Systems: Three Easy Pieces
Arpaci-Dusseau
📖
Database Internals
Alex Petrov
📖
Database Systems
Hector Garcia-Molina
📖
Designing Data-Intensive Applications
Martin Kleppmann
📖
Site Reliability Engineering
Google SRE Team
Distributed Systems
📖
Distributed Systems
Maarten van Steen
📖
Understanding Distributed Systems
Roberto Vitillo
📖
Building Microservices
Sam Newman
📖
Release It!
Michael Nygard
Architecture & Design
📖
The Pragmatic Programmer
Andrew Hunt & David Thomas
📖
Clean Code
Robert C. Martin
📖
Refactoring
Martin Fowler
📖
Domain-Driven Design
Eric Evans
📖
Patterns of Enterprise Application Architecture
Martin Fowler
📖
System Design Interview
Alex Xu
Programming Languages
📖
The Go Programming Language
Alan A. A. Donovan
📖
Concurrency in Go
Katherine Cox-Buday
📖
Java Concurrency in Practice
Brian Goetz
📖
Introduction To Information Retrieval
Christopher Manning
Engineering Career
📖
The Mythical Man-Month
Fred Brooks
📖
Staff Engineer
Will Larson
📖
An Elegant Puzzle
Will Larson
Nerd Classics
Computing History
📖
The Soul of a New Machine
Tracy Kidder
📖
The Dream Machine
M. Mitchell Waldrop
📖
Where Wizards Stay Up Late
Katie Hafner
📖
Masters of Doom
David Kushner
📖
The Cuckoo's Egg
Clifford Stoll
Hacker Culture & Essays
📖
Hackers & Painters
Paul Graham
📖
Coders at Work
Peter Seibel
📖
The Phoenix Project
Gene Kim
📖
The Code Book: The Science of Secrecy
Simon Singh
Science & Mathematics
📖
Gödel, Escher, Bach: An Eternal Golden Braid
Douglas Hofstadter
📖
Surely You're Joking, Mr. Feynman!
Richard Feynman
📖
The Art of Doing Science and Engineering
Richard Hamming
📖
The Principles of Mathematics
Bertrand Russell
📖
The Elegant Universe
Brian Greene
📖
Chaos: Making a New Science
James Gleick
📖
The Information: A History, A Theory, A Flood
James Gleick
📖
What If? Serious Scientific Answers to Absurd Questions
Randall Munroe
AI, ML & The Future
Practical AI/ML Engineering
📖
AI Engineering
Chip Huyen
📖
Designing Machine Learning Systems
Chip Huyen
📖
Build a Large Language Model (From Scratch)
Sebastian Raschka
📖
Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow
Aurélien Géron
📖
Natural Language Processing with Transformers
Tunstall, von Werra, Wolf
📖
Machine Learning Design Patterns
Lakshmanan, Robinson, Munn
📖
The Hundred-Page Machine Learning Book
Andriy Burkov
Theory & Foundations
📖
Deep Learning
Goodfellow, Bengio, Courville
📖
Pattern Recognition and Machine Learning
Christopher Bishop
📖
Reinforcement Learning: An Introduction
Sutton & Barto
📖
Mathematics for Machine Learning
Deisenroth, Faisal, Ong
📖
Dive into Deep Learning
Zhang, Lipton, Li, Smola
📖
Why Machines Learn: The Elegant Math Behind Modern AI
Anil Ananthaswamy
📖
Linear Algebra Done Right
Sheldon Axler
📖
The Book of Why
Judea Pearl
📖
A Thousand Brains
Jeff Hawkins
AI Safety, Ethics & Philosophy
📖
Co-Intelligence: Living and Working with AI
Ethan Mollick
📖
The Alignment Problem
Brian Christian
📖
Human Compatible
Stuart Russell
📖
Superintelligence
Nick Bostrom
📖
Life 3.0
Max Tegmark
Business & Leadership
Startups & Entrepreneurship
📖
Zero to One
Peter Thiel
📖
The Lean Startup
Eric Ries
📖
Blitzscaling
Reid Hoffman
📖
The Innovator's Dilemma
Clayton Christensen
Management & Leadership
📖
High Output Management
Andy Grove
📖
The Hard Thing About Hard Things
Ben Horowitz
📖
Good to Great
Jim Collins
📖
Measure What Matters
John Doerr
📖
The Diary of a CEO
Steven Bartlett
Self-Improvement & Philosophy
Productivity & Habits
📖
Atomic Habits
James Clear
📖
Deep Work
Cal Newport
📖
Think Again
Adam Grant
Philosophy & Psychology
📖
Meditations
Marcus Aurelius
📖
Man's Search for Meaning
Viktor Frankl
📖
The Courage to Be Disliked
Ichiro Kishimi
📖
Thinking, Fast and Slow
Daniel Kahneman
📖
The Psychology of Money
Morgan Housel
Sci-Fi & Fiction
Classic AI & Consciousness
📖
I, Robot
Isaac Asimov
📖
2001: A Space Odyssey
Arthur C. Clarke
📖
Do Androids Dream of Electric Sheep?
Philip K. Dick
📖
Neuromancer
William Gibson
Simulation & Digital Minds
📖
Permutation City
Greg Egan
📖
Diaspora
Greg Egan
📖
Blindsight
Peter Watts
📖
Sum: Forty Tales from the Afterlives
David Eagleman
Singularity & Superintelligence
📖
A Fire Upon the Deep
Vernor Vinge
📖
Rainbows End
Vernor Vinge
📖
Accelerando
Charles Stross
AI Ethics & Development
📖
The Lifecycle of Software Objects
Ted Chiang
📖
Exhalation
Ted Chiang
📖
Stories of Your Life and Others
Ted Chiang
Post-Scarcity & AI Societies
📖
The Culture Series (Player of Games)
Iain M. Banks
📖
The Diamond Age
Neal Stephenson
📖
The Moon is a Harsh Mistress
Robert A. Heinlein
📖
Rendezvous with Rama
Arthur C. Clarke
Cyberpunk & Near-Future
📖
Snow Crash
Neal Stephenson
📖
Cryptonomicon
Neal Stephenson
📖
The Three-Body Problem
Cixin Liu
📖
Project Hail Mary
Andy Weir
Dystopian & Philosophical
📖
1984
George Orwell
📖
Brave New World
Aldous Huxley
📖
Contact
Carl Sagan
Epic Space Opera
📖
Dune
Frank Herbert
📖
Foundation
Isaac Asimov
📖
Ender's Game
Orson Scott Card
📖
The Hitchhiker's Guide to the Galaxy
Douglas Adams
📜
Papershelf
Research papers that shaped modern computing. Links to freely available versions where possible.
Machine Learning & AI
●
Attention Is All You Need
●
Language Models are Few-Shot Learners (GPT-3)
●
Improving Language Understanding by Generative Pre-Training (GPT)
●
BERT: Pre-training of Deep Bidirectional Transformers
●
PaLM: Scaling Language Modeling with Pathways
●
LLaMA: Open and Efficient Foundation Language Models
●
Deep Residual Learning for Image Recognition (ResNet)
●
ImageNet Classification with Deep CNNs (AlexNet)
●
Generative Adversarial Networks
●
Neural Machine Translation by Jointly Learning to Align and Translate
●
Word2Vec: Distributed Representations of Words
●
Scaling Laws for Neural Language Models
●
Training Compute-Optimal Large Language Models (Chinchilla)
●
SWE-CI: Evaluating Agent Capabilities via Continuous Integration
●
Retrieval-Augmented Generation (RAG)
●
Chain-of-Thought Prompting
●
Constitutional AI
●
Faster sorting algorithms discovered using deep RL (AlphaDev)
Google Infrastructure (The Classics)
●
The Google File System
●
MapReduce: Simplified Data Processing on Large Clusters
●
Bigtable: A Distributed Storage System for Structured Data
●
Spanner: Google's Globally-Distributed Database
●
F1: A Distributed SQL Database That Scales
●
The Chubby Lock Service
●
Borg: Large-scale Cluster Management at Google
●
Omega: Flexible, Scalable Schedulers for Large Compute Clusters
●
Dremel: Interactive Analysis of Web-Scale Datasets
●
Megastore: Scalable Highly Available Storage
●
Mesa: Geo-Replicated Data Warehousing
●
Pregel: Large-Scale Graph Processing
●
FlumeJava: Easy, Efficient Data-Parallel Pipelines
●
MillWheel: Fault-Tolerant Stream Processing
●
Dapper: Large-Scale Distributed Systems Tracing
●
Zanzibar: Google's Consistent, Global Authorization System
●
Monarch: Google's Planet-Scale Monitoring Infrastructure
●
Slicer: Auto-Sharding for Datacenter Applications
●
B4: Experience with a Globally-Deployed Software Defined WAN
●
Jupiter Rising: A Decade of Clos Topologies in Google's Datacenter
●
Maglev: A Fast and Reliable Software Network Load Balancer
●
TensorFlow: A System for Large-Scale Machine Learning
●
Pathways: Asynchronous Distributed Dataflow for ML
Distributed Systems & Databases
●
Dynamo: Amazon's Highly Available Key-value Store
●
Amazon Aurora: Design Considerations
●
Amazon DynamoDB: A Scalable NoSQL Database
●
Firecracker: Lightweight Virtualization
●
A Relational Model of Data for Large Shared Data Banks
●
Raft Consensus Algorithm
●
Paxos Made Simple
●
Time, Clocks, and Ordering of Events
●
Kafka: A Distributed Messaging System
●
The Log: What every software engineer should know
●
CockroachDB: The Resilient Geo-Distributed SQL Database
●
FoundationDB: A Distributed Unbundled Transactional Key Value Store
Data Structures & Algorithms
●
The LSM-Tree (Log-Structured Merge-Tree)
●
Space/Time Trade-offs in Hash Coding (Bloom Filters)
●
HyperLogLog: Cardinality Estimation
●
Skip Lists: A Probabilistic Alternative to Balanced Trees
●
The Bw-Tree: A B-tree for New Hardware
●
Consistent Hashing and Random Trees
●
Cuckoo Filter: Better Than Bloom
●
B-Trees and Database Systems
●
Gorilla: Fast Time Series Database
●
SIEVE: Efficient Cache Eviction
Classics & Foundational
●
Go To Statement Considered Harmful
●
No Silver Bullet - Essence and Accident in Software
●
Out of the Tar Pit
●
The UNIX Time-Sharing System
●
Bitcoin: A Peer-to-Peer Electronic Cash System
●
CAP Theorem: Twelve Years Later
●
How Complex Systems Fail
●
End-to-End Arguments in System Design
●
A Note on Distributed Computing
●
Reflections on Trusting Trust
📑
Essential Reads
The essays and guides I keep sending people, in the order I'd read them. If you want to understand modern AI from first principles, this is the shortest path I know.
Neural Nets & Transformers, Illustrated
From embeddings to attention to full architectures. Read in order and the transformer stops being magic.
1
The Illustrated Word2Vec
· Jay Alammar
Embeddings: the idea underneath everything else. Start here.
2Understanding LSTM Networks
· Christopher Olah
The classic that taught a generation what memory in a network means.
3The Unreasonable Effectiveness of Recurrent Neural Networks
· Andrej Karpathy
The 2015 essay that aged into prophecy: character models writing code and prose.
4The Illustrated Transformer
· Jay Alammar
Still the best first explanation of attention ever written.
5The Annotated Transformer
· Harvard NLP
The 'Attention Is All You Need' paper, rebuilt line by line in working PyTorch.
6The Illustrated GPT-2
· Jay Alammar
How decoder-only models actually generate text, one token at a time.
7Understanding and Coding Self-Attention From Scratch
· Sebastian Raschka
Attention in raw code, no framework magic.
8Attention Variants, Visualized
· Sebastian Raschka
Multi-head to grouped-query to sliding-window, in pictures.
9The Big LLM Architecture Gallery
· Sebastian Raschka
Every major open model architecture, side by side.
10Understanding Multimodal LLMs
· Sebastian Raschka
How text models learned to see.
11What Is ChatGPT Doing … and Why Does It Work?
· Stephen Wolfram
The longest, most patient answer to 'but how does it actually work?'
Training, Tuning & Reasoning
How models are actually trained and tuned, and why most failures are debugging failures.
1
A Recipe for Training Neural Networks
· Andrej Karpathy
The debugging discipline that separates practitioners from tourists.
2Yes, You Should Understand Backprop
· Andrej Karpathy
Why the abstraction leaks, and when you'll pay for not knowing.
3Practical Tips for Finetuning LLMs
· Sebastian Raschka
What actually matters when you tune: data, ranks, and evals.
4LoRA and DoRA From Scratch
· Sebastian Raschka
Parameter-efficient fine-tuning, implemented by hand.
5Understanding Reasoning LLMs
· Sebastian Raschka
What 'thinking' models changed, minus the hype.
Agents & AI Engineering
The gap between a model and a product. This is where I spend my working hours.
1
LLM Powered Autonomous Agents
· Lilian Weng
The survey that defined the agent stack: planning, memory, tools.
2Context Engineering
· Philipp Schmid
Prompting grew up. This is what it became.
3The Components of a Coding Agent
· Sebastian Raschka
A coding agent, disassembled into its working parts.
4The State of LLMs
· Sebastian Raschka
The annual field report worth reading end to end.
5Defeating Nondeterminism in LLM Inference
· Thinking Machines Lab
Why temperature 0 still isn't deterministic, and how to fix it.
6Getting Better at Using Coding Agents
· Sankalp
Honest field notes from serious daily agent use.
7Principles of Building AI Agents (free book)
· Sam Bhagwat
A short free book covering the 80% of agent-building that matters.
8ML System Design: 500+ Case Studies
· curated on GitHub
How real companies actually ship ML, in their own write-ups.
9AI at Work: Getting More Done With Less Hype
· Perplexity
A sober take on workplace AI, from a company selling it.
Perspective & Classics
Short, old, and still winning arguments today.
1
The Bitter Lesson
· Rich Sutton
One page. Explains the last 70 years of AI and predicts the next 10.
2The AI Revolution: The Road to Superintelligence
· Tim Urban / Wait But Why
The 2015 essay that made half the industry take AGI seriously.
3How to Read a Paper
· S. Keshav
The three-pass method. Read this before your next arXiv tab.
🌐
Blogs I Follow
My actual feed. If a post from one of these shows up, I read it.
Big Tech & AI Labs
Scale-ups & Unicorns
Infrastructure & DevTools
AI & ML Writers
Andrej Karpathy
Lilian Weng
Chris Olah
Jay Alammar
Chip Huyen
Eugene Yan
Sebastian Raschka
Simon Willison
Hamel Husain
Nicholas Carlini
Michael Nielsen
Greg Brockman
Ferenc Huszár (inFERENCe)
Finbarr Timbers
Sander Dieleman
Aleksa Gordić
Simon Boehm
Alfredo Canziani
Emilien Dupont
Alessio Devoto
naklecha
Maharshi
Sumit
Mat Miller
Max Brenndoerfer
Alexandru Burlacu
neuralnets
rsrch.space
Machine Learning Mastery
🎓
Learn & Connect
Where I'd practice, and where the good conversations happen.
Platforms & Courses
Neural Networks: Zero to Hero
Karpathy's from-scratch course. The single best on-ramp to deep learning.
Dive into Deep Learning (d2l.ai)
A full interactive textbook with runnable code in every chapter.
Math Academy
The most efficient way I know to rebuild math foundations as an adult.
Khan Academy Math
Free, patient, and covers everything up to the ML prerequisites.
MIT OpenCourseWare
Real MIT courses, free. Start with 18.06 Linear Algebra.
Brilliant
Interactive problem-solving for math and CS intuition.
AI by Hand
Tom Yeh's worked-by-hand exercises. Do attention with a pencil once.
TensorTonic
Implement ML algorithms and GPU kernels from scratch, in the browser.
LLM Architecture Gallery
Raschka's interactive tour of every major LLM architecture.
Learn Your Way
Google's experiment in AI-adapted textbooks.
Applied AI Prep
Drill-based prep for applied-AI and agent engineering interviews: 850+ real loop questions and structured courses.
Deep Learning Fundamentals
Raschka's free video course, from unit one to production.
Intro to Machine Learning
The companion ML course. Classical foundations first.
Math Foundations for ML
A community-built Notion roadmap of exactly what math to learn, in order.
Intuition and Insights
A growing Notion collection of ML intuitions worth stealing.
Artificial Intelligence (GitBook)
Leonardo Araujo Santos's free book of AI notes and derivations.
Communities
r/MachineLearning
Research discussion, paper threads, and the occasional legendary AMA.
r/deeplearning
More beginner-friendly DL questions and project feedback.
r/LocalLLaMA
The best signal anywhere on running open models yourself.
r/ChatGPT
Where the mainstream meets the models. Useful for sensing the zeitgeist.