module 01 / 08
Foundations
Build the base mental model: neural nets, transformers, language models, and why scale changed software.
the point
Everything else is easier when you understand what the model is doing, what it is not doing, and where the core abstractions came from.
start here
go deeper
understanding check
You should be able to…
- explain what a neural network learns
- describe attention without hiding behind the word attention
- explain tokens, embeddings, next-token prediction, and context windows
- know why transformers replaced earlier sequence models
prove it by building
Build a tiny autocomplete model
beginner · python, pytorch or tinygrad, a small text corpus
make
a notebook or script that trains a tiny character-level model and samples text
definition of done
show training loss, sample output, and a short note on what improved after tuning