Module 01 / 08
Foundations
Build the base mental model: neural nets, transformers, language models, and why scale changed software.
The point
Everything else is easier when you understand what the model is doing, what it is not doing, and where the core abstractions came from.
Start with
Read next
Understanding check
You should be able to…
- explain what a neural network learns
- describe attention without hiding behind the word attention
- explain tokens, embeddings, next-token prediction, and context windows
- know why transformers replaced earlier sequence models
Practice with an agent
Test what a model actually knows
beginner · codex or claude, markdown, screenshots
Make
a short evidence log of five controlled model probes, including one confident error and one context-window failure
You know it works when
show the input, output, claim you checked, external evidence, and what you would trust differently next time
Go deeper by building
Build a tiny autocomplete model
intermediate · python, pytorch or tinygrad, a small text corpus
Make
a notebook or script that trains a tiny character-level model and samples text
You know it works when
show training loss, sample output, and a short note on what improved after tuning