Transformers & LLMs
The architecture behind ChatGPT, explained visually
Tokens, embeddings, attention, and training — the complete mental model of how large language models work.
Intermediate4 modules · 12 lessons~103 min1040 XPAligned with Hugging Face LLM CourseStart with: NLP FoundationsStart with: Deep Learning
Progress0/12
Module
From Words to Vectors
Tokenization and embeddings
Tokenization6 min · 60 XPTokenization
Attention Is All You Need10 min · 100 XPAttention Is All You Need
Embeddings: Meaning as Geometry8 min · 80 XPEmbeddings: Meaning as Geometry
Module
Inside the Transformer
Anatomy of the block: heads, positions, and why scale surprises us
The Full Block9 min · 90 XPThe Full Block
Multi-Head Attention8 min · 85 XPMulti-Head Attention
Positional Encoding8 min · 80 XPPositional Encoding
Scale Changes Everything10 min · 100 XPScale Changes Everything
Module
Training LLMs
Pretraining, fine-tuning, RLHF
The Next-Token Game8 min · 80 XPThe Next-Token Game
RLHF: Teaching Models Manners8 min · 85 XPRLHF: Teaching Models Manners
Module
Using LLMs
Decoding, context, and choosing between prompting and fine-tuning
Decoding Strategies9 min · 90 XPDecoding Strategies
Context Windows & Memory9 min · 90 XPContext Windows & Memory
Fine-Tuning vs Prompting10 min · 100 XPFine-Tuning vs Prompting
Capstone project
Attention explorer
Use a pretrained model to inspect attention patterns or embeddings on sentences you choose. Find one case where attention does something surprising and document it.
Deliverable: A notebook or write-up with attention/embedding visualizations and your interpretation.
Milestones
Finish every lesson to earn the Transformer Architecture Certificate.