Phase 07

Transformers Deep Dive

The architecture that changed everything. Understand every layer.

01Why Transformers — The Problems with RNNsLearn~45 minutes 02Self-Attention from ScratchBuild~90 minutesQuiz 03Multi-Head AttentionBuild~75 minutes 04Positional Encoding — Sinusoidal, RoPE, ALiBiBuild~45 minutes 05The Full Transformer — Encoder + DecoderBuild~75 minutes 06BERT — Masked Language ModelingBuild~45 minutes 07GPT — Causal Language ModelingBuild~75 minutes 08T5, BART — Encoder-Decoder ModelsLearn~45 minutes 09Vision Transformers (ViT)Build~45 minutes 10Audio Transformers — Whisper ArchitectureLearn~45 minutes 11Mixture of Experts (MoE)Build~45 minutes 12KV Cache, Flash Attention & Inference OptimizationBuild~75 minutes 13Scaling LawsLearn~45 minutes 14Build a Transformer from Scratch — The CapstoneBuild~120 minutes 15Attention Variants — Sliding Window, Sparse, DifferentialBuild~60 minutes 16Speculative Decoding — Draft, Verify, RepeatBuild~60 minutes 17Fitting Scaling Laws from Your Own RunsBuild~65 minutesQuiz

“Stay hungry, stay foolish.”

— Steve Jobs