Training Optimizations Deep Dive: How I Made the A100 Actually Work
The complete technical reference for achieving 16x speedup. Every optimization explained with code and diagrams.
Season-1
The Results Are In (And My Wallet Is Empty)
Final loss curves, the damage to my compute budget, and 22 lessons I paid dearly to learn.
11 Training Challenges and How I Solved Them
A comprehensive guide to every way I shot myself in the foot training GPT-2 Small. Learn from my pain.
Scaling Up: From Tiny Model to GPT-2 Small
How I went from ‘cute toy model’ to ‘134 million parameters that need an A100 to breathe.’
Data Preparation: Building a 12GB Training Corpus
How I built a 12GB ChatGPT-style conversational dataset and implemented BPE tokenization for efficient training.
Why I Decided to Build a Language Model from Scratch
Because apparently using someone else’s model was too easy. Here’s how I tortured myself by training GPT from scratch.
