Data Preparation: Building a 12GB Training Corpus
Where I learned that 90% of ML is just cleaning data and crying about file sizes.
Scaling
Scaling Up: From Tiny Model to GPT-2 Small
How I went from ‘cute toy model’ to ‘134 million parameters that need an A100 to breathe.’
