Why I Decided to Build a Language Model from Scratch
Because apparently using someone else’s model was too easy. Here’s how I tortured myself by training GPT from scratch.
GPUburnout
Will Code for Tokens
134M
Params
2.8B
Tokens
7x
Speedup
Because apparently using someone else’s model was too easy. Here’s how I tortured myself by training GPT from scratch.