Why I Decided to Build a Language Model from Scratch
Because apparently using someone else’s model was too easy. Here’s how I tortured myself by training GPT from scratch.
GPUburnout
Will Code for Tokens
S1
GPT-2 134M
S2
Llama 1B
Because apparently using someone else’s model was too easy. Here’s how I tortured myself by training GPT from scratch.