I Built the Smallest LLM I Could to Understand How They Work

I use LLMs every day and had no real idea what happens inside one. So I built the smallest one I could: a transformer with roughly 108,000 parameters that learns to write Shakespeare one character at