Viru Jade

I Built a Tokenizer From Scratch. Here Is What I Learned.

I am learning AI engineering. Not from a degree. Not from a bootcamp. Just me, Python, and a lot of confusion. While studying how Transformers work, I kept reading the word tokenization everywhere. Ev