Teaching Llama 3 to be PoliteSpent ~$3 on A100 (80GB) instance on RunPod8 min read·blog.ifkash.devFeb 25
I Pretrained a 360M LLaMA-Style Language Model from Scratch on 6B FineWeb Tokens (Single H100)Explore pretraining a 360M LLaMA-style language model on 6B tokens using a single H100 GPU, with reproducible results5 min read·blog.ifkash.devFeb 8
From Karpathy's micrograd to smoltorch: Understanding Autograd from First PrinciplesWhy I Built My Own Deep Learning Framework? After watching Andrej Karpathy’s micrograd lecture, I had a realization: I’d been using PyTorch for months, but I didn’t really understand how autograd worked. Sure, I could call .backward() and get gradien...7 min read·blog.ifkash.devNov 18, 2025
I Built a Tiny Vector Database (and Pointed It at FAISS)Vector DBs are everywhere these days: Pinecone, Weaviate, Qdrant, Chroma, FAISS … you name it! Most of them are full-featured systems with servers, APIs, dashboards, the works. Sometimes the best way to demystify hype is to build it yourself. Here’s ...4 min read·blog.ifkash.devAug 28, 2025
The instability of a softmax functionExplains the numerical instability of the softmax function and how to fix it using vector transformations2 min read·blog.ifkash.devMar 5, 2025