STSakshi Tyagiinsakshityagi.hashnode.dev·11h ago · 6 min readThe Inference Wall: Where LLM Latency and Cost GoPart 1 of 4 : Serving LLMs in Production. Here's a thing that surprises almost every team the first time they ship an LLM: the model that felt fast in the demo gets expensive and sluggish in productio00
STSakshi Tyagiinsakshityagi.hashnode.dev·10h ago · 5 min readSelf-Host vs. API for LLMs, Actually CostedPart 4 of 4 Serving LLMs in Production. The first three parts were about making inference fast: where the cost actually hides, batching to fill the GPU, and hauling fewer bytes per word. All of it fee00
STSakshi Tyagiinsakshityagi.hashnode.dev·10h ago · 6 min readMaking LLM Decode Cheaper: Quantization and MorePart 3 of 4 Serving LLMs in Production. Part 2 filled the idle GPU by serving many users at once. But we still haven't touched the core problem from Part 1: every single word the model writes drags th00
STSakshi Tyagiinsakshityagi.hashnode.dev·10h ago · 6 min readBatching and the Idle GPU: Serving LLMs FasterPart 2 of 4 Serving LLMs in Production. Part 1 left us with an uncomfortable fact: when the model writes its answer the slow, one-word-at-a-time decode phase the GPU's powerful math units sit mostly i00
AAAhmed Adawyinadawy.hashnode.dev·2d ago · 3 min readPractical Python Implementation: Simulating Lattice-Based Key Generation--- --- Understanding the mathematical foundation behind post-quantum cryptography using a simple Python implementation. --- Why Lattice Cryptography? Quantum computers threaten many of today's public00
WKWesam Khallafinpytorchfromgroundup.hashnode.dev·3d ago · 7 min readWhat Does `keepdim` Do in PyTorch? (And the Silent Bug When You Forget It)You wrote x.sum(dim=1) to get a row total, divided the original tensor by it, and either got RuntimeError: The size of tensor a (3) must match the size of tensor b (2) — or, worse, got no error at all00
TTharunintharunpandya.hashnode.dev·6d ago · 30 min readEigenvalues and Eigenvectors, Explained for Complete Beginners — The Final Blog in the Linear Algebra for ML SeriesTake a normal photograph and stretch it sideways in an editing app — pull the left and right edges apart, like taffy. Watch what happens to different arrows you could draw on that photo. An arrow poin00
TTharunintharunpandya.hashnode.dev·Jul 22 · 25 min readMatrix Inverse, Determinant & Linear Transformation — What Actually Happens Inside a Machine Learning Model When It "Learns"A few months ago, I locked myself out of my own apartment. I stood outside, called a locksmith, and watched him work. He looked at the lock, cut a fresh key, turned it, and — click — the door opened. 10
AJAman Jainincurious-pm.hashnode.dev·Jul 22 · 10 min readTemperature, Top-k, and Top-p: How Decoding Changes What a GPT Model Says TL;DR A GPT model does not directly choose its next word. More precisely, it assigns probabilities to thousands of possible next tokens. Decoding is the step that converts those probabilities into an 00
HHHao Huainh2c-blogs.hashnode.dev·Jul 21 · 6 min readConceptual Training and Inference Workflow of a Deep Learning Model for Supervised Learning TaskTL;DR This page conceptually describes the inference workflow of a Deep Learning Model, wherein a pretrained Deep Learning Model may receive an input for the inference of an output according to the p00