--- title: "#inference articles | Hashnode" description: "Explore 127 articles tagged #inference on Hashnode. Tutorials, guides, and how-tos from the community." url: "https://hashnode.com/tag/inference" --- Tag # #inference 127 posts · 0 followers [Admin Osmexo](https://hashnode.com/@osmexo) · [13h ago](https://hashnode.com/posts/building-with-ai-is-easy-paying-for-it-at-scale-is-not-welcome-to-the-token-economy/6af445dacb6c9b876ec1d6e6) ### [Building With AI Is Easy. Paying for It at Scale Is Not: Welcome to the Token Economy](https://osmexo.hashnode.dev/building-with-ai-is-easy-paying-for-it-at-scale-is-not-welcome-to-the-token-economy?utm_source=hashnode&utm_medium=feed) An estimated 500T+ LLM tokens a week, what they cost a real app, and how to pay less per token 7 min read · [osmexo.hashnode.dev](https://osmexo.hashnode.dev/building-with-ai-is-easy-paying-for-it-at-scale-is-not-welcome-to-the-token-economy?utm_source=hashnode&utm_medium=feed) [Pratyush Ranjan](https://hashnode.com/@pratyushranjan02) · [4d ago](https://hashnode.com/posts/paged-attention-in-ai-inference/6aeed0a74f4b146adf854c72) ### [Paged Attention in AI Inference](https://ai-inference-pratyush.hashnode.dev/paged-attention-in-ai-inference?utm_source=hashnode&utm_medium=feed) Learn how paged attention cuts KV cache waste. Build a mini version in PyTorch and benchmark memory use against a contiguous cache. 9 min read · [ai-inference-pratyush.hashnode.dev](https://ai-inference-pratyush.hashnode.dev/paged-attention-in-ai-inference?utm_source=hashnode&utm_medium=feed) [1](https://hashnode.com/posts/paged-attention-in-ai-inference/6aeed0a74f4b146adf854c72) [Robert Collins](https://hashnode.com/@robbiemu) · [5d ago](https://hashnode.com/posts/from-numerical-differences-to-bitwise-prediction-in-core-ml-arithmetic/6aee002668becd2cfcbf0974) ### [From Numerical Differences to Bitwise Prediction in Core ML Arithmetic](https://selfenrichment.hashnode.dev/from-numerical-differences-to-bitwise-prediction-in-core-ml-arithmetic?utm_source=hashnode&utm_medium=feed) How targeted probes recover Core ML arithmetic rules and predict outputs bit for bit across tested CPU, GPU, and Neural Engine paths. 77 min read · [selfenrichment.hashnode.dev](https://selfenrichment.hashnode.dev/from-numerical-differences-to-bitwise-prediction-in-core-ml-arithmetic?utm_source=hashnode&utm_medium=feed) [Avneesh Jadhav](https://hashnode.com/@avneeshjadhav) · [Nov 4](https://hashnode.com/posts/self-hosting-qwen-3-8-27b/6aeb33a7f824903c7be5643a) ### [Self-hosting Qwen 3.8 27B](https://avneeshjadhav.hashnode.dev/self-hosting-qwen-3-8-27b?utm_source=hashnode&utm_medium=feed) Self-hosted Qwen 3.8 27B on a GPU pod with 2x Nvidia RTX 3090 GPUs and vLLM. 1 min read · [avneeshjadhav.hashnode.dev](https://avneeshjadhav.hashnode.dev/self-hosting-qwen-3-8-27b?utm_source=hashnode&utm_medium=feed) [Damiano Bianda](https://hashnode.com/@damianobianda) · [Oct 27](https://hashnode.com/posts/one-token-leaked-155-ai-responses-inside-cloudflare-s-glm-5-3-kv-cache-exposure/6ae0756515d6ef68f24d9d3c) ### [One Token Leaked 155 AI Responses: Inside Cloudflare's GLM-5.3 KV Cache Exposure](https://damianobianda.hashnode.dev/one-token-leaked-155-ai-responses-inside-cloudflare-s-glm-5-3-kv-cache-exposure?utm_source=hashnode&utm_medium=feed) A one-token placeholder plus a KV pool misconfig leaked 155 AI responses across Cloudflare tenants. The bug, the fix, and the lessons. 8 min read · [damianobianda.hashnode.dev](https://damianobianda.hashnode.dev/one-token-leaked-155-ai-responses-inside-cloudflare-s-glm-5-3-kv-cache-exposure?utm_source=hashnode&utm_medium=feed) [Pratyush Ranjan](https://hashnode.com/@pratyushranjan02) · [Oct 26](https://hashnode.com/posts/kv-cache-optimisation-in-ai-inference/6adf5f5e5bb6fa9a24a51c06) ### [KV Cache Optimisation in AI Inference](https://ai-inference-pratyush.hashnode.dev/kv-cache-optimisation-in-ai-inference?utm_source=hashnode&utm_medium=feed) Learn how the KV cache speeds up LLM inference. We build a small GPT in PyTorch and benchmark cached vs uncached generation: a 4.5x speedup. 8 min read · [ai-inference-pratyush.hashnode.dev](https://ai-inference-pratyush.hashnode.dev/kv-cache-optimisation-in-ai-inference?utm_source=hashnode&utm_medium=feed) [1](https://hashnode.com/posts/kv-cache-optimisation-in-ai-inference/6adf5f5e5bb6fa9a24a51c06) [Adi Shik](https://hashnode.com/@adishik) · [Oct 25](https://hashnode.com/posts/tensward-benchmarks/6addd87f018bf08f1a7a03c6) ### [What a 1997 chess engine taught me about LLM inference](https://openingthehood.hashnode.dev/tensward-benchmarks?utm_source=hashnode&utm_medium=feed) TL;DR You don't need an LLM agent to optimize inference. Across six tuning tracks on an NVIDIA L4, Tensward's expert rules and heuristics matched frontier LLM agents running at maximum effort - Astra 10 min read · [openingthehood.hashnode.dev](https://openingthehood.hashnode.dev/tensward-benchmarks?utm_source=hashnode&utm_medium=feed) [Rounak Show](https://hashnode.com/@ShowRounak) · [Oct 23](https://hashnode.com/posts/prefill-and-decode-in-llm/6adbcdc33be84bb709241848) ### [The Two Lives of an LLM Request: Prefill and Decode](https://rounakshow.hashnode.dev/prefill-and-decode-in-llm?utm_source=hashnode&utm_medium=feed) The strange pause Okay, so lets say, you open a chat app and you paste a long document. Then, you ask, "Hey, can you summarize this?" You hit enter. Nothing happens. One second. Two seconds. Maybe thr 14 min read · [rounakshow.hashnode.dev](https://rounakshow.hashnode.dev/prefill-and-decode-in-llm?utm_source=hashnode&utm_medium=feed) [Sachin Rajput](https://hashnode.com/@sachinrajput) · [Oct 22](https://hashnode.com/posts/one-request-end-to-end-tokenize-prefill-decode-stream/6ad9e47163fdbe14d18a20a6) ### [One Request, End to End: Tokenize, Prefill, Decode, Stream](https://sachinrajput.hashnode.dev/one-request-end-to-end-tokenize-prefill-decode-stream?utm_source=hashnode&utm_medium=feed) What really happens in the second after you ask a chatbot a question? A trip through the Word Factory, explained simply. 16 min read · [sachinrajput.hashnode.dev](https://sachinrajput.hashnode.dev/one-request-end-to-end-tokenize-prefill-decode-stream?utm_source=hashnode&utm_medium=feed) [RONI DAS](https://hashnode.com/@ronidas) · [Oct 17](https://hashnode.com/posts/why-did-one-way-of-splitting-metas-816b-model-run-59-faster-on-the-same-two-macs/6ad3493be282d6003117a7ed) ### [Why did one way of splitting Meta's 816B model run 59% faster on the same two Macs?](https://systemdesignacademy.hashnode.dev/why-did-one-way-of-splitting-metas-816b-model-run-59-faster-on-the-same-two-macs?utm_source=hashnode&utm_medium=feed) Apple ran Muse Spark Open across Mac Studios. Split by layers, it decoded 15.74 tokens a second. Split inside each layer, 24.99. The reason is the parallelism trade-off interviewers ask about. 8 min read · [systemdesignacademy.hashnode.dev](https://systemdesignacademy.hashnode.dev/why-did-one-way-of-splitting-metas-816b-model-run-59-faster-on-the-same-two-macs?utm_source=hashnode&utm_medium=feed)