SShubham Tatvamasi·2d agovLLM Deep Dive: How PagedAttention, Continuous Batching and the V1 Engine Serve LLMs in ProductionEvery layer above it routes, scales and governs requests. vLLM is the layer that turns GPU memory into tokens.52 min read·blog.shubhamtatvamasi.com63CKS
DDmytro Nasyrov·2d agoWhen Does Self-Hosted Inference Beat a Managed API? A 12-Month Operating ModelA fictional workload ledger with capacity steps, usable-throughput assumptions and an unpriced-operations register.19 min read·pharos-production.hashnode.dev
NNoah Grinstein·2d agoWhy Kubernetes Is Hard for AI Infrastructure Teams: The Biggest Operational Pain Points and How to Reduce ThemWhy Kubernetes is hard for AI infrastructure teams: GPU scheduling, drivers, model storage, and serving pain points, and how to reduce them.19 min read·controlplane-eng.hashnode.dev1O
JJakson Tate·5d agoDeepSeek DeepInfer: Production Inference Architecture, DSpark Optimization & Bare-Metal GPU DeploymentAn SRE guide to DeepSeek-V4.1-Flash inference, SWA bounded replay, speculative decoding, and production serving requirements.6 min read·servermo.hashnode.dev
JJakson Tate·5d agoMegatron-DeepSpeed 3D Parallelism: Scaling 100B+ LLM TrainingA production-grade SRE guide to configuring TP, PP, DP, ZeRO optimizations, and NCCL RoCEv2 networks on bare-metal GPU clusters.9 min read·servermo.hashnode.dev
JJakson Tate·5d agoTensorRT-LLM Production Deployment: FP8, NVFP4, MoE & Disaggregated ServingAn SRE blueprint for PyTorch backend selection, quantization, MoE parallelism, NIXL disaggregated serving, and bare-metal GPU infrastructure.8 min read·servermo.hashnode.dev
DDmytro Nasyrov·Nov 1Five Model-Routing Architectures for Cost-Controlled AI Systems ComparedOne comparison matrix and a worked budget ledger for choosing how each AI request may spend, escalate or stop.17 min read·pharos-production.hashnode.dev1
GGeorge Ezejiofor·Oct 29Canary Releases for an ML Model on KServe: Replayed Traffic, Sidecar Metrics and Automatic RollbackCanary Releases for an ML Model on KServe: Replayed Traffic, Sidecar Metrics and Automatic Rollback23 min read·georgeezejiofor.com
PPrem Choudhary·Oct 28Argocd & ARC-github-runner on EKS with TerraformArgocd & ARC-github-runner on EKS with Terraform8 min read·prem14choudhary.hashnode.dev
PPrem Choudhary·Oct 28Observability Stack on EKS via TERRAFORM....!!Observability Stack on EKS via TERRAFORM11 min read·prem14choudhary.hashnode.dev
