Inside NVIDIA Dynamo: A Deep Technical Tour of the Datacenter-Scale Inference StackDisaggregated prefill/decode, KV-aware routing, NIXL, KVBM, the SLA Planner, and the Kubernetes machinery behind them29 min read·blog.shubhamtatvamasi.com62SS4d ago
vLLM Deep Dive: How PagedAttention, Continuous Batching and the V1 Engine Serve LLMs in ProductionEvery layer above it routes, scales and governs requests. vLLM is the layer that turns GPU memory into tokens.52 min read·blog.shubhamtatvamasi.com63CKSNov 9
KAI Scheduler Deep Dive: How NVIDIA Schedules GPUs on KubernetesQueues, gang scheduling, fractional GPUs and topology-aware placement: how NVIDIA’s open-source scheduler shares GPUs35 min read·blog.shubhamtatvamasi.com23AAOct 22
NVIDIA Dynamo vs KServe: Which Layer Should Own Your LLM Inference?Cache-aware routing, prefill/decode disaggregation and autoscaling compared, and how to choose per model pool13 min read·blog.shubhamtatvamasi.com1Oct 19
Running LLMs on Kubernetes in Production: KServe, vLLM, llm-d, Envoy AI Gateway, and LiteLLMOne OpenAI-compatible endpoint, model-aware routing, GPU-aware scheduling, and scaling driven by inference pressure, not CPU26 min read·blog.shubhamtatvamasi.com2Oct 11