---
title: "Discussion on \"Inside NVIDIA Dynamo: A Deep Technical Tour of the Datacenter-Scale Inference Stack\" | Hashnode"
description: "Discussion on \"Inside NVIDIA Dynamo: A Deep Technical Tour of the Datacenter-Scale Inference Stack\". Disaggregated prefill/decode, KV-aware routing, NIXL, KVBM, the SLA Planner, and the Kubernetes machinery behind them"
url: "https://hashnode.com/posts/inside-nvidia-dynamo-deep-dive/6afc4f47fb30ba10dd112f43"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Inside NVIDIA Dynamo: A Deep Technical Tour of the Datacenter-Scale Inference Stack](https://blog.shubhamtatvamasi.com/inside-nvidia-dynamo-deep-dive?utm_source=hashnode&utm_medium=feed)

Disaggregated prefill/decode, KV-aware routing, NIXL, KVBM, the SLA Planner, and the Kubernetes machinery behind them

10h ago · 29 min read · blog.shubhamtatvamasi.com  
Read the article

-   [#nvidia](https://hashnode.com/tag/nvidia)
-   [#llm](https://hashnode.com/tag/llm)
-   [#kubernetes](https://hashnode.com/tag/kubernetes)
-   [#distributed-systems](https://hashnode.com/tag/distributed-systems)
-   [#machine-learning](https://hashnode.com/tag/machine-learning)

## Written by

[Shubham Tatvamasi](https://hashnode.com/@ShubhamTatvamasi)

@ShubhamTatvamasi

  
[View profile](https://hashnode.com/@ShubhamTatvamasi)

AI infrastructure engineer. Inference, GPUs and GitOps on Kubernetes.

## Responses 1

Markdown supported · ⌘↵ to post

[SOUMADIP PATRA](https://hashnode.com/@soumacloud)

[40m ago](https://hashnode.com/posts/inside-nvidia-dynamo-deep-dive/6afc4f47fb30ba10dd112f43/comment/6afcdbf2f440f6004fa0943a)

A good read, I think. NVIDIA is doing a lot with Dynamo, and reducing cold-start times is great. However, I think wasted compute and idle CPU resources are still open challenges.
