Search posts, tags, users, and pages
Disaggregated prefill/decode, KV-aware routing, NIXL, KVBM, the SLA Planner, and the Kubernetes machinery behind them
Shubham Tatvamasi
@ShubhamTatvamasi
AI infrastructure engineer. Inference, GPUs and GitOps on Kubernetes.
Markdown supported · ⌘↵ to post
SOUMADIP PATRA
A good read, I think. NVIDIA is doing a lot with Dynamo, and reducing cold-start times is great. However, I think wasted compute and idle CPU resources are still open challenges.
SOUMADIP PATRA
A good read, I think. NVIDIA is doing a lot with Dynamo, and reducing cold-start times is great. However, I think wasted compute and idle CPU resources are still open challenges.