GPU Utilization Is a Counter, Not a Cause
nvidia-smi reads 97% the entire window. The red gaps in the cause-side timeline are the throughput the GPU lost while the counter sat green.
TL;DR
A vLLM server reads 97% GPU utilization on nvidia-smi
ingero.hashnode.dev10 min read