JJahninconatusai.hashnode.dev·Aug 26 · 3 min readSGLang outputs endless repetition on NVFP4 models: the FP8 lm_head bugYour NVFP4 model serves fine on vLLM but outputs an endlessly repeated phrase on SGLang, from the very first token, even on a trivial prompt. The response content comes back empty, every request ends 00
JJahninconatusai.hashnode.dev·Aug 25 · 4 min readThe sm_120 shared-memory cliff: why FP8 KV cache crashes vLLM on workstation BlackwellIf you run vLLM with `--kv-cache-dtype fp8` on a DeepSeek-family (MLA) model and your GPU is a GB10, an RTX PRO 6000, or any workstation or consumer Blackwell card, there is a decent chance the engine00
