Qwen3.8 27B at 256K: 50 TPS on a 24 GB GPU
The quantized model from this write-up is public on Hugging Face: cdiamond/Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF - GGUF, F16 vision projector, SHA-256 sums, tensor-type map and the pinned llama.cpp patch
piszczek.hashnode.dev14 min read