Blockwise FP8 on Ada GPUs, part 2: my kernel was correct and 4x too slow
How I found register spills in the main loop, fixed them, and got a 1.42x end-to-end speedup on an L4
Written by
C++/CUDA Engineer · LLM Inference & Systems
C++/CUDA engineer building LLM inference and GPU code from scratch: verbum.cpp, Lattice, RAAG. 2 CUDA PRs merged into llama.cpp. I write about inference, GPU performance, and measuring things honestly.
Responses
Markdown supported · ⌘↵ to post
No responses yet.