[CUDA in Practice] SGEMM — Beating cuBLAS: A Deep Dive into Peak-Performance Matrix Multiplication in Pure CUDA C++A Deep Dive into Peak-Performance Matrix Multiplication in Pure CUDA C++27 min read·wingedge777.hashnode.devMay 29
[CUDA in Practice] RoPE — Why Kernel Fusion in Hand-Written Operators Matters: Reducing Memory Traffic and Launch OverheadAI compilers are evolving fast. In many cases, torch.compile plus JIT optimization in PyTorch can deliver striking speedups, to the point that people often say, "hand-written operators are no longer n8 min read·wingedge777.hashnode.dev1May 24
[CUDA in Practice] Matrix Transpose — From Padding to XOR Swizzle: The Art of Shared Memory Access OptimizationNote: Text translated by AI. Code crafted by human. Matrix transpose is one of the most fundamental operations in deep learning and high-performance computing. The deceptively simple coordinate swap33 min read·wingedge777.hashnode.devApr 9
A Deep Dive into DeviceQuery: Understanding Your GPU Hardware0. Preface Before writing a single line of high-performance CUDA code, you must know your silicon. deviceQuery is often the first command a developer runs, yet its output is usually ignored. This post9 min read·wingedge777.hashnode.devApr 5