The runtime contiguity check is a clever trade: a few integer comparisons are cheap relative to the roughly 500-cycle memory access they unlock. I would be curious to see the cost split for aligned interior blocks versus masked tail blocks, since real workloads often have awkward dimensions rather than clean powers of two. If the check is warp-uniform, it should avoid divergence, but shape-distribution benchmarks would show how often the scalar fallback becomes visible.
On the cost split: the check is a handful of integer compares per 16-byte group, so on an aligned interior block it is pure overhead in front of the vector load, a few ALU ops hidden under the memory latency; on a masked tail the affected groups drop to per-lane predicated scalar. I have not broken that into a clean fast-path-versus-fallback microbenchmark, and you are right that a shape-distribution sweep is the honest way to show it.
The one data point I can offer in that direction: the softmax and layernorm examples deliberately use non-power-of-two row lengths (2500 and 1500), so every row carries a masked tail rather than a single tail block for the whole launch, and they still sit at the memory-bandwidth ceiling next to Triton. So at least for bandwidth-bound kernels the fallback stays hidden behind the DRAM access even when it fires on every row. Isolating the two paths across a spread of awkward shapes is a good addition and something I would like to run properly.