The trap isn't only assuming few active params means fast. It's that plus ranking models first and runtimes second, when the runtime is what decides whether expert sparsity ever hits the silicon. Wrong kernels still walk the dense path, and you pay for experts that never fire on that hardware.
Andrew
The trap isn't only assuming few active params means fast. It's that plus ranking models first and runtimes second, when the runtime is what decides whether expert sparsity ever hits the silicon. Wrong kernels still walk the dense path, and you pay for experts that never fire on that hardware.