Production GPU Training is 34% Slower. Show Me Why

A single slow GPU - a straggler - in a 1,000-node training cluster idles 999 healthy GPUs at every AllReduce barrier. The job does not crash. There is no error message. GPU stragglers just make traini