Discussion on "4 GPUs, 2x speed: why PyTorch DDP stalled on a PCIe-only 4x RTX A6000, and two flags that took it to 3.6x"

Runcrate · 2026-10-26T09:24:40.758Z

Four GPUs should train about four times faster than one. On a rented 4x RTX A6000 machine this morning (2026-10-26), plain PyTorch DDP gave me 2x. This is the notebook of finding out why, and the two

Discussion on "4 GPUs, 2x speed: why PyTorch DDP stalled on a PCIe-only 4x RTX A6000, and two flags that took it to 3.6x" | Hashnode

Search Hashnode

4 GPUs, 2x speed: why PyTorch DDP stalled on a PCIe-only 4x RTX A6000, and two flags that took it to 3.6x

Responses