Move every gradient. Avoid one central bottleneck.

Watch workers exchange gradient chunks around a ring, then change worker count, payload size, bandwidth, and link latency to see when distributed training scales.

Collective timeline

Ring steps14
Communication0 ms
Bytes per worker0 MB
Parallel efficiency0%

Why the ring stays bandwidth-efficient.

Reduce-scatter

Workers circulate chunks and add partial gradients. After N-1 steps, each worker owns one fully reduced chunk.

All-gather

The reduced chunks circulate for another N-1 steps until every worker reconstructs the complete gradient.

Latency still accumulates

Each phase needs N-1 synchronized rounds. Small payloads or many workers can become latency-bound.

Primary sources

Bandwidth optimal all-reduce algorithms

NCCL collectives

PyTorch distributed