https://developer.nvidia.com/nccl

NVIDIA Collective Communications Library (NCCL)

NVIDIA NCCL 英伟达 NCCL

The NVIDIA Collective Communication Library (NCCL) implements multi-GPU and multi-node communication primitives optimized for NVIDIA GPUs and Networking. NCCL provides routines such as all-gather, all-reduce, broadcast, reduce, reduce-scatter as well as point-to-point send and receive that are optimized to achieve high bandwidth and low latency over PCIe and NVLink high-speed interconnects within a node and over NVIDIA Mellanox Network across nodes.

NVIDIA 集体通信库 (NCCL) 实现了针对 NVIDIA GPU 和网络进行优化的多 GPU 和多节点通信原语。 NCCL 提供全收集、全归约、广播、归约、归约分散以及点对点发送和接收等例程，这些例程经过优化，可在 PCIe 和 NVLink 高速互连上实现高带宽和低延迟。一个节点以及跨节点的 NVIDIA Mellanox 网络。