⚠ Switch to EXCALIDRAW VIEW in the MORE OPTIONS menu of this document. ⚠ You can decompress Drawing data with the command palette: ‘Decompress current Excalidraw file’. For more info check in plugin settings under ‘Saving’
Excalidraw Data
Text Elements
NVLS
NVLS Tree
ncclFuncAllReduce
单节点
多节点
Scatter
Gather
reduce / broadcast
headRank
reduce(Send to network) & broadcast(Recv from nerwork)
headRank
Scatter
Gather
reduce(Send to network) & broadcast(Recv from nerwork)
reduce → NVSwitch → broadcast，一层完成。
每个 GPU 在通信树中有一个 parent (treeUp) 和多个 child (treeDown[])。
reduce从叶子向上流，broadcast从根向下流。