Question: Layering of various Communication libraries

[alokprasad]

Can you describe layering of the various backend (ncclx,nccl,gloo) and integration with
ctran or ncclx

Is it that Ctran is abstraction over NCCL, RCCL , MCCL. Then what is NCCLX and RCCLX ?
Can i assume it is structure like below

CTRAN
↓
XCCLX (e.g., NCCLX, RCCLX)
↓
XCCL (e.g., NCCL, RCCL, MSCCL, Gloo)

[d4l3k]

It's the opposite -- CTran is the low level transport library used by NCCLX.

NCCLX can either use NCCL or CTran collectives. Torchcomms is the abstraction layer that allows you to use different backends.

torchcomms -> NCCLX -> CTran
torchcomms -> RCCLX -> CTran

[alokprasad]

@d4l3k what is MCCL here>?

[d4l3k]

MCCL -> CTran

It's another library built on CTran with a focus on fault tolerance. There's some tweaks to make the kernels more reliable when errors occur as compared to the perf tuned NCCL/NCCLX kernels