[InfiniBand Optimization] Which of the following NCCL environment variables enable SHARP aggregation with NCCL when using the NCCL-SHARP plugin? Pick…

To enable NVIDIA SHARP (Scalable Hierarchical Aggregation and Reduction Protocol) with the NCCL-SHARP plugin, two key steps are required, controlled by separate environment variables. First, NCCLCOLLNETENABLE=1 must be set to activate the CollNet (Collective Network) plugin framework within NCCL, which is the mechanism that loads the SHARP plugin. Second, SHARP requires a resource group of participating nodes. The NCCLSHARPAUTOINIT=1 variable instructs the SHARP daemon (sharpd) to automatically create a default group for the job. Without both the plugin framework being enabled and a resource group being available, SHARP aggregation cannot be utilized.

B. NCCLALGO=CollNet: This variable forces NCCL to use the CollNet algorithm, bypassing its internal heuristics. While it ensures SHARP is used, it is not required for simply enabling it; NCCL can select it automatically.

C. NCCLSPECTRUMENABLE=1: This is not a valid NCCL environment variable. NVIDIA Spectrum is a brand of Ethernet switches, and this variable is unrelated to enabling the InfiniBand-based SHARP feature in NCCL.

1. NVIDIA NCCL Documentation (nccl-collnet-sharp plugin): The documentation for the SHARP plugin explicitly states the requirement for the CollNet framework to be enabled. "To enable SHARP plugin, the NCCLCOLLNETENABLE environment variable must be set to 1." This directly supports answer A.

Source: NVIDIA Corporation, nccl-collnet-sharp plugin README.md. This can be found within the HPC-X toolkit or NVIDIA NGC container documentation.

2. NVIDIA Developer Blog: An official blog post detailing the use of SHARP with DGX systems confirms the use of NCCLSHARPAUTOINIT. It states, "The environment variable NCCLSHARPAUTOINIT=1 can be set to automatically create a default SHARP group for the job." This is a necessary step for SHARP to function in many environments and supports answer D.

Source: NVIDIA Developer Blog, "Bringing HPC and AI Together with NVIDIA DGX-2," May 29, 2019.

3. NVIDIA Mellanox HPC-X User Manual: The user manual for the HPC-X toolkit, which includes SHARP, describes the architecture. It details that SHARP requires a running SHARP Aggregation Manager daemon (sharpd) and the creation of resource groups for jobs. The auto-initialization feature simplifies this process.

Source: NVIDIA HPC-X Toolkit User Manual, v2.17, Section 3.10.2.1, "Automatic SHARP Group Creation."