vllm-project/vllm · critical · RuntimeError

DeepEPv2 requires NCCL GIN (GPU-Initiated Networking). This

Error message

DeepEPv2 requires NCCL GIN (GPU-Initiated Networking). This usually means IBGDA-capable InfiniBand NICs or drivers are not available. See tools/ep_kernels/README.md for requirements.

What it means

Raised by _check_gin_support when query_nccl_gin_type reports GIN type 0: the NCCL communicator exists and was queried successfully, but GPU-Initiated Networking is not enabled. DeepEPv2's low-latency kernels drive the NIC from the GPU and require IBGDA-capable InfiniBand (or equivalent GIN support); without it vLLM aborts with a pointer to tools/ep_kernels/README.md.

Source

Thrown at vllm/distributed/device_communicators/all2all.py:1059

        )

    def _check_gin_support(self, group) -> None:
        from vllm.utils.nccl import query_nccl_gin_type

        # ProcessGroupNCCL creates communicators lazily. Initialize this exact
        # group before querying so a null comm pointer is not mistaken for
        # missing GIN support.
        probe = torch.zeros(1, device="cuda")
        torch.distributed.all_reduce(probe, group=group)

        gin_type = query_nccl_gin_type(group)
        if gin_type is None:
            raise RuntimeError(
                "DeepEPv2 communicator properties query failed; "
                "networking capability could not be determined."
            )
        if gin_type == 0:
            raise RuntimeError(
                "DeepEPv2 requires NCCL GIN (GPU-Initiated Networking). "
                "This usually means IBGDA-capable InfiniBand NICs or drivers "
                "are not available. See tools/ep_kernels/README.md for "
                "requirements."
            )

    def get_handle(self, kwargs):
        import deep_ep  # type: ignore[import-not-found]

        num_experts = kwargs.pop("num_experts", 256)
        buffer_kwargs = self._make_all2all_kwargs(**kwargs)
        if not self._gin_checked:
            self._check_gin_support(buffer_kwargs["group"])
            self._gin_checked = True
        logger.debug("DeepEP v2 all2all args %s", buffer_kwargs)
        handle: deep_ep.ElasticBuffer = self.handle_cache.get_or_create(
            buffer_kwargs, deep_ep.ElasticBuffer
        )

View on GitHub (pinned to c794754062)

Solutions

1. Follow tools/ep_kernels/README.md in the vLLM repo: install IBGDA-capable InfiniBand NICs/drivers and a NCCL build with GIN support.
2. Verify with a minimal NCCL GIN/IBGDA test before launching vLLM (nccl_ibgda_test or the repo's EP kernel tools).
3. If the hardware cannot provide GIN, select a non-DeepEPv2 all2all backend for this cluster.

Example fix

# before: DeepEPv2 selected on an Ethernet-only node -> RuntimeError
# after
VLLM_ALL2ALL_BACKEND=<non-deepep backend>  # or install IBGDA NICs + GIN-enabled NCCL per tools/ep_kernels/README.md

from vllm.utils.nccl import query_nccl_gin_type
import torch

probe = torch.zeros(1, device="cuda")
torch.distributed.all_reduce(probe, group=gpu_group)
gin = query_nccl_gin_type(gpu_group)
if gin == 0:
    raise SystemExit("cluster lacks NCCL GIN/IBGDA; DeepEPv2 unavailable — see tools/ep_kernels/README.md")

def gin_supported(group) -> bool:
    from vllm.utils.nccl import query_nccl_gin_type
    probe = torch.zeros(1, device="cuda")
    torch.distributed.all_reduce(probe, group=group)
    gin = query_nccl_gin_type(group)
    return gin is not None and gin != 0

try:
    manager = DeepEPv2Manager(gpu_group)
except RuntimeError as e:
    if "requires NCCL GIN" in str(e):
        select_non_deepep_backend_and_relaunch()
    raise

When it happens

Trigger: Enabling the DeepEPv2 all2all backend on a node whose NCCL was built/run without GIN — no IBGDA-capable NICs, old OFED drivers, or NCCL_IB_GDR / RELAXED_CUDA_MEM_OPS-style GIN prerequisites not met.

Common situations: Running DeepEP/EP workloads on Ethernet-only or virtualized clusters; RDMA stack present but OFED too old for IBGDA; NCCL built without CUDA/GDR support; GPU lacks GPUDirect (e.g. consumer GPUs).

Related errors

• DeepEPv2 communicator properties query failed; networking ca
• Image generation should not fail
• cannot continue the final message when the last message is n
• {self.communicator} communicator is incompatible with async
• Unknown all2all backend: {self.all2all_backend}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/2293c53eaa9d7b87. Report an issue: GitHub.