{"record":{"id":"2293c53eaa9d7b87","repo":"vllm-project/vllm","slug":"deepepv2-requires-nccl-gin-gpu-initiated-networki","errorCode":null,"errorMessage":"DeepEPv2 requires NCCL GIN (GPU-Initiated Networking). This usually means IBGDA-capable InfiniBand NICs or drivers are not available. See tools/ep_kernels/README.md for requirements.","messagePattern":"DeepEPv2 requires NCCL GIN \\(GPU-Initiated Networking\\)\\. This usually means IBGDA-capable InfiniBand NICs or drivers are not available\\. See tools/ep_kernels/README\\.md for requirements\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"vllm/distributed/device_communicators/all2all.py","lineNumber":1059,"sourceCode":"        )\n\n    def _check_gin_support(self, group) -> None:\n        from vllm.utils.nccl import query_nccl_gin_type\n\n        # ProcessGroupNCCL creates communicators lazily. Initialize this exact\n        # group before querying so a null comm pointer is not mistaken for\n        # missing GIN support.\n        probe = torch.zeros(1, device=\"cuda\")\n        torch.distributed.all_reduce(probe, group=group)\n\n        gin_type = query_nccl_gin_type(group)\n        if gin_type is None:\n            raise RuntimeError(\n                \"DeepEPv2 communicator properties query failed; \"\n                \"networking capability could not be determined.\"\n            )\n        if gin_type == 0:\n            raise RuntimeError(\n                \"DeepEPv2 requires NCCL GIN (GPU-Initiated Networking). \"\n                \"This usually means IBGDA-capable InfiniBand NICs or drivers \"\n                \"are not available. See tools/ep_kernels/README.md for \"\n                \"requirements.\"\n            )\n\n    def get_handle(self, kwargs):\n        import deep_ep  # type: ignore[import-not-found]\n\n        num_experts = kwargs.pop(\"num_experts\", 256)\n        buffer_kwargs = self._make_all2all_kwargs(**kwargs)\n        if not self._gin_checked:\n            self._check_gin_support(buffer_kwargs[\"group\"])\n            self._gin_checked = True\n        logger.debug(\"DeepEP v2 all2all args %s\", buffer_kwargs)\n        handle: deep_ep.ElasticBuffer = self.handle_cache.get_or_create(\n            buffer_kwargs, deep_ep.ElasticBuffer\n        )","sourceCodeStart":1041,"sourceCodeEnd":1077,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/distributed/device_communicators/all2all.py#L1041-L1077","documentation":"Raised by _check_gin_support when query_nccl_gin_type reports GIN type 0: the NCCL communicator exists and was queried successfully, but GPU-Initiated Networking is not enabled. DeepEPv2's low-latency kernels drive the NIC from the GPU and require IBGDA-capable InfiniBand (or equivalent GIN support); without it vLLM aborts with a pointer to tools/ep_kernels/README.md.","triggerScenarios":"Enabling the DeepEPv2 all2all backend on a node whose NCCL was built/run without GIN — no IBGDA-capable NICs, old OFED drivers, or NCCL_IB_GDR / RELAXED_CUDA_MEM_OPS-style GIN prerequisites not met.","commonSituations":"Running DeepEP/EP workloads on Ethernet-only or virtualized clusters; RDMA stack present but OFED too old for IBGDA; NCCL built without CUDA/GDR support; GPU lacks GPUDirect (e.g. consumer GPUs).","solutions":["Follow tools/ep_kernels/README.md in the vLLM repo: install IBGDA-capable InfiniBand NICs/drivers and a NCCL build with GIN support.","Verify with a minimal NCCL GIN/IBGDA test before launching vLLM (nccl_ibgda_test or the repo's EP kernel tools).","If the hardware cannot provide GIN, select a non-DeepEPv2 all2all backend for this cluster."],"exampleFix":"# before: DeepEPv2 selected on an Ethernet-only node -> RuntimeError\n# after\nVLLM_ALL2ALL_BACKEND=<non-deepep backend>  # or install IBGDA NICs + GIN-enabled NCCL per tools/ep_kernels/README.md","handlingStrategy":"validation","validationCode":"from vllm.utils.nccl import query_nccl_gin_type\nimport torch\n\nprobe = torch.zeros(1, device=\"cuda\")\ntorch.distributed.all_reduce(probe, group=gpu_group)\ngin = query_nccl_gin_type(gpu_group)\nif gin == 0:\n    raise SystemExit(\"cluster lacks NCCL GIN/IBGDA; DeepEPv2 unavailable — see tools/ep_kernels/README.md\")","typeGuard":"def gin_supported(group) -> bool:\n    from vllm.utils.nccl import query_nccl_gin_type\n    probe = torch.zeros(1, device=\"cuda\")\n    torch.distributed.all_reduce(probe, group=group)\n    gin = query_nccl_gin_type(group)\n    return gin is not None and gin != 0","tryCatchPattern":"try:\n    manager = DeepEPv2Manager(gpu_group)\nexcept RuntimeError as e:\n    if \"requires NCCL GIN\" in str(e):\n        select_non_deepep_backend_and_relaunch()\n    raise","preventionTips":["Validate IBGDA/GIN with the tools under tools/ep_kernels/ during cluster bring-up.","Keep OFED drivers and NCCL current enough for IBGDA; document the required minimum per site.","Do not schedule DeepEPv2 jobs onto nodes without InfiniBand GIN support."],"tags":["vllm","deepep","nccl","ibgda","infiniband","distributed"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}