sgl-project/sglang · error · ValueError

NCCL only supports CUDA, ROCm and MUSA backends.

Error message

NCCL only supports CUDA, ROCm and MUSA backends.

What it means

PyNcclWrapper needs to locate the NCCL shared library (libnccl.so.2 on CUDA, librccl.so.1 on ROCm/HIP, libmccl.so.2 on MUSA) before loading it via ctypes. find_nccl_library picks the file based on torch.version.cuda/hip/musa; if none is set, PyTorch was built for a different backend and the wrapper refuses with ValueError.

Source

Thrown at python/sglang/srt/distributed/device_communicators/pynccl_wrapper.py:63

    """

    # so_file can be set to None in sglang
    so_file = os.environ.get("SGLANG_NCCL_SO_PATH", None)

    # manually load the nccl library
    if so_file:
        logger.info(
            "Found nccl from environment variable SGLANG_NCCL_SO_PATH=%s", so_file
        )
    else:
        if torch.version.cuda is not None:
            so_file = "libnccl.so.2"
        elif torch.version.hip is not None:
            so_file = "librccl.so.1"
        elif hasattr(torch.version, "musa") and torch.version.musa is not None:
            so_file = "libmccl.so.2"
        else:
            raise ValueError("NCCL only supports CUDA, ROCm and MUSA backends.")
        logger.debug("Found nccl from library %s", so_file)
    return so_file


# === export types and functions from nccl to Python ===
# for the original nccl definition, please check
# https://github.com/NVIDIA/nccl/blob/master/src/nccl.h.in

ncclResult_t = ctypes.c_int
ncclComm_t = ctypes.c_void_p
ncclWindow_t = ctypes.c_void_p


# Sentinels NCCL uses to mark "use the default value", see
# NCCL_CONFIG_UNDEF_INT / NCCL_CONFIG_UNDEF_PTR / NCCL_API_MAGIC in nccl.h.in.
NCCL_CONFIG_UNDEF_INT = -(2**31)  # INT_MIN
NCCL_CONFIG_UNDEF_PTR = None
NCCL_API_MAGIC = 0xCAFEBEEF

View on GitHub (pinned to 0132848349)

Solutions

1. Install a CUDA (or ROCm/MUSA) build of PyTorch matching your system, e.g. pip install torch --index-url https://download.pytorch.org/whl/cu121
2. Verify the backend: python -c 'import torch; print(torch.version.cuda, torch.version.hip)' — one must be non-None
3. Ensure the corresponding NCCL library (libnccl.so.2 / librccl.so.1) is installed and on LD_LIBRARY_PATH
4. If CPU-only execution is intended, avoid the pynccl/custom-parallel code paths

Example fix

# before
pip list | grep torch  # torch (CPU-only build) -> ValueError

# after
pip uninstall torch
pip install torch --index-url https://download.pytorch.org/whl/cu121

import torch
backend = torch.version.cuda or torch.version.hip or getattr(torch.version, 'musa', None)
if backend is None:
    raise RuntimeError('Need CUDA/ROCm/MUSA torch build for PyNcclWrapper')

def has_gpu_backend() -> bool:
    import torch
    return (torch.version.cuda is not None
            or torch.version.hip is not None
            or getattr(torch.version, 'musa', None) is not None)

try:
    wrapper = PyNcclWrapper(...)
except ValueError as e:
    if 'only supports CUDA, ROCm and MUSA' in str(e):
        # skip pynccl custom allreduce, use torch.distributed defaults
        ...
    raise

When it happens

Trigger: Constructing PyNcclWrapper (custom allreduce / pynccl path) in a PyTorch build where torch.version.cuda, torch.version.hip, and torch.version.musa are all None — e.g. CPU-only or other-accelerator torch builds — so no NCCL library can be selected.

Common situations: Running sglang distributed init with a CPU-only torch wheel (often accidentally installed when CUDA torch was intended); a torch build for an unsupported accelerator; importing distributed utilities in unit tests on a CPU CI machine.

Related errors

• indices must be on q's device {device}, got {indices.device}
• Pi05 action state broadcast returned None
• --enable-tp-lm-head-all-to-all requires an available PyNCCL
• Error happened when batch testing peer-to-peer access from {
• Decode context parallel (decode_context_parallel_size > 1) i

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/fe2cd1fe140135f7. Report an issue: GitHub.