sgl-project/sglang · error · ValueError

--mm-feature-transport=cuda_ipc only supports a single node.

Error message

--mm-feature-transport=cuda_ipc only supports a single node.

What it means

ServerArgs validation rejects --mm-feature-transport=cuda_ipc in multi-node deployments (cfg.nnodes != 1). CUDA IPC handles are only valid between processes on the same physical node; they cannot cross network boundaries, so multimodal feature transfer over IPC is restricted to single-node launches.

Source

Thrown at python/sglang/srt/server_args.py:9010

            handle_kind = "CUDA FABRIC" if cfg.nnodes > 1 else "POSIX FD"
            logger.info(
                "Using CUDA VMM for multimodal features with %s sharing: "
                "reserving up to %d MiB on base GPU %d across %d tokenizer "
                "worker(s). This reduces KV cache headroom; a full pool falls "
                "back to inline CPU transport.",
                handle_kind,
                pool_budget_mb,
                cfg.base_gpu_id,
                cfg.tokenizer_worker_num,
            )

        if requested_transport == "cuda_ipc":
            if not is_cuda():
                raise ValueError(
                    "--mm-feature-transport=cuda_ipc requires NVIDIA CUDA."
                )
            if cfg.nnodes != 1:
                raise ValueError(
                    "--mm-feature-transport=cuda_ipc only supports a single node."
                )

            pool_budget_mb = envs.SGLANG_MM_FEATURE_CACHE_MB.get()
            logger.info(
                "Using CUDA IPC for multimodal features: reserving up to %d MiB "
                "on base GPU %d across %d tokenizer worker(s). This reduces KV "
                "cache headroom; a full pool falls back to CPU transport.",
                pool_budget_mb,
                cfg.base_gpu_id,
                cfg.tokenizer_worker_num,
            )
            logger.info(
                "CUDA IPC pool-handle caching is %s. It reuses mappings to the "
                "existing bounded pool without reserving another pool; set "
                "SGLANG_USE_IPC_POOL_HANDLE_CACHE=0 to disable it.",
                (
                    "enabled"

View on GitHub (pinned to 0132848349)

Solutions

1. Drop --mm-feature-transport cuda_ipc and use the network-capable default transport for multi-node runs
2. Restrict cuda_ipc usage to single-node deployments (--nnodes 1 or not set)
3. Add per-topology config: cuda_ipc when nnodes==1, default otherwise

Example fix

# before
# node0
python -m sglang.launch_server --nnodes 2 --node-rank 0 ... --mm-feature-transport cuda_ipc
# after
python -m sglang.launch_server --nnodes 2 --node-rank 0 ...  # default transport

def resolve_transport(nnodes: int) -> str | None:
    if nnodes > 1:
        return None  # use default network-capable transport
    return "cuda_ipc" if torch.cuda.is_available() else None

When it happens

Trigger: Starting a multi-node SGLang cluster (--nnodes 2 or more, or --dp ... with multiple hosts) with --mm-feature-transport cuda_ipc on any node; the nnodes check fails during arg resolution regardless of GPU vendor.

Common situations: Scaling a working single-node IPC setup to multi-node without changing the transport flag; copy-pasted launch scripts across nodes; distributed inference of multimodal models with tensor/data parallelism spanning hosts.

Related errors

• Multi-node weight cache daemons (nnodes > 1) require --dist-
• Waiting for main node timeout!
• {selection_error}{component_suffix}
• No compatible attention backend is available{component_suffi
• NCCL only supports CUDA, ROCm and MUSA backends.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e56019cf0ede8a04. Report an issue: GitHub.