{"record":{"id":"08c97a139f5848fb","repo":"sgl-project/sglang","slug":"mm-feature-transport-cuda-ipc-requires-nvidia-cu","errorCode":null,"errorMessage":"--mm-feature-transport=cuda_ipc requires NVIDIA CUDA.","messagePattern":"--mm-feature-transport=cuda_ipc requires NVIDIA CUDA\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":9006,"sourceCode":"                    \"--mm-feature-transport=cuda_vmm is not supported with \"\n                    \"SGLANG_RUST_SERVER.\"\n                )\n            pool_budget_mb = envs.SGLANG_MM_FEATURE_CACHE_MB.get()\n            handle_kind = \"CUDA FABRIC\" if cfg.nnodes > 1 else \"POSIX FD\"\n            logger.info(\n                \"Using CUDA VMM for multimodal features with %s sharing: \"\n                \"reserving up to %d MiB on base GPU %d across %d tokenizer \"\n                \"worker(s). This reduces KV cache headroom; a full pool falls \"\n                \"back to inline CPU transport.\",\n                handle_kind,\n                pool_budget_mb,\n                cfg.base_gpu_id,\n                cfg.tokenizer_worker_num,\n            )\n\n        if requested_transport == \"cuda_ipc\":\n            if not is_cuda():\n                raise ValueError(\n                    \"--mm-feature-transport=cuda_ipc requires NVIDIA CUDA.\"\n                )\n            if cfg.nnodes != 1:\n                raise ValueError(\n                    \"--mm-feature-transport=cuda_ipc only supports a single node.\"\n                )\n\n            pool_budget_mb = envs.SGLANG_MM_FEATURE_CACHE_MB.get()\n            logger.info(\n                \"Using CUDA IPC for multimodal features: reserving up to %d MiB \"\n                \"on base GPU %d across %d tokenizer worker(s). This reduces KV \"\n                \"cache headroom; a full pool falls back to CPU transport.\",\n                pool_budget_mb,\n                cfg.base_gpu_id,\n                cfg.tokenizer_worker_num,\n            )\n            logger.info(\n                \"CUDA IPC pool-handle caching is %s. It reuses mappings to the \"","sourceCodeStart":8988,"sourceCodeEnd":9024,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L8988-L9024","documentation":"ServerArgs validation rejects --mm-feature-transport=cuda_ipc when the runtime does not detect an NVIDIA CUDA GPU. The cuda_ipc transport passes multimodal feature tensors between processes via CUDA IPC handles, which only exist on NVIDIA GPUs with CUDA drivers. The check runs is_cuda() during server argument resolution before any model loads.","triggerScenarios":"Launching the SGLang server with --mm-feature-transport cuda_ipc on a machine where is_cuda() returns False (no NVIDIA GPU, ROCm/AMD-only node, CPU-only box, or CUDA driver/device not visible to the process).","commonSituations":"Running on AMD ROCm or CPU-only dev boxes; CUDA_VISIBLE_DEVICES set to an empty string or hiding all devices; container without GPU passthrough; defaulting the flag in a shared launch script that also runs on non-CUDA nodes.","solutions":["Verify an NVIDIA GPU is visible: run nvidia-smi and check CUDA_VISIBLE_DEVICES isn't empty/masking devices","Remove --mm-feature-transport cuda_ipc or switch to the default transport (e.g. --mm-feature-transport cpu_copy or omit the flag)","On ROCm/AMD systems, use a supported non-IPC transport for multimodal features"],"exampleFix":"# before\npython -m sglang.launch_server --model Qwen2.5-VL-7B --mm-feature-transport cuda_ipc\n# after (non-CUDA node)\npython -m sglang.launch_server --model Qwen2.5-VL-7B","handlingStrategy":"validation","validationCode":"import torch\ndef can_use_cuda_ipc_transport() -> bool:\n    return torch.cuda.is_available() and torch.version.cuda is not None\n\ntransport = \"cuda_ipc\" if can_use_cuda_ipc_transport() else None\n# then: launch_server_args.mm_feature_transport = transport","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Gate --mm-feature-transport cuda_ipc behind a torch.cuda.is_available() check in launch wrappers","Log the detected GPU vendor/model at startup so transport mismatches are obvious","Run nvidia-smi in CI smoke tests before exercising IPC-based multimodal paths"],"tags":["sglang","cuda","multimodal","ipc","hardware-requirement"],"backgroundTag":"cuda-device-unavailable","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}