sgl-project/sglang · error · RuntimeError

A CUDA VMM-enabled model must provide a multimodal processor

Error message

A CUDA VMM-enabled model must provide a multimodal processor

What it means

Thrown during CudaVmmFeatureTransport init when mm feature transport is set to 'cuda_vmm' but no multimodal processor was passed. The VMM transport path requires the mm processor to register/size feature layouts, so a None processor is unrecoverable at this point.

Source

Thrown at python/sglang/srt/utils/cuda_vmm_transport_utils.py:939

        tensor_bytes = packed_buffer[
            self._packed_relative_offset : self._packed_relative_offset
            + self.data_nbytes
        ]
        reconstructed = tensor_bytes.view(self.dtype).reshape(self.shape)
        self.reconstruct_tensor = reconstructed
        self._consumer_acknowledged = True
        return reconstructed


class CudaVmmFeatureTransport:
    """Tokenizer-owned VMM transport for one tokenizer worker."""

    def __init__(self, server_args, mm_processor) -> None:
        self.pool: CudaVmmMemoryPool | None = None
        if get_mm().mm_feature_transport != "cuda_vmm":
            return
        if mm_processor is None:
            raise RuntimeError(
                "A CUDA VMM-enabled model must provide a multimodal processor"
            )

        per_worker_pool_size = get_mm_feature_pool_size_per_worker(
            MM_FEATURE_CACHE_SIZE, server_args.tokenizer_worker_num
        )
        self.pool = CudaVmmMemoryPool(
            memory_size=per_worker_pool_size,
            recycle_interval=MM_ITEM_MEMORY_POOL_RECYCLE_INTERVAL,
            base_gpu_id=server_args.base_gpu_id,
            consumer_count=get_vmm_feature_consumer_count(),
            allow_posix_fallback=server_args.nnodes == 1,
        )

    def prepare_for_dispatch(
        self,
        mm_inputs_batch: Iterable[MultimodalProcessorOutput | None],
    ) -> list[MultimodalDataItem]:

View on GitHub (pinned to 0132848349)

Solutions

1. Disable cuda_vmm mm feature transport (use default transport) for models without a multimodal processor
2. Ensure the model implementation provides its mm processor to the transport constructor
3. Verify the model actually supports multimodal VMM transport before enabling the flag

Example fix

# before
server_args.mm_feature_transport = "cuda_vmm"  # model has no mm processor

# after
server_args.mm_feature_transport = "default"

if server_args.mm_feature_transport == "cuda_vmm" and model.mm_processor is None:
    server_args.mm_feature_transport = "default"

def supports_cuda_vmm_transport(model) -> bool:
    return model.mm_processor is not None

When it happens

Trigger: Constructing the transport with server args enabling cuda_vmm mm_feature_transport and mm_processor=None; a model implementation that fails to expose its multimodal processor; non-multimodal model launched with cuda_vmm transport enabled.

Common situations: Enabling --mm-feature-transport cuda_vmm (or equivalent) on a text-only or partially supported model; version changes in the model's processor registration API.

Related errors

• CUDA VMM feature transport requires each feature field to co
• {selection_error}{component_suffix}
• f"Unsupported patch_size type: {type(patch_size)}"
• batching config rule requires max_batch_size
• For multimodal input processing do not set `enable_tokenizer

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7ff17d9b8717165b. Report an issue: GitHub.