{"record":{"id":"7ff17d9b8717165b","repo":"sgl-project/sglang","slug":"a-cuda-vmm-enabled-model-must-provide-a-multimodal","errorCode":null,"errorMessage":"A CUDA VMM-enabled model must provide a multimodal processor","messagePattern":"A CUDA VMM-enabled model must provide a multimodal processor","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/utils/cuda_vmm_transport_utils.py","lineNumber":939,"sourceCode":"        tensor_bytes = packed_buffer[\n            self._packed_relative_offset : self._packed_relative_offset\n            + self.data_nbytes\n        ]\n        reconstructed = tensor_bytes.view(self.dtype).reshape(self.shape)\n        self.reconstruct_tensor = reconstructed\n        self._consumer_acknowledged = True\n        return reconstructed\n\n\nclass CudaVmmFeatureTransport:\n    \"\"\"Tokenizer-owned VMM transport for one tokenizer worker.\"\"\"\n\n    def __init__(self, server_args, mm_processor) -> None:\n        self.pool: CudaVmmMemoryPool | None = None\n        if get_mm().mm_feature_transport != \"cuda_vmm\":\n            return\n        if mm_processor is None:\n            raise RuntimeError(\n                \"A CUDA VMM-enabled model must provide a multimodal processor\"\n            )\n\n        per_worker_pool_size = get_mm_feature_pool_size_per_worker(\n            MM_FEATURE_CACHE_SIZE, server_args.tokenizer_worker_num\n        )\n        self.pool = CudaVmmMemoryPool(\n            memory_size=per_worker_pool_size,\n            recycle_interval=MM_ITEM_MEMORY_POOL_RECYCLE_INTERVAL,\n            base_gpu_id=server_args.base_gpu_id,\n            consumer_count=get_vmm_feature_consumer_count(),\n            allow_posix_fallback=server_args.nnodes == 1,\n        )\n\n    def prepare_for_dispatch(\n        self,\n        mm_inputs_batch: Iterable[MultimodalProcessorOutput | None],\n    ) -> list[MultimodalDataItem]:","sourceCodeStart":921,"sourceCodeEnd":957,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/utils/cuda_vmm_transport_utils.py#L921-L957","documentation":"Thrown during CudaVmmFeatureTransport init when mm feature transport is set to 'cuda_vmm' but no multimodal processor was passed. The VMM transport path requires the mm processor to register/size feature layouts, so a None processor is unrecoverable at this point.","triggerScenarios":"Constructing the transport with server args enabling cuda_vmm mm_feature_transport and mm_processor=None; a model implementation that fails to expose its multimodal processor; non-multimodal model launched with cuda_vmm transport enabled.","commonSituations":"Enabling --mm-feature-transport cuda_vmm (or equivalent) on a text-only or partially supported model; version changes in the model's processor registration API.","solutions":["Disable cuda_vmm mm feature transport (use default transport) for models without a multimodal processor","Ensure the model implementation provides its mm processor to the transport constructor","Verify the model actually supports multimodal VMM transport before enabling the flag"],"exampleFix":"# before\nserver_args.mm_feature_transport = \"cuda_vmm\"  # model has no mm processor\n\n# after\nserver_args.mm_feature_transport = \"default\"","handlingStrategy":"validation","validationCode":"if server_args.mm_feature_transport == \"cuda_vmm\" and model.mm_processor is None:\n    server_args.mm_feature_transport = \"default\"","typeGuard":"def supports_cuda_vmm_transport(model) -> bool:\n    return model.mm_processor is not None","tryCatchPattern":null,"preventionTips":["Only enable cuda_vmm mm transport for multimodal models with processors","Fail fast at startup with a config check"],"tags":["cuda","vmm","multimodal","config"],"backgroundTag":"missing-required-dependency","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}