{"record":{"id":"65671aa4e919a8fa","repo":"vllm-project/vllm","slug":"cannot-run-the-multi-modal-processor-on-device-ty","errorCode":null,"errorMessage":"Cannot run the multi-modal processor on {device_type!r}: this instance also runs the language model. The processor would share the device with the model's forward pass, so its transform kernels contend with that compute, and because it runs in the API-server process its allocations are outside the memory the engine profiled for its KV cache -- risking OOM or a silently shrunken cache.\nAccelerator preprocessing is only supported on an encode-only instance of an encode/prefill/decode deployment (an EC producer that is not also a consumer), which runs no forward pass and allocates no KV cache.\nUse --mm-processor-device=cpu, or drop \"device\" from --mm-processor-kwargs.","messagePattern":"Cannot run the multi-modal processor on \\{device_type!r\\}: this instance also runs the language model\\. The processor would share the device with the model's forward pass, so its transform kernels contend with that compute, and because it runs in the API-server process its allocations are outside the memory the engine profiled for its KV cache -- risking OOM or a silently shrunken cache\\.\nAccelerator preprocessing is only supported on an encode-only instance of an encode/prefill/decode deployment \\(an EC producer that is not also a consumer\\), which runs no forward pass and allocates no KV cache\\.\nUse --mm-processor-device=cpu, or drop \"device\" from --mm-processor-kwargs\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/multimodal.py","lineNumber":441,"sourceCode":"                reachable from here, and because a field assigned after\n                construction would not re-trigger this config's validators.\n\n        Raises:\n            ValueError: If the requested device is not a torch device, or if it\n                is the accelerator on an instance that also runs the language\n                model.\n        \"\"\"\n        from vllm.platforms import current_platform\n\n        device_type = self.get_mm_processor_device_type()\n        accelerator = current_platform.device_type\n        if device_type is None or accelerator in (\"\", \"cpu\"):\n            return\n        if device_type != accelerator:\n            return\n\n        if ec_config is None or not ec_config.is_encode_only:\n            raise ValueError(\n                f\"Cannot run the multi-modal processor on {device_type!r}: this \"\n                \"instance also runs the language model. The processor would \"\n                \"share the device with the model's forward pass, so its \"\n                \"transform kernels contend with that compute, and because it \"\n                \"runs in the API-server process its allocations are outside the \"\n                \"memory the engine profiled for its KV cache -- risking OOM or \"\n                \"a silently shrunken cache.\\n\"\n                \"Accelerator preprocessing is only supported on an encode-only \"\n                \"instance of an encode/prefill/decode deployment (an EC \"\n                \"producer that is not also a consumer), which runs no forward \"\n                \"pass and allocates no KV cache.\\n\"\n                'Use --mm-processor-device=cpu, or drop \"device\" from '\n                \"--mm-processor-kwargs.\"\n            )\n\n        logger.info_once(\n            \"Running the multi-modal processor on %s. Override with \"\n            \"--mm-processor-device=cpu.\",","sourceCodeStart":423,"sourceCodeEnd":459,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/multimodal.py#L423-L459","documentation":"validate_mm_processor_device() blocks accelerator multimodal preprocessing when the requested mm_processor_kwargs device equals the platform accelerator AND the instance is not an encode-only EC producer. Reason: the processor would share the GPU with the LM forward pass, contending for compute, and its allocations sit outside the memory the engine profiled for the KV cache — risking OOM or a silently shrunken cache. Only encode-only nodes (no forward pass, no KV cache) may run preprocessing on the accelerator.","triggerScenarios":"Calling cfg.validate_mm_processor_device(ec_config) (done during engine init) with device 'cuda' on a CUDA platform when ec_config is None or ec_config.is_encode_only is False — i.e. any normal single-process or P/D deployment that adds --mm-processor-device cuda.","commonSituations":"Trying to 'speed up' image/video preprocessing by moving it to GPU on a standard vLLM server; setting device: cuda in mm_processor_kwargs without running an encode/prefill/decode disaggregated deployment with a dedicated encode-only producer.","solutions":["Drop accelerator preprocessing: use --mm-processor-device=cpu or remove \"device\" from --mm-processor-kwargs.","If you genuinely need accelerator preprocessing, run it on an encode-only instance of an EC (encode/prefill/decode) deployment — a producer node that is not also a consumer — and configure ECTransferConfig accordingly.","If this is a real encoder+LM disaggregation setup, verify the EC config reached this instance (ec_config non-None and is_encode_only true); a wiring/config bug can make it appear as a combined instance."],"exampleFix":"# before\nvllm serve model --mm-processor-kwargs '{\"device\": \"cuda\"}'\n\n# after\nvllm serve model --mm-processor-device cpu","handlingStrategy":"validation","validationCode":"def check_mm_processor_device_allowed(device: str | None, is_encode_only_instance: bool) -> None:\n    if device in (\"cuda\", \"gpu-type accelerator\") and not is_encode_only_instance:\n        raise SystemExit(\"Accelerator preprocessing needs an encode-only EC instance; use cpu\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Default to --mm-processor-device cpu; only request the accelerator on encode-only EC producer nodes.","In disaggregated deployments, assert ec_config.is_encode_only before injecting device=cuda into mm_processor_kwargs."],"tags":["vllm","config","multimodal","device","gpu","oom","disaggregation"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}