vllm-project/vllm · error · ValueError

Cannot run the multi-modal processor on {device_type!r}: thi

Error message

Cannot run the multi-modal processor on {device_type!r}: this instance also runs the language model. The processor would share the device with the model's forward pass, so its transform kernels contend with that compute, and because it runs in the API-server process its allocations are outside the memory the engine profiled for its KV cache -- risking OOM or a silently shrunken cache.
Accelerator preprocessing is only supported on an encode-only instance of an encode/prefill/decode deployment (an EC producer that is not also a consumer), which runs no forward pass and allocates no KV cache.
Use --mm-processor-device=cpu, or drop "device" from --mm-processor-kwargs.

What it means

validate_mm_processor_device() blocks accelerator multimodal preprocessing when the requested mm_processor_kwargs device equals the platform accelerator AND the instance is not an encode-only EC producer. Reason: the processor would share the GPU with the LM forward pass, contending for compute, and its allocations sit outside the memory the engine profiled for the KV cache — risking OOM or a silently shrunken cache. Only encode-only nodes (no forward pass, no KV cache) may run preprocessing on the accelerator.

Source

Thrown at vllm/config/multimodal.py:441

                reachable from here, and because a field assigned after
                construction would not re-trigger this config's validators.

        Raises:
            ValueError: If the requested device is not a torch device, or if it
                is the accelerator on an instance that also runs the language
                model.
        """
        from vllm.platforms import current_platform

        device_type = self.get_mm_processor_device_type()
        accelerator = current_platform.device_type
        if device_type is None or accelerator in ("", "cpu"):
            return
        if device_type != accelerator:
            return

        if ec_config is None or not ec_config.is_encode_only:
            raise ValueError(
                f"Cannot run the multi-modal processor on {device_type!r}: this "
                "instance also runs the language model. The processor would "
                "share the device with the model's forward pass, so its "
                "transform kernels contend with that compute, and because it "
                "runs in the API-server process its allocations are outside the "
                "memory the engine profiled for its KV cache -- risking OOM or "
                "a silently shrunken cache.\n"
                "Accelerator preprocessing is only supported on an encode-only "
                "instance of an encode/prefill/decode deployment (an EC "
                "producer that is not also a consumer), which runs no forward "
                "pass and allocates no KV cache.\n"
                'Use --mm-processor-device=cpu, or drop "device" from '
                "--mm-processor-kwargs."
            )

        logger.info_once(
            "Running the multi-modal processor on %s. Override with "
            "--mm-processor-device=cpu.",

View on GitHub (pinned to c794754062)

Solutions

  1. Drop accelerator preprocessing: use --mm-processor-device=cpu or remove "device" from --mm-processor-kwargs.
  2. If you genuinely need accelerator preprocessing, run it on an encode-only instance of an EC (encode/prefill/decode) deployment — a producer node that is not also a consumer — and configure ECTransferConfig accordingly.
  3. If this is a real encoder+LM disaggregation setup, verify the EC config reached this instance (ec_config non-None and is_encode_only true); a wiring/config bug can make it appear as a combined instance.

Example fix

# before
vllm serve model --mm-processor-kwargs '{"device": "cuda"}'

# after
vllm serve model --mm-processor-device cpu
Defensive patterns

Strategy: validation

Validate before calling

def check_mm_processor_device_allowed(device: str | None, is_encode_only_instance: bool) -> None:
    if device in ("cuda", "gpu-type accelerator") and not is_encode_only_instance:
        raise SystemExit("Accelerator preprocessing needs an encode-only EC instance; use cpu")

Prevention

When it happens

Trigger: Calling cfg.validate_mm_processor_device(ec_config) (done during engine init) with device 'cuda' on a CUDA platform when ec_config is None or ec_config.is_encode_only is False — i.e. any normal single-process or P/D deployment that adds --mm-processor-device cuda.

Common situations: Trying to 'speed up' image/video preprocessing by moving it to GPU on a standard vLLM server; setting device: cuda in mm_processor_kwargs without running an encode/prefill/decode disaggregated deployment with a dedicated encode-only producer.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/65671aa4e919a8fa. Report an issue: GitHub.