vllm-project/vllm · error · ValueError

Invalid "device" in mm_processor_kwargs: {device!r}. Expecte

Error message

Invalid "device" in mm_processor_kwargs: {device!r}. Expected a torch device such as "cpu", "cuda" or "cuda:0".

What it means

MultiModalConfig.get_mm_processor_device_type() parses mm_processor_kwargs['device'] with torch.device(); any string torch.device() rejects (RuntimeError/TypeError/ValueError) is re-raised as a ValueError telling you to use a torch device spec like 'cpu', 'cuda', 'cuda:0'. This is the single parse point — validate_mm_processor_device() surfaces it during startup.

Source

Thrown at vllm/config/multimodal.py:409

        accepts -- `"cuda"`, `"cuda:1"`, `torch.device(...)`, or a bare index.
        Normalising through torch rather than parsing the string keeps the
        non-string forms from slipping past a caller's comparison.

        Returns:
            The device type, or None when no device is requested.

        Raises:
            ValueError: If `device` is not something `torch.device` accepts.
                `validate_mm_processor_device` is what surfaces this during
                startup, so the value is only parsed once.
        """
        device = (self.mm_processor_kwargs or {}).get("device")
        if device is None:
            return None
        try:
            return torch.device(device).type  # type: ignore[arg-type]
        except (RuntimeError, TypeError, ValueError):
            raise ValueError(
                f'Invalid "device" in mm_processor_kwargs: {device!r}. Expected a '
                'torch device such as "cpu", "cuda" or "cuda:0".'
            ) from None

    def validate_mm_processor_device(self, ec_config: ECTransferConfig | None) -> None:
        """Check `mm_processor_kwargs["device"]` for this deployment.

        The only place the requested device is validated, so it runs even on a
        CPU-only platform: the value is parsed before any early return.

        Args:
            ec_config: The deployment's EC config, or None when it is not an
                encode/prefill/decode deployment. Passed in because it is not
                reachable from here, and because a field assigned after
                construction would not re-trigger this config's validators.

        Raises:
            ValueError: If the requested device is not a torch device, or if it

View on GitHub (pinned to c794754062)

Solutions

  1. Use a valid torch device string: 'cpu', 'cuda', or a concrete index like 'cuda:0'.
  2. Validate the JSON passed to --mm-processor-kwargs (jq or python -m json.tool) so device stays a string.
  3. Prefer the dedicated convenience flag --mm-processor-device cpu instead of embedding 'device' in the kwargs dict.

Example fix

# before
vllm serve model --mm-processor-kwargs '{"device": "gpu"}'

# after
vllm serve model --mm-processor-kwargs '{"device": "cuda:0"}'
# or simply
vllm serve model --mm-processor-device cpu
Defensive patterns

Strategy: type-guard

Validate before calling

import torch

def normalize_device(kwargs: dict) -> dict:
    dev = kwargs.get("device")
    if dev is not None:
        torch.device(dev)  # raises here with a clear traceback if invalid
    return kwargs

Type guard

def is_valid_torch_device(dev) -> bool:
    if not isinstance(dev, str):
        return False
    try:
        torch.device(dev)
        return True
    except (RuntimeError, TypeError, ValueError):
        return False

Prevention

When it happens

Trigger: Passing --mm-processor-kwargs '{"device": "gpu"}' or "device=CPU!" or a non-string type like {"device": 0}; any value for which torch.device(value) throws.

Common situations: Confusing torch device names with accelerator labels ('gpu', 'npu' misspelled, 'device:0' CUDA-style syntax); malformed JSON on the CLI producing an int; copy-pasting device strings from other frameworks (e.g. JAX or TensorFlow device specs).

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/bfae13986e6a7151. Report an issue: GitHub.