What causes "Unknown dtype: {dtype}" in vllm-project/vllm?

Normalize dtype to a str or torch.dtype before constructing engine/model config (validate at your config boundary).

vllm-project/vllm · error · ValueError

Unknown dtype: {dtype}

Error message

Unknown dtype: {dtype}

What it means

The else-branch of _get_and_verify_dtype: dtype is neither a str nor a torch.dtype (e.g. None slipping through, an int, or a numpy dtype), so it cannot be interpreted. Note this path formats with {dtype} (no !r), unlike the string branch.

Source

Thrown at vllm/config/model.py:2286

    model_type = config.model_type

    if isinstance(dtype, str):
        dtype = dtype.lower()
        if dtype == "auto":
            # Set default dtype from model config
            torch_dtype = _resolve_auto_dtype(
                model_type,
                config_dtype,
                is_pooling_model=is_pooling_model,
            )
        else:
            if dtype not in _STR_DTYPE_TO_TORCH_DTYPE:
                raise ValueError(f"Unknown dtype: {dtype!r}")
            torch_dtype = _STR_DTYPE_TO_TORCH_DTYPE[dtype]
    elif isinstance(dtype, torch.dtype):
        torch_dtype = dtype
    else:
        raise ValueError(f"Unknown dtype: {dtype}")

    _check_valid_dtype(model_type, torch_dtype)

    if torch_dtype != config_dtype:
        if torch_dtype == torch.float32:
            # Upcasting to float32 is allowed.
            logger.info("Upcasting %s to %s.", config_dtype, torch_dtype)
        elif config_dtype == torch.float32:
            # Downcasting from float32 to float16 or bfloat16 is allowed.
            logger.info("Downcasting %s to %s.", config_dtype, torch_dtype)
        else:
            # Casting between float16 and bfloat16 is allowed with a warning.
            logger.warning("Casting %s to %s.", config_dtype, torch_dtype)

    return torch_dtype


def _get_head_dtype(

View on GitHub (pinned to c794754062)

Solutions

Normalize dtype to a str or torch.dtype before constructing engine/model config (validate at your config boundary).
Map numpy dtypes to torch dtypes (e.g. numpy.float16 -> torch.float16) if that is the source.

Example fix

# before
llm = LLM(model='my-model', dtype=np.float16)
# after
import torch
llm = LLM(model='my-model', dtype=torch.float16)

Defensive patterns

Strategy: type-guard

Validate before calling

import torch
def normalize_dtype(d):
    if isinstance(d, str):
        return d.lower()
    if isinstance(d, torch.dtype):
        return d
    raise TypeError(f'dtype must be str or torch.dtype, got {type(d)!r}')

Type guard

import torch
def is_resolvable_dtype(d: object) -> bool:
    return isinstance(d, (str, torch.dtype))

Try / catch

except ValueError as e:
    if 'Unknown dtype' in str(e):
        coerce numpy dtypes to torch (np.float16 -> torch.float16) and retry once

Prevention

Validate that dtype comes from a str or torch.dtype before passing it into engine config.
Never let None or arbitrary config-file values reach dtype; default explicitly to 'auto'.
Map numpy dtypes at the framework boundary if you wrap vLLM.

When it happens

Trigger: Calling the dtype resolution path with a non-str, non-torch.dtype object — for example a numpy dtype, a Python None from a misconfigured default, or a custom enum.

Common situations: Programmatic construction of EngineArgs where dtype comes from unvalidated user input or a config file (YAML parses 'float16' fine but a typo'd key yields None); wrapping vLLM in another framework that passes numpy dtypes.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/4e3b03e38aafb2b9. Report an issue: GitHub.