What causes "Unknown dtype: {dtype!r}" in vllm-project/vllm?

Use a supported dtype name: 'float16', 'bfloat16', 'float32', 'auto' (and any others present in _STR_DTYPE_TO_TORCH_DTYPE).

vllm-project/vllm · error · ValueError

Unknown dtype: {dtype!r}

Error message

Unknown dtype: {dtype!r}

What it means

_get_and_verify_dtype accepts dtype as a string only if it is a key of _STR_DTYPE_TO_TORCH_DTYPE. An unrecognized string (wrong case is handled, but unknown names are not) raises ValueError with the offending value.

Source

Thrown at vllm/config/model.py:2281

    config_format: str | ConfigFormat = "hf",
) -> torch.dtype:
    config_dtype = ModelArchConfigConvertorBase.get_torch_dtype(
        config, model_id, revision=revision, config_format=config_format
    )
    model_type = config.model_type

    if isinstance(dtype, str):
        dtype = dtype.lower()
        if dtype == "auto":
            # Set default dtype from model config
            torch_dtype = _resolve_auto_dtype(
                model_type,
                config_dtype,
                is_pooling_model=is_pooling_model,
            )
        else:
            if dtype not in _STR_DTYPE_TO_TORCH_DTYPE:
                raise ValueError(f"Unknown dtype: {dtype!r}")
            torch_dtype = _STR_DTYPE_TO_TORCH_DTYPE[dtype]
    elif isinstance(dtype, torch.dtype):
        torch_dtype = dtype
    else:
        raise ValueError(f"Unknown dtype: {dtype}")

    _check_valid_dtype(model_type, torch_dtype)

    if torch_dtype != config_dtype:
        if torch_dtype == torch.float32:
            # Upcasting to float32 is allowed.
            logger.info("Upcasting %s to %s.", config_dtype, torch_dtype)
        elif config_dtype == torch.float32:
            # Downcasting from float32 to float16 or bfloat16 is allowed.
            logger.info("Downcasting %s to %s.", config_dtype, torch_dtype)
        else:
            # Casting between float16 and bfloat16 is allowed with a warning.
            logger.warning("Casting %s to %s.", config_dtype, torch_dtype)

View on GitHub (pinned to c794754062)

Solutions

Use a supported dtype name: 'float16', 'bfloat16', 'float32', 'auto' (and any others present in _STR_DTYPE_TO_TORCH_DTYPE).
Pass a torch.dtype object directly when using the Python API instead of a string alias.

Example fix

# before
vllm serve my-model --dtype fp16
# after
vllm serve my-model --dtype float16

Defensive patterns

Strategy: validation

Validate before calling

from vllm.config.model import _STR_DTYPE_TO_TORCH_DTYPE
def is_valid_dtype_str(s: object) -> bool:
    return isinstance(s, str) and s.lower() in _STR_DTYPE_TO_TORCH_DTYPE

Type guard

def is_valid_dtype_str(s: object) -> bool:
    return isinstance(s, str) and s.lower() in _STR_DTYPE_TO_TORCH_DTYPE

Try / catch

except ValueError as e:
    if 'Unknown dtype' in str(e):
        fall back to dtype='auto' and log the rejected value

Prevention

Use canonical names float16/bfloat16/float32/auto — not fp16/bf16/half shorthands.
Validate dtype strings at your config boundary with _STR_DTYPE_TO_TORCH_DTYPE before constructing engine args.
Pass torch.dtype objects in Python code to sidestep string parsing entirely.

When it happens

Trigger: Passing dtype strings like 'fp16', 'bf16', 'half', or a typo ('float164') as --dtype or dtype= in engine args.

Common situations: Using shorthand dtype names common in other frameworks (fp16/bf16 from PyTorch conventions) instead of vLLM's full names (float16/bfloat16); config templating that injects an empty or malformed dtype string.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/02ae815a95e11f06. Report an issue: GitHub.