What causes "Unknown dtype: {head_dtype}" in vllm-project/vllm?

Fix config.json so head_dtype is one of: omitted, "model", a dtype name string, or absent — never a number or structure.

vllm-project/vllm · error · ValueError

Unknown dtype: {head_dtype}

Error message

Unknown dtype: {head_dtype}

What it means

The terminal else of _get_head_dtype: head_dtype is not None, not the string 'model', not a str, and not a torch.dtype — it is some other object type (int, list, numpy dtype), so it cannot be interpreted and raises ValueError (formatted without !r).

Source

Thrown at vllm/config/model.py:2325

    head_dtype: str | torch.dtype | None = getattr(config, "head_dtype", None)

    if head_dtype == "model":
        return dtype
    elif isinstance(head_dtype, str):
        head_dtype = head_dtype.lower()
        if head_dtype not in _STR_DTYPE_TO_TORCH_DTYPE:
            raise ValueError(f"Unknown dtype: {head_dtype!r}")
        return _STR_DTYPE_TO_TORCH_DTYPE[head_dtype]
    elif isinstance(head_dtype, torch.dtype):
        return head_dtype
    elif head_dtype is None:
        if torch.float32 not in current_platform.supported_dtypes:
            return dtype
        if runner_type == "pooling":
            return torch.float32
        return dtype
    else:
        raise ValueError(f"Unknown dtype: {head_dtype}")


def _get_and_verify_max_len(
    hf_config: PretrainedConfig,
    model_arch_config: ModelArchitectureConfig,
    tokenizer_config: dict | None,
    max_model_len: int | None,
    disable_sliding_window: bool,
    sliding_window: int | None,
    spec_target_max_model_len: int | None = None,
    encoder_config: dict[str, Any] | None = None,
) -> int:
    """Get and verify the model's maximum length."""
    (derived_max_model_len, max_len_key) = (
        model_arch_config.derived_max_model_len_and_key
    )

    # If sliding window is manually disabled, max_length should be less

View on GitHub (pinned to c794754062)

Solutions

Fix config.json so head_dtype is one of: omitted, "model", a dtype name string, or absent — never a number or structure.
If per-head dtype control is unneeded, delete the field to use defaults.

Example fix

// before: config.json
"head_dtype": 16
// after
"head_dtype": "float16"

Defensive patterns

Strategy: type-guard

Validate before calling

import torch
def valid_head_dtype_value(hd) -> bool:
    return hd is None or hd == 'model' or isinstance(hd, (str, torch.dtype))

Type guard

import torch
def is_valid_head_dtype(hd: object) -> bool:
    return hd is None or hd == 'model' or isinstance(hd, (str, torch.dtype))

Prevention

Never store numeric dtype codes in head_dtype; JSON config fields must be strings or omitted.
Validate hand-edited config.json files with a schema before serving.
Automate checkpoint post-processing so experimental head_dtype values are normalized or stripped.

When it happens

Trigger: config.head_dtype set to a non-string, non-torch.dtype JSON value (e.g. 16, 2, or a nested list) in the checkpoint's config.json.

Common situations: Hand-edited or programmatically generated config.json where head_dtype was written as a numeric code; conversion scripts mapping framework enums to JSON incorrectly.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/fadf042e40b0a0a3. Report an issue: GitHub.