What causes "expected str or QuantKey, got {type(v).__name__}" in vllm-project/vllm?

Pass the quantization method as a string name, e.g. 'fp8', 'awq', 'gptq'.

vllm-project/vllm · error · TypeError

expected str or QuantKey, got {type(v).name}

Error message

expected str or QuantKey, got {type(v).__name__}

What it means

_coerce_quant_key is the pydantic BeforeValidator that coerces quantization names into QuantKey objects. It raises TypeError when the input value is neither None, a QuantKey instance, nor a str — e.g. an int, list, or dict was supplied where a quantization method name string is expected.

Source

Thrown at vllm/config/quantization.py:43

# User-facing names addressable from quantization_config.
QUANT_KEY_NAMES: dict[str, QuantKey] = {
    "fp8_per_tensor_static": kFp8StaticTensorSym,
    "fp8_per_tensor_dynamic": kFp8DynamicTensorSym,
    "fp8_per_token": kFp8DynamicTokenSym,
    "fp8_per_channel_static": kFp8StaticChannelSym,
    "fp8_per_block_static": kFp8Static128BlockSym,
    "fp8_per_block_dynamic": kFp8Dynamic128Sym,
    "mxfp8": kMxfp8Dynamic,
    "mxfp4": kMxfp4Dynamic,
    "int8_per_channel_static": kInt8StaticChannelSym,
}


def _coerce_quant_key(v: Any) -> QuantKey | None:
    if v is None or isinstance(v, QuantKey):
        return v
    if not isinstance(v, str):
        raise TypeError(f"expected str or QuantKey, got {type(v).__name__}")
    try:
        return QUANT_KEY_NAMES[v]
    except KeyError:
        raise ValueError(
            f"unknown quantization name {v!r}; "
            f"expected one of {sorted(QUANT_KEY_NAMES)}"
        ) from None


# Stop pydantic from introspecting QuantKey: it transitively contains a
# NamedTuple with `ClassVar[GroupShape]` declarations that pydantic refuses.
QuantKeyField = Annotated[
    QuantKey | None,
    GetPydanticSchema(
        lambda _src, _handler: core_schema.no_info_plain_validator_function(
            _coerce_quant_key
        )
    ),

View on GitHub (pinned to c794754062)

Solutions

Pass the quantization method as a string name, e.g. 'fp8', 'awq', 'gptq'.
Or pass None to leave quantization auto-detected, or a QuantKey instance.
Sanitize external input: str(...) coerce or validate isinstance(v, str) before building the config.

Example fix

# before
quant_cfg = QuantizationConfig(quantization=8)  # TypeError

# after
quant_cfg = QuantizationConfig(quantization='fp8')

Defensive patterns

Strategy: type-guard

Validate before calling

def coerce_quant_name(v):
    if v is None or isinstance(v, (str, QuantKey)):
        return v
    if isinstance(v, (int, float)):
        return str(v)  # or reject explicitly
    raise TypeError(f"quantization name must be str, got {type(v).__name__}")

Type guard

from vllm.config.quantization import QuantKey

def is_quant_name(v) -> TypeGuard[str | QuantKey | None]:
    return v is None or isinstance(v, (str, QuantKey))

Try / catch

try:
    cfg = ModelConfig(..., quantization=quant)
except TypeError as e:
    if "expected str or QuantKey" in str(e):
        cfg = ModelConfig(..., quantization=str(quant))
    else:
        raise

Prevention

Validate quantization values against isinstance(v, str) when configs come from JSON/CLI input.
Reference valid names via vllm.config.quantization.QUANT_KEY_NAMES instead of free-typing strings.

When it happens

Trigger: Passing a non-string value into a quantization config field that routes through _coerce_quant_key — e.g. quantization=8, quantization=['fp8'], or quantization={'method': 'fp8'} — instead of quantization='fp8' (or a QuantKey instance / None).

Common situations: Programmatically building quantization configs from untyped data (JSON numbers, parsed CLI ints like --quantization 8); passing an enum or object from another library where a name string is required.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/a7893dae9b089607. Report an issue: GitHub.