vllm-project/vllm · error · ValueError

unknown quantization name

Error message

unknown quantization name {v!r}; expected one of {sorted(QUANT_KEY_NAMES)}

What it means

Raised by _coerce_quant_key (a pydantic plain validator behind QuantKeyField) when a quantization spec string does not match any key in the QUANT_KEY_NAMES registry. The string is looked up in a fixed mapping of CLI-style quantization names to QuantKey singletons; a miss means the name is misspelled, removed, or not a registered per-layer quant key. The message lists all accepted names.

Solutions

  1. Copy the accepted list straight from the error message and use one of those exact strings
  2. If you meant a whole-preset, use a --quantization online shorthand (e.g. fp8_per_tensor) instead of a per-field QuantKey name
  3. Pass a QuantKey instance instead of a string when constructing configs programmatically
  4. If the name should exist, check you are on the vLLM version that registers it (grep QUANT_KEY_NAMES in vllm/config/quantization.py)

Example fix

# before
args = QuantizationConfigArgs(linear='fp8_per_tensor')

# after
args = QuantizationConfigArgs(linear='fp8_static_tensor_sym')  # exact QUANT_KEY_NAMES key
Defensive patterns

Strategy: validation

Validate before calling

from vllm.config.quantization import QUANT_KEY_NAMES

def valid_quant_name(name: str) -> bool:
    return name in QUANT_KEY_NAMES

Type guard

from vllm.config.quantization import QUANT_KEY_NAMES, QuantKey

def is_quant_key_name(v: object) -> bool:
    return isinstance(v, str) and v in QUANT_KEY_NAMES or isinstance(v, QuantKey)

Try / catch

try:
    QuantizationConfigArgs(linear=name)
except ValueError as e:
    if 'unknown quantization name' in str(e):
        raise SystemExit(f'bad quant name {name!r}; valid: {sorted(QUANT_KEY_NAMES)}') from e
    raise

Prevention

When it happens

Trigger: Passing a string like quantization_config={'linear': 'fp8_staict_tensor'} (typo) to QuantizationConfigArgs, or using a name that is an online CLI shorthand (e.g. 'fp8_per_tensor') where a QuantKey name is expected, or a name that only exists in an older/newer vLLM revision.

Common situations: Typos in quantization-config YAML/JSON; copy-pasting a --quantization shorthand into --quantization-config; version drift after QUANT_KEY_NAMES was renamed or entries were added/removed.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/1306b0790954bced. Report an issue: GitHub.

Appendix: source

Thrown at vllm/config/quantization.py:47

    "fp8_per_token": kFp8DynamicTokenSym,
    "fp8_per_channel_static": kFp8StaticChannelSym,
    "fp8_per_block_static": kFp8Static128BlockSym,
    "fp8_per_block_dynamic": kFp8Dynamic128Sym,
    "mxfp8": kMxfp8Dynamic,
    "mxfp4": kMxfp4Dynamic,
    "int8_per_channel_static": kInt8StaticChannelSym,
}


def _coerce_quant_key(v: Any) -> QuantKey | None:
    if v is None or isinstance(v, QuantKey):
        return v
    if not isinstance(v, str):
        raise TypeError(f"expected str or QuantKey, got {type(v).__name__}")
    try:
        return QUANT_KEY_NAMES[v]
    except KeyError:
        raise ValueError(
            f"unknown quantization name {v!r}; "
            f"expected one of {sorted(QUANT_KEY_NAMES)}"
        ) from None


# Stop pydantic from introspecting QuantKey: it transitively contains a
# NamedTuple with `ClassVar[GroupShape]` declarations that pydantic refuses.
QuantKeyField = Annotated[
    QuantKey | None,
    GetPydanticSchema(
        lambda _src, _handler: core_schema.no_info_plain_validator_function(
            _coerce_quant_key
        )
    ),
]


@config

View on GitHub (pinned to c794754062)