vllm-project/vllm · error · ValueError

online shorthand {v!r} does not define a {field_name} spec

Error message

online shorthand {v!r} does not define a {field_name} spec

What it means

QuantizationConfigArgs coerces string values on its linear/moe fields via _coerce_spec. When the string matches an _ONLINE_SHORTHANDS preset, the preset's corresponding field is copied; if that preset leaves this field None (it does not define a linear or moe spec), this ValueError fires. It means you applied a shorthand that only quantizes the other layer type.

Source

Thrown at vllm/config/quantization.py:107

    """Spec applied to ``LinearBase`` layers."""

    moe: QuantSpec | None = None
    """Spec applied to ``FusedMoEFactory`` layers."""

    ignore: list[str] = Field(default_factory=list)
    """Layers to skip quantization for."""

    @field_validator("linear", "moe", mode="before")
    @classmethod
    def _coerce_spec(cls, v: Any, info: ValidationInfo) -> Any:
        if not isinstance(v, str):
            return v
        field_name = info.field_name
        assert field_name is not None
        if v in _ONLINE_SHORTHANDS:
            spec = getattr(_ONLINE_SHORTHANDS[v], field_name)
            if spec is None:
                raise ValueError(
                    f"online shorthand {v!r} does not define a {field_name} spec"
                )
            return spec
        return QuantSpec(weight=_coerce_quant_key(v))


# CLI shorthands accepted by `--quantization`. Each desugars to a full
# QuantizationConfigArgs; activation overrides go through quantization_config.
_ONLINE_SHORTHANDS: dict[str, QuantizationConfigArgs] = {
    "fp8_per_tensor": QuantizationConfigArgs(
        linear=QuantSpec(weight=kFp8StaticTensorSym),
        moe=QuantSpec(weight=kFp8StaticTensorSym),
    ),
    "fp8_per_block": QuantizationConfigArgs(
        linear=QuantSpec(weight=kFp8Static128BlockSym),
        moe=QuantSpec(weight=kFp8Static128BlockSym),
    ),
    # Per-output-channel weight scale + dynamic per-token activation.

View on GitHub (pinned to c794754062)

Solutions

1. Use the shorthand at the top level (--quantization / the quantization field) so the preset's own field mapping is respected
2. For the field that is None in the preset, supply an explicit QuantKey string (e.g. 'fp8_static_tensor_sym') instead of the shorthand
3. Check _ONLINE_SHORTHANDS in vllm/config/quantization.py to see which fields each preset defines before referencing it per-field

Example fix

# before
QuantizationConfigArgs(moe='some_linear_only_shorthand')

# after
QuantizationConfigArgs(moe='fp8_static_tensor_sym')

from vllm.config.quantization import _ONLINE_SHORTHANDS

def shorthand_covers(field: str, name: str) -> bool:
    return getattr(_ONLINE_SHORTHANDS[name], field) is not None

null

try:
    QuantizationConfigArgs(moe=shorthand)
except ValueError as e:
    if 'does not define a' in str(e):
        spec = 'fp8_static_tensor_sym'  # explicit fallback for that field
    else:
        raise

When it happens

Trigger: Setting QuantizationConfigArgs(moe='<shorthand>') where the shorthand in _ONLINE_SHORTHANDS was built with moe=None (e.g. an activation-only or linear-only preset), or vice versa for linear.

Common situations: Using a new online shorthand that only covers linear layers and applying it to the moe field; assuming every shorthand defines both linear and moe specs.

Related errors

• 'mm_encoder_fp8_scale_path' and 'mm_encoder_fp8_scale_save_p
• unknown quantization name {v!r}; expected one of {sorted(QUA
• quantization_config is only supported when quantization is o
• {model_config.dtype} is not supported for quantization metho
• padded_n is not supported with TRTLLM 8x4 scale layout.

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/dbec789421794b86. Report an issue: GitHub.