vllm-project/vllm · error · ValueError

quantization_config is only supported when quantization is o

Error message

quantization_config is only supported when quantization is one of {sorted(ONLINE_QUANT_SHORTHAND_NAMES)}, got quantization={quantization!r}

What it means

resolve() combines --quantization (a shorthand that must be one of ONLINE_QUANT_SHORTHAND_NAMES, the keys of _ONLINE_SHORTHANDS) with --quantization-config. If quantization is set to anything other than a known shorthand AND quantization_config is also supplied, the combination is rejected: a non-shorthand quantization value cannot be merged with an overrides dict.

Source

Thrown at vllm/config/quantization.py:172

    "online",
)


def resolve_quantization_config(
    quantization: str | None,
    quantization_config: dict[str, Any] | QuantizationConfigArgs | None,
) -> QuantizationConfigArgs | None:
    """Resolve `--quantization` shorthand and `--quantization-config` into a
    QuantizationConfigArgs.

    `quantization` is a CLI shorthand that desugars into a base config via
    `_ONLINE_SHORTHANDS`. `quantization_config` is a dict or pre-built args
    object. When both are given, fields explicitly set in `quantization_config`
    take precedence over the shorthand.
    """
    if quantization is not None and quantization not in ONLINE_QUANT_SHORTHAND_NAMES:
        if quantization_config is not None:
            raise ValueError(
                f"quantization_config is only supported when quantization is "
                f"one of {sorted(ONLINE_QUANT_SHORTHAND_NAMES)}, "
                f"got quantization={quantization!r}"
            )
        return None

    base = _ONLINE_SHORTHANDS.get(quantization) if quantization else None

    if quantization_config is None:
        return base

    if isinstance(quantization_config, dict):
        quantization_config = QuantizationConfigArgs(**quantization_config)

    if base is None:
        return quantization_config

    return QuantizationConfigArgs(

View on GitHub (pinned to c794754062)

Solutions

1. Use an exact ONLINE_QUANT_SHORTHAND_NAMES entry for --quantization (the message lists them)
2. Or drop --quantization-config and put the full desired spec inside quantization_config alone
3. Or drop --quantization and rely solely on quantization_config
4. If the value comes from a checkpoint's quant_method, route it through the checkpoint path, not the shorthand resolver

Example fix

# before
--quantization fp8 --quantization-config '{"linear": ...}'

# after
--quantization fp8_per_tensor --quantization-config '{"linear": ...}'

from vllm.config.quantization import ONLINE_QUANT_SHORTHAND_NAMES

def combinable(quantization: str | None, qcfg: object) -> bool:
    return quantization is None or quantization in ONLINE_QUANT_SHORTHAND_NAMES or qcfg is None

null

null

When it happens

Trigger: Running with --quantization fp8 --quantization-config '{\"k\": v}' where 'fp8' is not a key of _ONLINE_SHORTHANDS (the registered shorthand is e.g. fp8_per_tensor); or passing an arbitrary/checkpoint-derived quantization string together with a config dict programmatically.

Common situations: Using legacy quantization names ('fp8', 'awq') that predate the shorthand registry; passing quantization_config while leaving quantization set from a previous config file; version change that renamed shorthands.

Related errors

• 'mm_encoder_fp8_scale_path' and 'mm_encoder_fp8_scale_save_p
• unknown quantization name {v!r}; expected one of {sorted(QUA
• online shorthand {v!r} does not define a {field_name} spec
• {model_config.dtype} is not supported for quantization metho
• Hardware {requested!r} is not available for this model. Avai

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/3f6685a8271af3e2. Report an issue: GitHub.