vllm-project/vllm · error · ValueError

Quantization method specified in the model config ({quant_me

Error message

Quantization method specified in the model config ({quant_method}) does not match the quantization method specified in the `quantization` argument ({self.quantization}).

What it means

Error "Quantization method specified in the model config ({quant_method}) does not match the quantization method specified in the `quantization` argument ({self.quantization})." thrown in vllm-project/vllm.

Source

Thrown at vllm/config/model.py:1273

                        name in get_args(me_quant.QuantizationMethods)
                        and name not in overrides
                    ):
                        raise ValueError(
                            f"Quantization method {name} is an override but "
                            "is has not been added to the `overrides` list "
                            "above. This is necessary to ensure that the "
                            "overrides are checked in order of preference."
                        )
                    quant_method = quantization_override
                    self.quantization = quantization_override
                    break

            quant_method = quant_method if quant_method != "" else None
            # Verify quantization configurations.
            if self.quantization is None:
                self.quantization = quant_method
            elif self.quantization != quant_method:
                raise ValueError(
                    "Quantization method specified in the model config "
                    f"({quant_method}) does not match the quantization "
                    f"method specified in the `quantization` argument "
                    f"({self.quantization})."
                )

        if self.quantization is not None:
            if self.quantization not in supported_quantization:
                raise ValueError(
                    f"Unknown quantization method: {self.quantization}. Must "
                    f"be one of {supported_quantization}."
                )
            current_platform.verify_quantization(self.quantization)

        if self.quantization in me_quant.DEPRECATED_QUANTIZATION_METHODS:
            if self.allow_deprecated_quantization:
                logger.warning(
                    "The quantization method %s is deprecated "

View on GitHub (pinned to c794754062)

Solutions

1. Align the `quantization` argument with the quantization method in the model config, or omit the override to use the model's own setting.

When it happens

Trigger: Raised at vllm/config/model.py:1273 when validation fails: Quantization method in model config does not match the `quantization` argument. Typically triggered by an incompatible or incomplete vLLM configuration, an unsupported platform/backend combination, or a runtime resource/dependency that is missing.

Common situations: Commonly encountered at vllm/config/model.py:1273 during vLLM startup/config validation or runtime setup when: (1) conflicting CLI flags or config fields are combined, (2) the current platform (CUDA/ROCm/CPU/XPU) or installed optional packages do not support the requested feature, or (3) a required value is absent or out of range. Resolve by correcting the configuration as described in the message, or by selecting a supported alternative.

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/bcba2e4bca37d4c2. Report an issue: GitHub.