sgl-project/sglang · error · ValueError

Unknown quantization method: {self.quantization}. Must be on

Error message

Unknown quantization method: {self.quantization}. Must be one of {supported_quantization}.

What it means

The --quantization string is not in sglang's supported_quantization list, so _verify_quantization rejects it before model loading. Common with typos, renamed methods, or method names from other frameworks (e.g. vLLM-specific spellings).

Source

Thrown at python/sglang/srt/configs/model_config.py:1647

                        f"main model quantization ({self.quantization}). "
                        f"Using draft model's detected quantization: {quant_method}"
                    )
                    self.quantization = quant_method
                elif self.quantization in REQUANTIZATION_METHODS:
                    logger.info_once(
                        f"Requantizing from quant_method='{quant_method}' to the requested online quantization='{self.quantization}'. Beware that requantization may incur a loss in accuracy, the requantized model should be re-validated/re-evaluated. More details at https://docs.sglang.io/advanced_features/quantization.html#online-quantization."
                    )
                else:
                    raise ValueError(
                        "Quantization method specified in the model config "
                        f"({quant_method}) does not match the quantization "
                        f"method specified in the `quantization` argument "
                        f"({self.quantization})."
                    )

        if self.quantization is not None:
            if self.quantization not in supported_quantization:
                raise ValueError(
                    f"Unknown quantization method: {self.quantization}. Must "
                    f"be one of {supported_quantization}."
                )
            if is_hip() and self.quantization not in rocm_supported_quantization:
                raise ValueError(
                    f"{self.quantization} quantization is currently not "
                    f"supported in ROCm."
                )
            if self.quantization not in optimized_quantization_methods:
                # Don't warn for MXFP4/MXFP8 on SM100 since they have optimized kernels
                if not (
                    self.quantization in ["mxfp4", "mxfp8"] and is_sm100_supported()
                ):
                    logger.warning(
                        "%s quantization is not fully "
                        "optimized yet. The speed can be slower than "
                        "non-quantized models.",
                        self.quantization,

View on GitHub (pinned to 0132848349)

Solutions

1. Check the error message's supported list and correct the spelling/casing
2. Upgrade or align your sglang version with the docs for the method you want
3. If the method should exist, verify you're on a build that includes it (not a CPU-only or trimmed wheel)

Example fix

# before
python -m sglang.launch_server --model MODEL --quantization w8a8_int8
# after
python -m sglang.launch_server --model MODEL --quantization int8_w8a8  # exact supported name

from sglang.srt.layers.quantization import QUANTIZATION_METHODS  # or the supported list surfaced by the error
if requested_quant not in QUANTIZATION_METHODS:
    raise SystemExit(f"unsupported quantization {requested_quant}; pick from {sorted(QUANTIZATION_METHODS)}")

When it happens

Trigger: Passing --quantization <name> where name is misspelled (e.g. 'gptq_marlin ' with trailing space, 'int8_w8a8' in a version that renamed it) or simply not implemented in this build.

Common situations: Typos in launch scripts, upgrading sglang where quantization names were added/removed, using a name valid in vLLM/llama.cpp but not sglang.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• --quantization nvfp4_online is supported only on NVIDIA Blac
• --quantization nvfp4_online supports only --moe-runner-backe
• quantize_and_serve requires ModelOpt quantization (set with
• quantize_and_serve functionality is currently disabled due t
• Config list contains configs from 2 methods, must be only 1

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4c1f11442395f114. Report an issue: GitHub.