sgl-project/sglang · error · ValueError

Invalid quantization method: {quantization}. Available metho

Error message

Invalid quantization method: {quantization}. Available methods: {list(QUANTIZATION_METHODS.keys())}

What it means

SGLang's quantization registry does not recognize the requested quantization method name. get_quantization_config looks up the string in QUANTIZATION_METHODS (the merged base registry of supported quant schemes) and raises when it is absent, listing all valid names.

Source

Thrown at python/sglang/srt/layers/quantization/__init__.py:164

    )

# subset of above quant methods, supported on CPU
CPU_QUANTIZATION_METHODS = {
    "fp8": Fp8Config,
    "w8a8_int8": W8A8Int8Config,
    "compressed-tensors": CompressedTensorsConfig,
    "awq": AWQCPUConfig,
    "gptq": CPUGPTQConfig,
    "mxfp4": Mxfp4Config,
    "auto-round": AutoRoundConfig,
}

QUANTIZATION_METHODS = {**BASE_QUANTIZATION_METHODS}


def get_quantization_config(quantization: str) -> Type[QuantizationConfig]:
    if quantization not in QUANTIZATION_METHODS:
        raise ValueError(
            f"Invalid quantization method: {quantization}. "
            f"Available methods: {list(QUANTIZATION_METHODS.keys())}"
        )
    from sglang.srt.utils import is_cpu

    if is_cpu() and cpu_has_amx_support():
        if quantization not in CPU_QUANTIZATION_METHODS:
            raise ValueError(
                f"Invalid quantization method on CPU: {quantization}. "
                f"Available methods on CPU: {list(QUANTIZATION_METHODS.keys())}"
            )
        else:
            return CPU_QUANTIZATION_METHODS[quantization]

    if current_platform.is_out_of_tree():
        config = current_platform.get_quantization_config(quantization)

        # If the platform has a quantization config, use it else use the default

View on GitHub (pinned to 0132848349)

Solutions

1. Check the error's printed list and correct the flag to exactly one of the listed methods (e.g. 'awq', 'gptq', 'fp8', 'w8a8_int8')
2. Upgrade sglang to a version that supports the method
3. Install missing backend deps (sgl-kernel, CPU/AMX builds) so the method registers
4. If the checkpoint is quantized with a scheme SGLang lacks, use a compatible checkpoint or the generic 'gptq'/'awq' path

Example fix

# before
python -m sglang.launch_server --model model --quantization awq_marlin_typo
# after
python -m sglang.launch_server --model model --quantization awq_marlin

from sglang.srt.layers.quantization import QUANTIZATION_METHODS
if args.quantization not in QUANTIZATION_METHODS:
    raise SystemExit(f"Pick one of {sorted(QUANTIZATION_METHODS)}")

try:
    cfg = get_quantization_config(name)
except ValueError as e:
    if "Invalid quantization method" in str(e):
        log.error(f"Unknown quant {name}; valid: {list(QUANTIZATION_METHODS)}")
    raise

When it happens

Trigger: Passing --quantization <name> (or ServerArgs.quantization / a config with quantization_config.quant_method) whose string is not a key in QUANTIZATION_METHODS — e.g. a typo like 'aw_q', an uninstalled/optional scheme whose import was skipped, or a method only available in newer SGLang versions.

Common situations: Typos in CLI flags, copy-pasting quant names from vLLM/HF docs that SGLang doesn't support, using a stale SGLang version that lacks a new quant method, or a quant method that requires sgl-kernel/AOT deps not installed so it was never registered.

Related errors

• KernelSpec.target must be 'module:attr', got {self.target!r}
• world_size ({world_size}) is less than tensor_parallel_degre
• kv_cache_quant_config must be QVGKVQuantArgs or a dict
• --quantization nvfp4_online is supported only on NVIDIA Blac
• --quantization nvfp4_online supports only --moe-runner-backe

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0be4c963ff448436. Report an issue: GitHub.