sgl-project/sglang · error · ValueError

Invalid quantization method on CPU: {quantization}. Availabl

Error message

Invalid quantization method on CPU: {quantization}. Available methods on CPU: {list(QUANTIZATION_METHODS.keys())}

What it means

On a CPU machine with Intel AMX support, SGLang restricts quantization to the CPU-allowed subset (CPU_QUANTIZATION_METHODS) and raises when the requested method is not in it. This is a platform gate, not a registry miss — the method exists but is not supported on CPU+AMX.

Source

Thrown at python/sglang/srt/layers/quantization/__init__.py:172

    "gptq": CPUGPTQConfig,
    "mxfp4": Mxfp4Config,
    "auto-round": AutoRoundConfig,
}

QUANTIZATION_METHODS = {**BASE_QUANTIZATION_METHODS}


def get_quantization_config(quantization: str) -> Type[QuantizationConfig]:
    if quantization not in QUANTIZATION_METHODS:
        raise ValueError(
            f"Invalid quantization method: {quantization}. "
            f"Available methods: {list(QUANTIZATION_METHODS.keys())}"
        )
    from sglang.srt.utils import is_cpu

    if is_cpu() and cpu_has_amx_support():
        if quantization not in CPU_QUANTIZATION_METHODS:
            raise ValueError(
                f"Invalid quantization method on CPU: {quantization}. "
                f"Available methods on CPU: {list(QUANTIZATION_METHODS.keys())}"
            )
        else:
            return CPU_QUANTIZATION_METHODS[quantization]

    if current_platform.is_out_of_tree():
        config = current_platform.get_quantization_config(quantization)

        # If the platform has a quantization config, use it else use the default
        if config is not None:
            return config

    return QUANTIZATION_METHODS[quantization]


original_isinstance = builtins.isinstance

View on GitHub (pinned to 0132848349)

Solutions

1. Use one of the CPU-supported methods printed in the message (e.g. 'auto_round' 4-bit, w8a8 on CPU where supported)
2. Remove the --quantization flag and let the config be inferred if the checkpoint is CPU-compatible
3. Run on a GPU machine if you need the GPU-only scheme
4. Check the CPU_QUANTIZATION_METHODS dict in the same file for the exact current list

Example fix

# before
--quantization fp8   # on AMX CPU
# after
--quantization auto_round

from sglang.srt.utils import is_cpu
from sglang.srt.layers.quantization import CPU_QUANTIZATION_METHODS
if is_cpu() and q not in CPU_QUANTIZATION_METHODS:
    q = "auto_round"  # or fail fast with a clear message

try:
    get_quantization_config(q)
except ValueError as e:
    if "on CPU" in str(e):
        switch_to_gpu_backend_or_cpu_supported_quant()

When it happens

Trigger: Running sglang on an AMX-capable CPU host with --quantization set to a GPU-only or non-CPU-supported scheme (e.g. fp8, some marlin variants), so is_cpu() and cpu_has_amx_support() are true but the name is not in CPU_QUANTIZATION_METHODS.

Common situations: Porting a GPU deployment config to CPU boxes, running CPU CI on laptop/workstation AMX hardware, or forgetting to change the quantization flag when switching backends.

Related errors

• SGLang's AutoRound CPU inference path currently supports onl
• Currently, only 4bits is supported on CPU with AMX.
• The input size is not aligned with the quantized weight shap
• Nunchaku SVDQuant is only supported on NVIDIA CUDA GPUs (Amp
• Serialized W4A4 checkpoints are not supported on MPS

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4ab5cdf40a03eabb. Report an issue: GitHub.