{"record":{"id":"4ab5cdf40a03eabb","repo":"sgl-project/sglang","slug":"invalid-quantization-method-on-cpu-quantization","errorCode":null,"errorMessage":"Invalid quantization method on CPU: {quantization}. Available methods on CPU: {list(QUANTIZATION_METHODS.keys())}","messagePattern":"Invalid quantization method on CPU: (.+?)\\. Available methods on CPU: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/__init__.py","lineNumber":172,"sourceCode":"    \"gptq\": CPUGPTQConfig,\n    \"mxfp4\": Mxfp4Config,\n    \"auto-round\": AutoRoundConfig,\n}\n\nQUANTIZATION_METHODS = {**BASE_QUANTIZATION_METHODS}\n\n\ndef get_quantization_config(quantization: str) -> Type[QuantizationConfig]:\n    if quantization not in QUANTIZATION_METHODS:\n        raise ValueError(\n            f\"Invalid quantization method: {quantization}. \"\n            f\"Available methods: {list(QUANTIZATION_METHODS.keys())}\"\n        )\n    from sglang.srt.utils import is_cpu\n\n    if is_cpu() and cpu_has_amx_support():\n        if quantization not in CPU_QUANTIZATION_METHODS:\n            raise ValueError(\n                f\"Invalid quantization method on CPU: {quantization}. \"\n                f\"Available methods on CPU: {list(QUANTIZATION_METHODS.keys())}\"\n            )\n        else:\n            return CPU_QUANTIZATION_METHODS[quantization]\n\n    if current_platform.is_out_of_tree():\n        config = current_platform.get_quantization_config(quantization)\n\n        # If the platform has a quantization config, use it else use the default\n        if config is not None:\n            return config\n\n    return QUANTIZATION_METHODS[quantization]\n\n\noriginal_isinstance = builtins.isinstance\n","sourceCodeStart":154,"sourceCodeEnd":190,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/__init__.py#L154-L190","documentation":"On a CPU machine with Intel AMX support, SGLang restricts quantization to the CPU-allowed subset (CPU_QUANTIZATION_METHODS) and raises when the requested method is not in it. This is a platform gate, not a registry miss — the method exists but is not supported on CPU+AMX.","triggerScenarios":"Running sglang on an AMX-capable CPU host with --quantization set to a GPU-only or non-CPU-supported scheme (e.g. fp8, some marlin variants), so is_cpu() and cpu_has_amx_support() are true but the name is not in CPU_QUANTIZATION_METHODS.","commonSituations":"Porting a GPU deployment config to CPU boxes, running CPU CI on laptop/workstation AMX hardware, or forgetting to change the quantization flag when switching backends.","solutions":["Use one of the CPU-supported methods printed in the message (e.g. 'auto_round' 4-bit, w8a8 on CPU where supported)","Remove the --quantization flag and let the config be inferred if the checkpoint is CPU-compatible","Run on a GPU machine if you need the GPU-only scheme","Check the CPU_QUANTIZATION_METHODS dict in the same file for the exact current list"],"exampleFix":"# before\n--quantization fp8   # on AMX CPU\n# after\n--quantization auto_round","handlingStrategy":"validation","validationCode":"from sglang.srt.utils import is_cpu\nfrom sglang.srt.layers.quantization import CPU_QUANTIZATION_METHODS\nif is_cpu() and q not in CPU_QUANTIZATION_METHODS:\n    q = \"auto_round\"  # or fail fast with a clear message","typeGuard":null,"tryCatchPattern":"try:\n    get_quantization_config(q)\nexcept ValueError as e:\n    if \"on CPU\" in str(e):\n        switch_to_gpu_backend_or_cpu_supported_quant()","preventionTips":["Branch deployment configs by hardware (CPU vs GPU) with distinct quant flags","Document the CPU-supported quant list next to CPU deployment runbooks"],"tags":["quantization","cpu","amx","platform-support"],"backgroundTag":"unsupported-platform-feature","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}