{"record":{"id":"bacf4cf71dc42e0a","repo":"sgl-project/sglang","slug":"only-cuda-hip-and-xpu-support-awq-currently","errorCode":null,"errorMessage":"Only CUDA, HIP and XPU support AWQ currently.","messagePattern":"Only CUDA, HIP and XPU support AWQ currently\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"python/sglang/srt/layers/quantization/awq/awq.py","lineNumber":53,"sourceCode":"    AWQMoEScheme,\n    AWQXPULinearScheme,\n)\n\nif TYPE_CHECKING:\n    from sglang.srt.layers.moe.token_dispatcher import (\n        CombineInput,\n        StandardDispatchOutput,\n    )\n\nfrom sglang.srt.utils import is_cuda, is_hip, is_npu, is_xpu\n\n_is_cuda = is_cuda()\n_is_hip = is_hip()\n_is_xpu = is_xpu()\n_is_npu = is_npu()\n\nif not (_is_cuda or _is_hip or _is_xpu or _is_npu):\n    warnings.warn(f\"Only CUDA, HIP and XPU support AWQ currently.\")\n\nlogger = logging.getLogger(__name__)\n\n\nScalarType, scalar_types = get_scalar_types()\n\n\ndef is_layer_skipped_awq(prefix: str, modules_to_not_convert: List[str]):\n    return any(module_name in prefix for module_name in modules_to_not_convert)\n\n\nclass AWQConfig(QuantizationConfig):\n    \"\"\"Config class for AWQ.\n\n    Reference: https://arxiv.org/abs/2306.00978\n    \"\"\"\n\n    def __init__(","sourceCodeStart":35,"sourceCodeEnd":71,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/awq/awq.py#L35-L71","documentation":"Plain UserWarning emitted at import of sglang.srt.layers.quantization.awq.awq: AWQ quantization kernels are only implemented for CUDA, HIP, XPU (and NPU). On any other platform (e.g. CPU or unrecognized accelerators) the module warns that AWQ is unsupported.","triggerScenarios":"Importing the AWQ quantization module on a machine where is_cuda(), is_hip(), is_xpu(), and is_npu() are all false — e.g. CPU-only dev boxes or unsupported accelerators.","commonSituations":"Installing sglang on CPU-only hardware; running unit tests or imports on a laptop/CI runner without GPUs; later attempting to load an AWQ checkpoint which will then fail at kernel dispatch.","solutions":["Run on CUDA/HIP/XPU/NPU hardware for AWQ models","Use a non-AWQ checkpoint (e.g. unquantized or a supported quant format) on unsupported platforms","Suppress only if you knowingly import the module without executing AWQ paths"],"exampleFix":"# before\nmodel = AutoModelForCausalLM.from_pretrained(\"x/TheBloke-7B-AWQ\")  # on CPU\n# after\nmodel = AutoModelForCausalLM.from_pretrained(\"x/TheBloke-7B-GPTQ\")  # or run on CUDA","handlingStrategy":"validation","validationCode":"from sglang.utils import is_cuda, is_hip, is_xpu  # or sglang.srt.utils\nif not (is_cuda() or is_hip() or is_xpu()):\n    raise RuntimeError(\"AWQ requires CUDA/HIP/XPU; pick another quantization\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check platform before loading quantized checkpoints","Keep non-quantized fallback checkpoints for CPU dev/test"],"tags":["sglang","awq","quantization","platform-support","hardware"],"backgroundTag":"unsupported-platform","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}