{"record":{"id":"a92299306eaf4f2c","repo":"sgl-project/sglang","slug":"the-quantization-method-moe-wna16-awq-is-not-sup","errorCode":null,"errorMessage":"The quantization method moe_wna16 + awq is not supported for the current GPU. Minimum capability: {awq_min_capability}. Current capability: {device_capability}.","messagePattern":"The quantization method moe_wna16 \\+ awq is not supported for the current GPU\\. Minimum capability: (.+?)\\. Current capability: (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/moe_wna16.py","lineNumber":103,"sourceCode":"        self.bit8_pack_factor = 8 // self.weight_bits\n        self.lm_head_quantized = lm_head_quantized\n        self.linear_quant_method = linear_quant_method\n        self.full_config = full_config\n        self.use_marlin = False\n        # Avoid circular import\n\n        if self.linear_quant_method == \"gptq\":\n            self.use_marlin = GPTQMarlinConfig.is_gptq_marlin_compatible(full_config)\n        elif self.linear_quant_method == \"awq\":\n            capability_tuple = get_device_capability()\n            device_capability = (\n                -1\n                if capability_tuple is None\n                else capability_tuple[0] * 10 + capability_tuple[1]\n            )\n            awq_min_capability = AWQConfig.get_min_capability()\n            if device_capability < awq_min_capability:\n                raise ValueError(\n                    \"The quantization method moe_wna16 + awq is not supported \"\n                    \"for the current GPU. \"\n                    f\"Minimum capability: {awq_min_capability}. \"\n                    f\"Current capability: {device_capability}.\"\n                )\n        else:\n            raise ValueError(\"moe_wna16 only support gptq and awq.\")\n\n        if modules_to_not_convert is None:\n            self.modules_to_not_convert = []\n        else:\n            self.modules_to_not_convert = modules_to_not_convert\n\n    @classmethod\n    def get_name(cls) -> str:\n        return \"moe_wna16\"\n\n    @classmethod","sourceCodeStart":85,"sourceCodeEnd":121,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/moe_wna16.py#L85-L121","documentation":"The moe_wna16 + AWQ path uses the AWQ marlin kernel, which requires a minimum GPU compute capability (typically SM 75 / Turing and above). At init the server compares the current device's capability against AWQConfig.get_min_capability() and refuses to run when the GPU is too old.","triggerScenarios":"Loading an AWQ-quantized MoE model (moe_wna16 with quant_method 'awq') on a GPU with compute capability below the AWQ minimum — e.g. a Pascal (SM 60, e.g. GTX 1080, P100) or Volta (SM 70) card.","commonSituations":"Running on older datacenter GPUs (P100/V100), some cloud instances (AWS P2/P3, older K80), or a GPU whose capability cannot be detected so it defaults to -1; the latter makes any card fail the comparison.","solutions":["Run on a Turing or newer GPU (T4, A100, L4, H100, RTX 20xx+ or newer)","If the capability is being reported as -1 because the device couldn't be queried (non-CUDA visible device, MIG misconfiguration), fix GPU visibility (CUDA_VISIBLE_DEVICES) so torch.cuda.get_device_capability succeeds","Use a non-AWQ quantization format supported on your GPU (e.g. GPTQ marlin, or unquantized weights)"],"exampleFix":"# before: AWQ MoE on GTX 1080\npython -m sglang.launch_server --model some-model-awq-moe\n# after\nCUDA_VISIBLE_DEVICES=1 python -m sglang.launch_server --model some-model-awq-moe  # on a T4/A100+ GPU","handlingStrategy":"validation","validationCode":"import torch\ncap = torch.cuda.get_device_capability()\nmajor, minor = cap if cap else (-1, -1)\ndevice_cap = major * 10 + minor\nif device_cap < 75:  # AWQ marlin minimum\n    raise RuntimeError(f\"GPU SM {device_cap} too old for AWQ moe_wna16\")","typeGuard":null,"tryCatchPattern":"try:\n    server_args = ServerArgs(model=path, ...)\nexcept ValueError as e:\n    if \"capability\" in str(e):\n        # fall back to a GPTQ or unquantized checkpoint on this GPU\n        ...\n    raise","preventionTips":["Check torch.cuda.get_device_capability() before loading AWQ MoE models","Ensure CUDA_VISIBLE_DEVICES points at a real device so capability detection succeeds","Keep a Turing+ GPU for AWQ marlin workloads"],"tags":["quantization","awq","moe","gpu-capability","hardware"],"backgroundTag":"gpu-compute-capability-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}