{"record":{"id":"a5a9144b79b0cc5b","repo":"sgl-project/sglang","slug":"the-triton-wna16-moe-backend-only-supports-symmetr","errorCode":null,"errorMessage":"The Triton WNA16 MoE backend only supports symmetric INT4 group quantization with group_size=32 or 128 and no actorder.","messagePattern":"The Triton WNA16 MoE backend only supports symmetric INT4 group quantization with group_size=32 or 128 and no actorder\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py","lineNumber":845,"sourceCode":"                    logger.info_once(\n                        \"Using CompressedTensorsMxInt4MoE with flashinfer_trtllm backend\"\n                    )\n                    return CompressedTensorsMxInt4MoE(self, weight_quant=weight_quant)\n                elif _is_hip:\n                    logger.info_once(\"Using CompressedTensorsWNA16TritonMoE (ROCm)\")\n                    return CompressedTensorsWNA16TritonMoE(\n                        self, weight_quant=weight_quant\n                    )\n                else:\n                    moe_backend = get_moe_runner_backend()\n                    triton_supported = self._is_wna16_triton_moe_supported(weight_quant)\n                    use_blackwell_triton = (\n                        moe_backend.is_auto()\n                        and is_sm100_supported()\n                        and triton_supported\n                    )\n                    if moe_backend.is_triton() and not triton_supported:\n                        raise ValueError(\n                            \"The Triton WNA16 MoE backend only supports symmetric \"\n                            \"INT4 group quantization with group_size=32 or 128 and no \"\n                            \"actorder.\"\n                        )\n                    if moe_backend.is_triton() or use_blackwell_triton:\n                        reason = (\n                            \"SM100/SM103 auto default\"\n                            if use_blackwell_triton\n                            else \"moe_runner_backend=triton\"\n                        )\n                        logger.info_once(\n                            f\"Using CompressedTensorsWNA16TritonMoE ({reason})\"\n                        )\n                        return CompressedTensorsWNA16TritonMoE(\n                            self, weight_quant=weight_quant\n                        )\n                    logger.info_once(\"Using CompressedTensorsWNA16MarlinMoEMethod\")\n                    return CompressedTensorsWNA16MoE(self, weight_quant=weight_quant)","sourceCodeStart":827,"sourceCodeEnd":863,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py#L827-L863","documentation":"Raised when the Triton WNA16 MoE backend is explicitly selected (moe_backend=triton) but the checkpoint's quantization is not supported by it. The Triton WNA16 fused-MoE path only accepts symmetric INT4 group quantization with group_size 32 or 128 and no actorder (activation reordering). Any other weight format makes the Triton kernel unable to run.","triggerScenarios":"Server args set --moe-backend-type triton (or str_moe_backend=triton) while loading a compressed-tensors MoE model whose weight_quant is not symmetric INT4 group quant with group_size in {32,128}, or whose config has actorder enabled (e.g. W8A16, INT4 with group_size 16, or asymmetric INT4).","commonSituations":"User forces the Triton MoE backend on a W4A16 model with unsupported group size or an actorder/turn-everything-on compressed-tensors checkpoint; upgrading a model checkpoint format that added actorder; copying server flags from a different model.","solutions":["Remove the explicit triton moe_backend flag (or set auto) so a supported backend is chosen","Re-quantize the model to symmetric INT4 group quantization with group_size 32 or 128 and actorder disabled","Switch to a different MoE backend that supports the checkpoint's format (e.g. triton kernels default / cutlass)"],"exampleFix":"# before\npython -m sglang.launch_server --model qwen3-moe-w4a16-g16 --moe-backend-type triton\n# after\npython -m sglang.launch_server --model qwen3-moe-w4a16-g16","handlingStrategy":"validation","validationCode":"from compressed_tensors import QuantConfig\nq = QuantConfig.from_config_dict(model_cfg[\"quantization_config\"])\nw = q.get(\"weights\")\nok = (w is not None and w.get(\"num_bits\", 8) == 4 and w.get(\"strategy\") == \"group\"\n      and w.get(\"group_size\") in (32, 128) and not w.get(\"symmetric\") is False\n      and not w.get(\"actorder\", False))\nassert ok or not args_is_triton_backend, \"checkpoint unsupported by Triton WNA16\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Only pass --moe-backend-type triton for symmetric INT4 group-32/128 checkpoints without actorder","Prefer the auto backend so sglang picks a compatible path","Check group_size and actorder fields in the checkpoint's quantization_config before launching"],"tags":["quantization","moe","triton","int4","compressed-tensors"],"backgroundTag":"unsupported-quantization-config","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}