sgl-project/sglang · error · ValueError

The Triton WNA16 MoE backend only supports symmetric INT4 gr

Error message

The Triton WNA16 MoE backend only supports symmetric INT4 group quantization with group_size=32 or 128 and no actorder.

What it means

Raised when the Triton WNA16 MoE backend is explicitly selected (moe_backend=triton) but the checkpoint's quantization is not supported by it. The Triton WNA16 fused-MoE path only accepts symmetric INT4 group quantization with group_size 32 or 128 and no actorder (activation reordering). Any other weight format makes the Triton kernel unable to run.

Source

Thrown at python/sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py:845

                    logger.info_once(
                        "Using CompressedTensorsMxInt4MoE with flashinfer_trtllm backend"
                    )
                    return CompressedTensorsMxInt4MoE(self, weight_quant=weight_quant)
                elif _is_hip:
                    logger.info_once("Using CompressedTensorsWNA16TritonMoE (ROCm)")
                    return CompressedTensorsWNA16TritonMoE(
                        self, weight_quant=weight_quant
                    )
                else:
                    moe_backend = get_moe_runner_backend()
                    triton_supported = self._is_wna16_triton_moe_supported(weight_quant)
                    use_blackwell_triton = (
                        moe_backend.is_auto()
                        and is_sm100_supported()
                        and triton_supported
                    )
                    if moe_backend.is_triton() and not triton_supported:
                        raise ValueError(
                            "The Triton WNA16 MoE backend only supports symmetric "
                            "INT4 group quantization with group_size=32 or 128 and no "
                            "actorder."
                        )
                    if moe_backend.is_triton() or use_blackwell_triton:
                        reason = (
                            "SM100/SM103 auto default"
                            if use_blackwell_triton
                            else "moe_runner_backend=triton"
                        )
                        logger.info_once(
                            f"Using CompressedTensorsWNA16TritonMoE ({reason})"
                        )
                        return CompressedTensorsWNA16TritonMoE(
                            self, weight_quant=weight_quant
                        )
                    logger.info_once("Using CompressedTensorsWNA16MarlinMoEMethod")
                    return CompressedTensorsWNA16MoE(self, weight_quant=weight_quant)

View on GitHub (pinned to 0132848349)

Solutions

1. Remove the explicit triton moe_backend flag (or set auto) so a supported backend is chosen
2. Re-quantize the model to symmetric INT4 group quantization with group_size 32 or 128 and actorder disabled
3. Switch to a different MoE backend that supports the checkpoint's format (e.g. triton kernels default / cutlass)

Example fix

# before
python -m sglang.launch_server --model qwen3-moe-w4a16-g16 --moe-backend-type triton
# after
python -m sglang.launch_server --model qwen3-moe-w4a16-g16

from compressed_tensors import QuantConfig
q = QuantConfig.from_config_dict(model_cfg["quantization_config"])
w = q.get("weights")
ok = (w is not None and w.get("num_bits", 8) == 4 and w.get("strategy") == "group"
      and w.get("group_size") in (32, 128) and not w.get("symmetric") is False
      and not w.get("actorder", False))
assert ok or not args_is_triton_backend, "checkpoint unsupported by Triton WNA16"

When it happens

Trigger: Server args set --moe-backend-type triton (or str_moe_backend=triton) while loading a compressed-tensors MoE model whose weight_quant is not symmetric INT4 group quant with group_size in {32,128}, or whose config has actorder enabled (e.g. W8A16, INT4 with group_size 16, or asymmetric INT4).

Common situations: User forces the Triton MoE backend on a W4A16 model with unsupported group size or an actorder/turn-everything-on compressed-tensors checkpoint; upgrading a model checkpoint format that added actorder; copying server flags from a different model.

Related errors

• fuse_swiglu_interleaved set on an incompatible fused_moe cal
• Unsupported FusedMoe scheme: {weight_quant}, {input_quant}
• For Fused MoE layers, only {CompressionFormat.pack_quantized
• Unsupported weight quantization strategy: {self.weight_quant
• For Fused MoE layers, only {CompressionFormat.pack_quantized

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a5a9144b79b0cc5b. Report an issue: GitHub.