sgl-project/sglang · critical · RuntimeError

Unsupported FusedMoe scheme: {weight_quant}, {input_quant}

Error message

Unsupported FusedMoe scheme: {weight_quant}, {input_quant}

What it means

The MoE layer's (weight_quant, input_quant) combination matched none of the recognized compressed-tensors MoE schemes (FP8, dynamic W8A8, W4AFP8, dynamic W4A8, WNA16, etc.), so get_moe_scheme raises RuntimeError listing the offending quant configs.

Source

Thrown at python/sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py:901

                    "The W8A8Int8 Fused MoE scheme is implemented only for NPU for now."
                )
        elif self._is_wint4afp8(weight_quant, input_quant):
            # On NPU prefer the dedicated NPU W4A8Int8 path when activations are INT8.
            if _is_npu and self._is_dynamic_token_w4a8(weight_quant, input_quant):
                logger.info_once("Using NPUCompressedTensorsW4A8Int8DynamicMoE")
                return NPUCompressedTensorsW4A8Int8DynamicMoE(self)
            logger.info_once("Using CompressedTensorsW4AFP8MoE")
            return CompressedTensorsW4AFP8MoE(self, weight_quant, input_quant)
        elif self._is_dynamic_token_w4a8(weight_quant, input_quant):
            if _is_npu:
                logger.info_once("Using NPUCompressedTensorsW4A8Int8DynamicMoE")
                return NPUCompressedTensorsW4A8Int8DynamicMoE(self)
            else:
                raise NotImplementedError(
                    "The W4A8Int8 Fused MoE scheme is implemented only for NPU for now."
                )
        else:
            raise RuntimeError(
                f"Unsupported FusedMoe scheme: {weight_quant}, {input_quant}"
            )

    def get_linear_scheme(
        self,
        layer: torch.nn.Module,
        layer_name: Optional[str] = None,
        matched_target: Optional[str] = None,
    ) -> Optional[CompressedTensorsLinearScheme]:
        """
        compressed-tensors supports non uniform in the following way:

        targets of config_groups: There can be N config_groups which each
            have a quantization scheme. Each config_group has a list of targets
            which can be a full layer_name, a regex for a layer_name, or
            an nn.Module name.

        Detect whether a layer_name is found in any target and

View on GitHub (pinned to 0132848349)

Solutions

1. Print/inspect target_scheme_map for the MoE layer to see the actual (weight_quant, input_quant) and compare against supported schemes in get_moe_scheme
2. Re-quantize the model with a scheme sglang supports (FP8, W8A8, W4A16/WNA16, INT4 group)
3. Upgrade sglang to a version supporting the new scheme
4. Exclude MoE layers from quantization (ignore= model.experts) so they run unquantized

supported = {"FP8","INT8 (static)","dynamic INT8","W4A8FP8","dynamic W4A8INT8","WNA16"}
print(model_cfg["quantization_config"]["config"])
# manually compare weights/input_activations combos against sglang get_moe_scheme branches

try:
    model = sglang.Engine(model_path=...)
except RuntimeError as e:
    if "Unsupported FusedMoe scheme" in str(e):
        # log (weight_quant, input_quant) and fall back to unquantized/gptq checkpoint
        ...

When it happens

Trigger: Loading a compressed-tensors checkpoint whose MoE expert layers use an exotic or unrecognized quantization combo, e.g. new llmcompressor scheme versions, unsupported bit-widths, or unusual sparsity/scale structures; also when MoE layers exist but quant config has no weights scheme at all.

Common situations: Quantizing with a newer llmcompressor version producing scheme strings sglang doesn't parse; partially-quantized or custom-recipe checkpoints; version mismatch between checkpoint creation and sglang release.

Related errors

• The Triton WNA16 MoE backend only supports symmetric INT4 gr
• For Fused MoE layers, only {CompressionFormat.pack_quantized
• Unsupported weight quantization strategy: {self.weight_quant
• For Fused MoE layers, only {CompressionFormat.pack_quantized
• Unsupported ModelSlim MoE schemes for layer {prefix}: W13='{

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/054dff2955510128. Report an issue: GitHub.