sgl-project/sglang · error · ValueError

For Fused MoE layers, only {CompressionFormat.pack_quantized

Error message

For Fused MoE layers, only {CompressionFormat.pack_quantized.value} is supported for the following bits: {WNA16_SUPPORTED_BITS}

What it means

For wNa16 (e.g. W4A16) Fused MoE layers, the checkpoint must use CompressionFormat.pack-quantized and a supported bit width (WNA16_SUPPORTED_BITS). Any other format or bit combination is rejected when building the MoE scheme.

Source

Thrown at python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_wNa16_moe.py:88

        weight_quant: QuantizationArgs,
        num_gpu_experts: int = -1,
    ):
        self.quant_config = quant_config
        # Per-layer scheme already resolved by get_moe_scheme(); reuse it directly
        # (mixed-precision MoE has no "Linear" config group to fall back on).
        config = weight_quant
        self.num_bits = config.num_bits
        self.packed_factor = 32 // config.num_bits
        self.strategy = config.strategy
        self.group_size = config.group_size
        self.actorder = config.actorder
        self.sym = config.symmetric

        if not (
            self.quant_config.quant_format == CompressionFormat.pack_quantized.value
            and self.num_bits in WNA16_SUPPORTED_BITS
        ):
            raise ValueError(
                "For Fused MoE layers, only ",
                f"{CompressionFormat.pack_quantized.value} ",
                "is supported for the following bits: ",
                f"{WNA16_SUPPORTED_BITS}",
            )
        self.num_gpu_experts = num_gpu_experts

    @classmethod
    def get_min_capability(cls) -> int:
        # ampere and up
        return 80

    def create_weights(
        self,
        layer: torch.nn.Module,
        num_experts: int,
        hidden_size: int,
        intermediate_size_per_partition: int,

View on GitHub (pinned to 0132848349)

Solutions

1. Re-export with format pack-quantized and supported bits (4)
2. Verify quantization_config.format in config.json
3. Use a non-MoE path or unquantized model if the format can't change

Example fix

// before
"quantization_config": {"format": "int-quantized", "weights": {"num_bits": 4}}
// after
"quantization_config": {"format": "pack-quantized", "weights": {"num_bits": 4}}

assert cfg["quantization_config"]["format"] == "pack-quantized" and cfg["weights"]["num_bits"] in (4,)

When it happens

Trigger: A GPTQ/AWQ-quantized MoE model exported in a non-packed format (or unsupported num_bits) dispatched to CompressedTensorsWNA16MoE.__init__.

Common situations: Quantizing MoE models with llmcompressor but exporting 'int-quantized' format instead of packed; mixed-format checkpoints.

Related errors

• The Triton WNA16 MoE backend only supports symmetric INT4 gr
• Unsupported FusedMoe scheme: {weight_quant}, {input_quant}
• For Fused MoE layers, only {CompressionFormat.pack_quantized
• Unsupported weight quantization strategy: {self.weight_quant
• num_bits must be 4 or 8, got {}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a29fb9ecea99e08f. Report an issue: GitHub.