sgl-project/sglang · error · ValueError

Unsupported weight quantization strategy: {self.weight_quant

Error message

Unsupported weight quantization strategy: {self.weight_quant.strategy}

What it means

create_weights for the FP8 MoE scheme supports only TENSOR, CHANNEL (and BLOCK when weight_block_size is set) weight strategies for w13/w2 scale allocation. Any other strategy hits this raise.

Source

Thrown at python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_fp8_moe.py:209

                    num_experts,
                    2 * ((intermediate_size_per_partition + block_n - 1) // block_n),
                    (hidden_size + block_k - 1) // block_k,
                    dtype=torch.float32,
                ),
                requires_grad=False,
            )
            w2_weight_scale = torch.nn.Parameter(
                torch.ones(
                    num_experts,
                    (hidden_size + block_n - 1) // block_n,
                    (intermediate_size_per_partition + block_k - 1) // block_k,
                    dtype=torch.float32,
                ),
                requires_grad=False,
            )
            weight_quant_method = FusedMoeWeightScaleSupported.BLOCK.value
        else:
            raise ValueError(
                f"Unsupported weight quantization strategy: {self.weight_quant.strategy}"
            )

        layer.register_parameter("w13_weight_scale", w13_weight_scale)
        layer.register_parameter("w2_weight_scale", w2_weight_scale)
        # Add the quantization method used (per tensor/grouped/channel)
        # to ensure the weight scales are loaded in properly
        extra_weight_attrs.update({"quant_method": weight_quant_method})
        set_weight_attrs(w13_weight_scale, extra_weight_attrs)
        set_weight_attrs(w2_weight_scale, extra_weight_attrs)

        # INPUT_SCALES
        if self.static_input_scales:
            assert (
                self.input_quant.strategy == QuantizationStrategy.TENSOR
            ), "Only per-tensor quantization is supported for static input scales"
            w13_input_scale = torch.nn.Parameter(
                torch.ones(num_experts, dtype=torch.float32), requires_grad=False

View on GitHub (pinned to 0132848349)

Solutions

1. Inspect quantization_config weights strategy; re-quantize as channel, tensor, or block
2. Upgrade SGLang to a version supporting the new strategy
3. Validate the checkpoint was produced by a compatible llmcompressor version

Example fix

// before: "weights": {"strategy": "group"}
// after: "weights": {"strategy": "channel"}

assert cfg["quantization_config"]["weights"]["strategy"] in {"tensor", "channel", "block"}

When it happens

Trigger: A compressed-tensors FP8 MoE checkpoint whose weight_quant.strategy is something other than tensor/channel/block reaches the scale-allocation else branch.

Common situations: Novel quant strategies from newer llmcompressor versions; corrupted or hand-edited quant configs.

Related errors

• Unsupported weight strategy={self.strategy}, supported strat
• The hpc_ops MoE runner backend only supports FP8-quantized M
• The Triton WNA16 MoE backend only supports symmetric INT4 gr
• Unsupported FusedMoe scheme: {weight_quant}, {input_quant}
• For Fused MoE layers, only {CompressionFormat.pack_quantized

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6d4ba63ebdaf9784. Report an issue: GitHub.