sgl-project/sglang · error · ValueError

Unsupported weight quantization strategy

Error message

Unsupported weight quantization strategy: {self.weight_quant.strategy}

What it means

Raised while parsing a compressed-tensors W8A8 FP8 MoE config when the weight quantization strategy is one SGLang does not implement (only the channel/tensor strategies it supports are accepted). The unsupported strategy value is shown in the message.

Solutions

1. Inspect quantization_config weights strategy; re-quantize as channel, tensor, or block
2. Upgrade SGLang to a version supporting the new strategy
3. Validate the checkpoint was produced by a compatible llmcompressor version

Example fix

// before: "weights": {"strategy": "group"}
// after: "weights": {"strategy": "channel"}

assert cfg["quantization_config"]["weights"]["strategy"] in {"tensor", "channel", "block"}

When it happens

Trigger: A compressed-tensors FP8 MoE checkpoint whose weight_quant.strategy is something other than tensor/channel/block reaches the scale-allocation else branch.

Common situations: Novel quant strategies from newer llmcompressor versions; corrupted or hand-edited quant configs.

Related errors

• Unsupported weight strategy=
• For FP8 Fused MoE layer, we require either per tensor or…
• For Fused MoE layers, only
• For Fused MoE layers, only
• For INT8 Fused MoE layers, we require channelwise, dynamic…

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6d4ba63ebdaf9784. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_fp8_moe.py:209

                    num_experts,
                    2 * ((intermediate_size_per_partition + block_n - 1) // block_n),
                    (hidden_size + block_k - 1) // block_k,
                    dtype=torch.float32,
                ),
                requires_grad=False,
            )
            w2_weight_scale = torch.nn.Parameter(
                torch.ones(
                    num_experts,
                    (hidden_size + block_n - 1) // block_n,
                    (intermediate_size_per_partition + block_k - 1) // block_k,
                    dtype=torch.float32,
                ),
                requires_grad=False,
            )
            weight_quant_method = FusedMoeWeightScaleSupported.BLOCK.value
        else:
            raise ValueError(
                f"Unsupported weight quantization strategy: {self.weight_quant.strategy}"
            )

        layer.register_parameter("w13_weight_scale", w13_weight_scale)
        layer.register_parameter("w2_weight_scale", w2_weight_scale)
        # Add the quantization method used (per tensor/grouped/channel)
        # to ensure the weight scales are loaded in properly
        extra_weight_attrs.update({"quant_method": weight_quant_method})
        set_weight_attrs(w13_weight_scale, extra_weight_attrs)
        set_weight_attrs(w2_weight_scale, extra_weight_attrs)

        # INPUT_SCALES
        if self.static_input_scales:
            assert (
                self.input_quant.strategy == QuantizationStrategy.TENSOR
            ), "Only per-tensor quantization is supported for static input scales"
            w13_input_scale = torch.nn.Parameter(
                torch.ones(num_experts, dtype=torch.float32), requires_grad=False

View on GitHub (pinned to 0132848349)