sgl-project/sglang · error · ValueError

Unknown quantization strategy {self.strategy}

Error message

Unknown quantization strategy {self.strategy}

What it means

In process_weights_after_loading of the W8A8 FP8 scheme, only CHANNEL and TENSOR weight strategies have repacking branches; any other strategy reaches the fallthrough raise. It means the checkpoint's weight layout does not match a scheme the post-load path can process.

Source

Thrown at python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_fp8.py:220

                layer.weight_scale.requires_grad_(False)

            # On Blackwell, block-FP8 dispatches to DeepGEMM, which needs the
            # weight scales UE8M0-packed to match its UE8M0 activation scales.
            use_deepgemm_runner = (
                self.w8a8_block_fp8_linear
                is deepgemm_w8a8_block_fp8_linear_with_fallback
            )
            requant_block_scale_ue8m0_for_deepgemm(
                layer.weight,
                layer.weight_scale,
                self.weight_block_size,
                use_deepgemm_runner=use_deepgemm_runner,
                output_dtype=getattr(layer, "orig_dtype", None),
                weight_shape=layer.weight.shape,
            )

        else:
            raise ValueError(f"Unknown quantization strategy {self.strategy}")

        # INPUT SCALE
        if self.is_static_input_scheme and hasattr(layer, "input_scale"):
            layer.input_scale = Parameter(layer.input_scale.max(), requires_grad=False)
        else:
            layer.input_scale = None

    def apply_weights(
        self,
        layer: torch.nn.Module,
        x: torch.Tensor,
        bias: Optional[torch.Tensor] = None,
    ) -> torch.Tensor:
        if isinstance(x, tuple):
            # Pre-quantized activation from a fused RMSNorm+FP8 quant kernel:
            # x = (fp8_input, per_tensor_input_scale[, orig_dtype]).
            # apply_fp8_linear detects the fp8 dtype and skips re-quantizing;
            # orig_dtype (when present) sets the GEMM output dtype.

View on GitHub (pinned to 0132848349)

Solutions

1. Check quantization_config.weights.strategy in the checkpoint; use channel or tensor
2. If block quantization is desired, upgrade SGLang so the block-quantized FP8 path handles it
3. Re-quantize the model with a recipe matching W8A8 (channel weights + token dynamic activations)

assert cfg["quantization_config"]["weights"]["strategy"] in ("channel", "tensor", "block")

When it happens

Trigger: Loading a compressed-tensors W8A8 model whose weight strategy is BLOCK or GROUP while the scheme was constructed for fp8 dynamic/static linear layers, then process_weights_after_loading dispatches on self.strategy.

Common situations: Mixed recipes (block weights + token activations) slipping through init checks; newer quant formats not understood by the installed SGLang version.

Related errors

• Unsupported weight strategy={self.strategy}, supported strat
• Unsupported weight quantization strategy: {self.weight_quant
• Unknown quantization strategy {self.strategy}
• kv_scales supplied but unified_kv is {unified_kv.dtype}, exp
• int32-packed scale buffers require scale_ue8m0=True

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/afe0e4e5b61a2331. Report an issue: GitHub.