sgl-project/sglang · error · ValueError

Unknown quantization strategy {self.strategy}

Error message

Unknown quantization strategy {self.strategy}

What it means

The W8A8 INT8 scheme's post-load path handles only per-tensor and per-channel/token weight strategies; an unrecognized strategy falls through to this raise instead of silently producing wrong weights.

Source

Thrown at python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_int8.py:126

                weight=layer.weight,
                weight_scale=layer.weight_scale,
                logical_widths=layer.logical_widths,
            )

            layer.weight = Parameter(weight.t(), requires_grad=False)
            layer.weight_scale = Parameter(max_w_scale, requires_grad=False)

        # If channelwise, scales are already lined up, so just transpose.
        elif self.strategy == QuantizationStrategy.CHANNEL:
            weight = layer.weight
            weight_scale = layer.weight_scale.data

            layer.weight = Parameter(weight.t(), requires_grad=False)
            # required by torch.compile to be torch.nn.Parameter
            layer.weight_scale = Parameter(weight_scale, requires_grad=False)

        else:
            raise ValueError(f"Unknown quantization strategy {self.strategy}")

        # INPUT SCALE
        if self.is_static_input_scheme and hasattr(layer, "input_scale"):
            if self.input_symmetric:
                layer.input_scale = Parameter(
                    layer.input_scale.max(), requires_grad=False
                )
            else:
                input_scale = layer.input_scale
                input_zero_point = layer.input_zero_point

                # reconstruct the ranges
                int8_traits = torch.iinfo(torch.int8)
                azps = input_zero_point.to(dtype=torch.int32)
                range_max = (input_scale * (int8_traits.max - azps)).max()
                range_min = (input_scale * (int8_traits.min - azps)).min()

                scale = (range_max - range_min) / (int8_traits.max - int8_traits.min)

View on GitHub (pinned to 0132848349)

Solutions

1. Re-quantize with channelwise or per-tensor INT8 weights
2. Upgrade SGLang if a newer INT8 strategy should be supported
3. Verify quantization_config weights strategy string

assert cfg["quantization_config"]["weights"]["strategy"] in ("tensor", "channel")

When it happens

Trigger: A compressed-tensors INT8 checkpoint with a weight strategy other than TENSOR/CHANNEL reaching process_weights_after_loading in compressed_tensors_w8a8_int8.py.

Common situations: Group-wise INT8 quant configs unsupported by this scheme; version skew between checkpoint producer and runtime.

Related errors

• Unknown quantization strategy {self.strategy}
• For INT8 Fused MoE layers, we require channelwise, dynamic p
• Comfy INT8 embedding weights support lookup only
• Quanto tensor/map prefixes do not match: missing metadata={s
• Unsupported Quanto weight type for {prefix!r}: {quantization

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5b1ad423221e4e6a. Report an issue: GitHub.