sgl-project/sglang · error · ValueError

QuantConfig has static quantization, but found activation sc

Error message

QuantConfig has static quantization, but found activation scales are None.

What it means

The FP8 MoE config declares static input quantization (dynamic == False), but after loading, w13_input_scale or w2_input_scale is None. The fused kernels require a single activation scale, so the missing calibrated scales abort post-load processing.

Source

Thrown at python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_fp8_moe.py:246

            )
            layer.register_parameter("w13_input_scale", w13_input_scale)
            set_weight_attrs(w13_input_scale, extra_weight_attrs)

            w2_input_scale = torch.nn.Parameter(
                torch.ones(num_experts, dtype=torch.float32), requires_grad=False
            )
            layer.register_parameter("w2_input_scale", w2_input_scale)
            set_weight_attrs(w2_input_scale, extra_weight_attrs)
        else:
            layer.w13_input_scale = None
            layer.w2_input_scale = None

    def process_weights_after_loading(self, layer: torch.nn.Module | FusedMoE) -> None:
        # Fp8 moe kernels require a single activation scale.
        # We take the max of all the scales in case they differ.
        if self.static_input_scales:
            if layer.w13_input_scale is None or layer.w2_input_scale is None:
                raise ValueError(
                    "QuantConfig has static quantization, but found "
                    "activation scales are None."
                )
            if not all_close_1d(layer.w13_input_scale) or not all_close_1d(
                layer.w2_input_scale
            ):
                logger.warning(
                    "Found input_scales that are not equal for "
                    "fp8 MoE layer. Using the maximum across experts "
                    "for each layer."
                )
            layer.w13_input_scale = torch.nn.Parameter(
                layer.w13_input_scale.max(), requires_grad=False
            )
            layer.w2_input_scale = torch.nn.Parameter(
                layer.w2_input_scale.max(), requires_grad=False
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Re-run quantization ensuring calibration scales are saved (input_scale keys present in safetensors)
2. Re-quantize with dynamic input quantization instead
3. Inspect the checkpoint files for missing *.input_scale tensors and re-export them

Example fix

# check: python -c "from safetensors import safe_open; ..." verify input_scale keys exist
# fix: re-quantize with llmcompressor static input scales properly exported

from safetensors import safe_open
# verify activation scales exist
keys = set()
for f in shard_files:
    with safe_open(f, framework="pt") as fh: keys.update(fh.keys())
assert any("input_scale" in k for k in keys) or cfg["input_quant"].get("dynamic", True)

When it happens

Trigger: Checkpoint advertises static input scales but the input_scale tensors are absent from the safetensors weights, so layer.w13_input_scale/w2_input_scale are None in process_weights_after_loading.

Common situations: Quantization pipeline that wrote the config but failed to save activation scale tensors; truncated or mixed checkpoints; a conversion script dropping small scale tensors.

Related errors

• The hpc_ops MoE runner backend only supports FP8-quantized M
• For FP8 Fused MoE layer, we require either per tensor or cha
• The output_size of gate's and up's weight = {intermediate_si
• The input_size of down's weight = {intermediate_size_per_par
• Unsupported weight quantization strategy: {self.weight_quant

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b8e777fb416db577. Report an issue: GitHub.