sgl-project/sglang · error · ValueError

The block-wise quantization only supports dynamic activation

Error message

The block-wise quantization only supports dynamic activation scheme for now, but got {activation_scheme} activation scheme.

What it means

Block-wise FP8 quantization in SGLang only implements dynamic activation scaling (activations are quantized per-token at runtime); static per-tensor activation scales cannot be combined with weight block scales. Fp8LinearConfig raises when weight_block_size is set but activation_scheme is "static" (or anything not "dynamic").

Source

Thrown at python/sglang/srt/layers/quantization/fp8.py:272

                    layer.strip()
                    for layer in ignored_layers_str.split(",")
                    if layer.strip()
                ]
            )
        self.packed_modules_mapping = packed_modules_mapping or {}
        self.use_mxfp8 = use_mxfp8
        self.kv_cache_quant_algo = kv_cache_quant_algo
        if weight_block_size is not None:
            if not is_checkpoint_fp8_serialized:
                raise ValueError(
                    "The block-wise quantization only supports fp8-serialized checkpoint for now."
                )
            if len(weight_block_size) != 2:
                raise ValueError(
                    f"The quantization block size of weight must have 2 dimensions, but got {len(weight_block_size)} dimensions."
                )
            if activation_scheme != "dynamic":
                raise ValueError(
                    f"The block-wise quantization only supports dynamic activation scheme for now, but got {activation_scheme} activation scheme."
                )
        if self.use_mxfp8:
            if weight_block_size is None:
                weight_block_size = [1, 32]
            elif weight_block_size != [1, 32]:
                raise ValueError("MXFP8 requires weight_block_size=[1, 32].")
        self.weight_block_size = weight_block_size

    def get_name(self) -> str:
        return "mxfp8" if self.use_mxfp8 else "fp8"

    @classmethod
    def get_supported_act_dtypes(cls) -> List[torch.dtype]:
        return [torch.bfloat16, torch.half]

    def get_min_capability(self) -> int:
        if is_npu():

View on GitHub (pinned to 0132848349)

Solutions

1. Set activation_scheme to "dynamic" in the quantization_config when weight_block_size is present
2. Use a checkpoint quantized with dynamic activation scheme (the standard for DeepSeek FP8 releases)
3. Remove weight_block_size if you must keep static activations (per-tensor FP8 path)

Example fix

// before
{"quant_method":"fp8","weight_block_size":[128,128],"activation_scheme":"static"}
// after
{"quant_method":"fp8","weight_block_size":[128,128],"activation_scheme":"dynamic"}

if qcfg.get("weight_block_size") and qcfg.get("activation_scheme") != "dynamic":
    qcfg["activation_scheme"] = "dynamic"

def block_quant_scheme_ok(qcfg: dict) -> bool:
    return qcfg.get("weight_block_size") is None or qcfg.get("activation_scheme") == "dynamic"

When it happens

Trigger: Fp8LinearConfig(weight_block_size=[128,128], activation_scheme="static") — e.g. a checkpoint quantized with a static per-tensor activation scale but block-wise weights, or a config.json that defaulted to static.

Common situations: Checkpoints quantized with tools producing mixed static/block-wise configs; users flipping activation_scheme while keeping block sizes; copy-pasted configs between models.

Related errors

• The block-wise quantization only supports fp8-serialized che
• Found static activation scheme for checkpoint that was not s
• QuantConfig has static quantization, but found activation sc
• The hpc_ops MoE runner backend requires static activation sc
• kv_scales supplied but unified_kv is {unified_kv.dtype}, exp

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0d6df10ba04f3adc. Report an issue: GitHub.