{"record":{"id":"0d6df10ba04f3adc","repo":"sgl-project/sglang","slug":"the-block-wise-quantization-only-supports-dynamic-0d6df1","errorCode":null,"errorMessage":"The block-wise quantization only supports dynamic activation scheme for now, but got {activation_scheme} activation scheme.","messagePattern":"The block-wise quantization only supports dynamic activation scheme for now, but got (.+?) activation scheme\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/fp8.py","lineNumber":272,"sourceCode":"                    layer.strip()\n                    for layer in ignored_layers_str.split(\",\")\n                    if layer.strip()\n                ]\n            )\n        self.packed_modules_mapping = packed_modules_mapping or {}\n        self.use_mxfp8 = use_mxfp8\n        self.kv_cache_quant_algo = kv_cache_quant_algo\n        if weight_block_size is not None:\n            if not is_checkpoint_fp8_serialized:\n                raise ValueError(\n                    \"The block-wise quantization only supports fp8-serialized checkpoint for now.\"\n                )\n            if len(weight_block_size) != 2:\n                raise ValueError(\n                    f\"The quantization block size of weight must have 2 dimensions, but got {len(weight_block_size)} dimensions.\"\n                )\n            if activation_scheme != \"dynamic\":\n                raise ValueError(\n                    f\"The block-wise quantization only supports dynamic activation scheme for now, but got {activation_scheme} activation scheme.\"\n                )\n        if self.use_mxfp8:\n            if weight_block_size is None:\n                weight_block_size = [1, 32]\n            elif weight_block_size != [1, 32]:\n                raise ValueError(\"MXFP8 requires weight_block_size=[1, 32].\")\n        self.weight_block_size = weight_block_size\n\n    def get_name(self) -> str:\n        return \"mxfp8\" if self.use_mxfp8 else \"fp8\"\n\n    @classmethod\n    def get_supported_act_dtypes(cls) -> List[torch.dtype]:\n        return [torch.bfloat16, torch.half]\n\n    def get_min_capability(self) -> int:\n        if is_npu():","sourceCodeStart":254,"sourceCodeEnd":290,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/fp8.py#L254-L290","documentation":"Block-wise FP8 quantization in SGLang only implements dynamic activation scaling (activations are quantized per-token at runtime); static per-tensor activation scales cannot be combined with weight block scales. Fp8LinearConfig raises when weight_block_size is set but activation_scheme is \"static\" (or anything not \"dynamic\").","triggerScenarios":"Fp8LinearConfig(weight_block_size=[128,128], activation_scheme=\"static\") — e.g. a checkpoint quantized with a static per-tensor activation scale but block-wise weights, or a config.json that defaulted to static.","commonSituations":"Checkpoints quantized with tools producing mixed static/block-wise configs; users flipping activation_scheme while keeping block sizes; copy-pasted configs between models.","solutions":["Set activation_scheme to \"dynamic\" in the quantization_config when weight_block_size is present","Use a checkpoint quantized with dynamic activation scheme (the standard for DeepSeek FP8 releases)","Remove weight_block_size if you must keep static activations (per-tensor FP8 path)"],"exampleFix":"// before\n{\"quant_method\":\"fp8\",\"weight_block_size\":[128,128],\"activation_scheme\":\"static\"}\n// after\n{\"quant_method\":\"fp8\",\"weight_block_size\":[128,128],\"activation_scheme\":\"dynamic\"}","handlingStrategy":"validation","validationCode":"if qcfg.get(\"weight_block_size\") and qcfg.get(\"activation_scheme\") != \"dynamic\":\n    qcfg[\"activation_scheme\"] = \"dynamic\"","typeGuard":"def block_quant_scheme_ok(qcfg: dict) -> bool:\n    return qcfg.get(\"weight_block_size\") is None or qcfg.get(\"activation_scheme\") == \"dynamic\"","tryCatchPattern":null,"preventionTips":["Remember block-wise FP8 implies dynamic activations in SGLang","Sanitize configs converted from other runtimes"],"tags":["quantization","fp8","block-quant","activation-scheme"],"backgroundTag":"checkpoint-quant-config-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}