{"record":{"id":"fbbc7211145a659b","repo":"sgl-project/sglang","slug":"the-block-wise-quantization-only-supports-fp8-seri","errorCode":null,"errorMessage":"The block-wise quantization only supports fp8-serialized checkpoint for now.","messagePattern":"The block-wise quantization only supports fp8-serialized checkpoint for now\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/fp8.py","lineNumber":264,"sourceCode":"            log_info_on_rank0(logger, \"Detected fp8 checkpoint.\")\n        if activation_scheme not in ACTIVATION_SCHEMES:\n            raise ValueError(f\"Unsupported activation scheme {activation_scheme}\")\n        self.activation_scheme = activation_scheme\n        self.ignored_layers = ignored_layers or []\n        if ignored_layers_str := envs.SGLANG_FP8_IGNORED_LAYERS.get():\n            self.ignored_layers.extend(\n                [\n                    layer.strip()\n                    for layer in ignored_layers_str.split(\",\")\n                    if layer.strip()\n                ]\n            )\n        self.packed_modules_mapping = packed_modules_mapping or {}\n        self.use_mxfp8 = use_mxfp8\n        self.kv_cache_quant_algo = kv_cache_quant_algo\n        if weight_block_size is not None:\n            if not is_checkpoint_fp8_serialized:\n                raise ValueError(\n                    \"The block-wise quantization only supports fp8-serialized checkpoint for now.\"\n                )\n            if len(weight_block_size) != 2:\n                raise ValueError(\n                    f\"The quantization block size of weight must have 2 dimensions, but got {len(weight_block_size)} dimensions.\"\n                )\n            if activation_scheme != \"dynamic\":\n                raise ValueError(\n                    f\"The block-wise quantization only supports dynamic activation scheme for now, but got {activation_scheme} activation scheme.\"\n                )\n        if self.use_mxfp8:\n            if weight_block_size is None:\n                weight_block_size = [1, 32]\n            elif weight_block_size != [1, 32]:\n                raise ValueError(\"MXFP8 requires weight_block_size=[1, 32].\")\n        self.weight_block_size = weight_block_size\n\n    def get_name(self) -> str:","sourceCodeStart":246,"sourceCodeEnd":282,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/fp8.py#L246-L282","documentation":"Block-wise FP8 quantization (weight_block_size set, as in DeepSeek-V2/V3 style checkpoints) requires the checkpoint to actually be FP8-serialized, because block scales can only be consumed from an FP8 checkpoint layout. Fp8LinearConfig raises when weight_block_size is provided together with is_checkpoint_fp8_serialized=False.","triggerScenarios":"Fp8LinearConfig(weight_block_size=[128,128], is_checkpoint_fp8_serialized=False) — e.g. a config where quant_method is fp8 with block sizes but the weight files are BF16/FP16, or a quantization_config that omitted \"fmt\": \"e4m3\" so serialization detection failed.","commonSituations":"On-the-fly FP8 quantization of a BF16 DeepSeek checkpoint that carries leftover weight_block_size in config.json; manually merged configs where is_checkpoint_fp8_serialized was defaulted to False.","solutions":["Use a genuinely FP8-serialized checkpoint (e.g. deepseek-ai DeepSeek-V3 FP8 releases) that includes weight scales and is_checkpoint_fp8_serialized=True","If quantizing on the fly from BF16, remove weight_block_size (or the quantization_config block) so per-tensor/on-the-fly paths are used instead","Regenerate the quantization_config with quant_method=\"fp8\", activation_scheme=\"dynamic\", and FP8 weights"],"exampleFix":"// before\n{\"quant_method\": \"fp8\", \"weight_block_size\": [128,128]}  // BF16 weights\n// after\n{\"quant_method\": \"fp8\", \"activation_scheme\": \"dynamic\", \"weight_block_size\": [128,128], \"fmt\": \"e4m3\"}  // true FP8 checkpoint\n// or: remove quantization_config entirely for BF16","handlingStrategy":"validation","validationCode":"qcfg = model_config.quantization_config\nif qcfg.get(\"weight_block_size\") and not qcfg.get(\"is_checkpoint_fp8_serialized\", True):\n    raise SystemExit(\"block-wise FP8 requires an FP8-serialized checkpoint; use a real FP8 release\")","typeGuard":"def block_quant_config_is_consistent(qcfg: dict) -> bool:\n    if qcfg.get(\"weight_block_size\") is None:\n        return True\n    return bool(qcfg.get(\"is_checkpoint_fp8_serialized\")) or qcfg.get(\"fmt\") == \"e4m3\"","tryCatchPattern":null,"preventionTips":["Never mix block sizes into non-FP8 checkpoints","Prefer unmodified official FP8 checkpoint configs","Validate quant config pairs before launch"],"tags":["quantization","fp8","block-quant","checkpoint-config"],"backgroundTag":"checkpoint-quant-config-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}