{"record":{"id":"bb686cfe19ce8af5","repo":"sgl-project/sglang","slug":"mxfp8-requires-weight-block-size-1-32","errorCode":null,"errorMessage":"MXFP8 requires weight_block_size=[1, 32].","messagePattern":"MXFP8 requires weight_block_size=\\[1, 32\\]\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/fp8.py","lineNumber":279,"sourceCode":"        self.kv_cache_quant_algo = kv_cache_quant_algo\n        if weight_block_size is not None:\n            if not is_checkpoint_fp8_serialized:\n                raise ValueError(\n                    \"The block-wise quantization only supports fp8-serialized checkpoint for now.\"\n                )\n            if len(weight_block_size) != 2:\n                raise ValueError(\n                    f\"The quantization block size of weight must have 2 dimensions, but got {len(weight_block_size)} dimensions.\"\n                )\n            if activation_scheme != \"dynamic\":\n                raise ValueError(\n                    f\"The block-wise quantization only supports dynamic activation scheme for now, but got {activation_scheme} activation scheme.\"\n                )\n        if self.use_mxfp8:\n            if weight_block_size is None:\n                weight_block_size = [1, 32]\n            elif weight_block_size != [1, 32]:\n                raise ValueError(\"MXFP8 requires weight_block_size=[1, 32].\")\n        self.weight_block_size = weight_block_size\n\n    def get_name(self) -> str:\n        return \"mxfp8\" if self.use_mxfp8 else \"fp8\"\n\n    @classmethod\n    def get_supported_act_dtypes(cls) -> List[torch.dtype]:\n        return [torch.bfloat16, torch.half]\n\n    def get_min_capability(self) -> int:\n        if is_npu():\n            return 0  # NPU bypasses CUDA capability checks\n        if _is_musa:\n            return 31\n        if self.use_mxfp8 and _is_hip and _is_gfx95_supported:\n            return 95\n        if self.use_mxfp8 and _mxfp8_to_block_fp8_required:\n            return 94","sourceCodeStart":261,"sourceCodeEnd":297,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/fp8.py#L261-L297","documentation":"MXFP8 (microscaling FP8, OCP MX format) uses a fixed 1x32 block (32 elements per scale), so any weight_block_size other than [1, 32] is rejected by Fp8LinearConfig when use_mxfp8 is true. [1, 32] is the only legal MX block.","triggerScenarios":"Fp8LinearConfig(use_mxfp8=True, weight_block_size=[128,128]) — a config that enables mxfp8 (often auto-detected from quant_method \"mxfp8\" or compressed-tensors MX schemes) but also carries a legacy FP8 block size.","commonSituations":"Editing a DeepSeek FP8 [128,128] config to switch to MXFP8 without removing the block size; checkpoints with both keys present; mixing config fragments from different quant formats.","solutions":["Remove weight_block_size from the quantization_config (it defaults to [1,32] for MXFP8) or set it exactly to [1, 32]","Use a genuine MXFP8 checkpoint whose config already omits or sets [1,32]"],"exampleFix":"// before\n{\"quant_method\":\"mxfp8\",\"weight_block_size\":[128,128]}\n// after\n{\"quant_method\":\"mxfp8\"}  // weight_block_size defaults to [1,32]","handlingStrategy":"validation","validationCode":"if qcfg.get(\"use_mxfp8\") or qcfg.get(\"quant_method\") == \"mxfp8\":\n    wbs = qcfg.get(\"weight_block_size\")\n    if wbs is not None and list(wbs) != [1, 32]:\n        qcfg.pop(\"weight_block_size\")  # default [1,32]","typeGuard":"def is_legal_mxfp8_block(wbs) -> bool:\n    return wbs is None or list(wbs) == [1, 32]","tryCatchPattern":null,"preventionTips":["MX means 1x32 blocks — don't reuse FP8 [128,128] configs","Delete stale keys when switching quant formats"],"tags":["quantization","mxfp8","config-validation"],"backgroundTag":"checkpoint-quant-config-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}