{"record":{"id":"c1bd2bb772f9e0e4","repo":"sgl-project/sglang","slug":"the-quantization-block-size-of-weight-must-have-2-c1bd2b","errorCode":null,"errorMessage":"The quantization block size of weight must have 2 dimensions, but got {len(weight_block_size)} dimensions.","messagePattern":"The quantization block size of weight must have 2 dimensions, but got (.+?) dimensions\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/fp8.py","lineNumber":268,"sourceCode":"        self.ignored_layers = ignored_layers or []\n        if ignored_layers_str := envs.SGLANG_FP8_IGNORED_LAYERS.get():\n            self.ignored_layers.extend(\n                [\n                    layer.strip()\n                    for layer in ignored_layers_str.split(\",\")\n                    if layer.strip()\n                ]\n            )\n        self.packed_modules_mapping = packed_modules_mapping or {}\n        self.use_mxfp8 = use_mxfp8\n        self.kv_cache_quant_algo = kv_cache_quant_algo\n        if weight_block_size is not None:\n            if not is_checkpoint_fp8_serialized:\n                raise ValueError(\n                    \"The block-wise quantization only supports fp8-serialized checkpoint for now.\"\n                )\n            if len(weight_block_size) != 2:\n                raise ValueError(\n                    f\"The quantization block size of weight must have 2 dimensions, but got {len(weight_block_size)} dimensions.\"\n                )\n            if activation_scheme != \"dynamic\":\n                raise ValueError(\n                    f\"The block-wise quantization only supports dynamic activation scheme for now, but got {activation_scheme} activation scheme.\"\n                )\n        if self.use_mxfp8:\n            if weight_block_size is None:\n                weight_block_size = [1, 32]\n            elif weight_block_size != [1, 32]:\n                raise ValueError(\"MXFP8 requires weight_block_size=[1, 32].\")\n        self.weight_block_size = weight_block_size\n\n    def get_name(self) -> str:\n        return \"mxfp8\" if self.use_mxfp8 else \"fp8\"\n\n    @classmethod\n    def get_supported_act_dtypes(cls) -> List[torch.dtype]:","sourceCodeStart":250,"sourceCodeEnd":286,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/fp8.py#L250-L286","documentation":"Fp8LinearConfig requires weight_block_size to be a 2-element iterable [block_n, block_k] because FP8 block quantization scales form a 2D grid over the weight matrix. Passing a scalar, a 1-element or 3+-element list fails this validation.","triggerScenarios":"Fp8LinearConfig(weight_block_size=128) or weight_block_size=[128] / [128,128,128]; typically from a config.json that stored a scalar block size or an int-parsed value.","commonSituations":"Configs written by external quantization tools that use a single block size parameter; hand-edited quantization_config; schema drift between checkpoint producers.","solutions":["Set weight_block_size to the checkpoint's actual 2D block, e.g. [128, 128]","Verify in the model's config.json and fix the quantization_config entry","If loading a real DeepSeek FP8 checkpoint, its shipped config already has the correct [128,128] — prefer it over modified copies"],"exampleFix":"// before\n\"weight_block_size\": 128\n// after\n\"weight_block_size\": [128, 128]","handlingStrategy":"validation","validationCode":"wbs = qcfg.get(\"weight_block_size\")\nif wbs is not None and (not isinstance(wbs, (list, tuple)) or len(wbs) != 2):\n    raise SystemExit(f\"weight_block_size must be [block_n, block_k], got {wbs!r}\")","typeGuard":"def is_two_element_block_size(w) -> bool:\n    return isinstance(w, (list, tuple)) and len(w) == 2 and all(isinstance(x, int) and x > 0 for x in w)","tryCatchPattern":null,"preventionTips":["Schema-check quantization_config types, not just presence","Use official checkpoint configs as templates"],"tags":["quantization","fp8","config-validation","shape-validation"],"backgroundTag":"invalid-config-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}