{"record":{"id":"707b590c9a3a6f3d","repo":"sgl-project/sglang","slug":"the-input-size-of-down-s-weight-intermediate-si-707b59","errorCode":null,"errorMessage":"The input_size of down's weight = {intermediate_size_per_partition} is not divisible by weight quantization block_k = {block_k}.","messagePattern":"The input_size of down's weight = (.+?) is not divisible by weight quantization block_k = (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_fp8_moe.py","lineNumber":119,"sourceCode":"            layer.weight_block_size = self.weight_block_size\n            tp_size = get_parallel().tp_size\n            block_n, block_k = (\n                self.weight_block_size[0],\n                self.weight_block_size[1],\n            )\n            # NOTE: To ensure proper alignment of the block-wise quantization\n            # scales, the output_size of the weights for both the gate and up\n            # layers must be divisible by block_n.\n            # Required by column parallel or enabling merged weights\n            if intermediate_size_per_partition % block_n != 0:\n                raise ValueError(\n                    f\"The output_size of gate's and up's weight = \"\n                    f\"{intermediate_size_per_partition} is not divisible by \"\n                    f\"weight quantization block_n = {block_n}.\"\n                )\n            if tp_size > 1 and intermediate_size_per_partition % block_k != 0:\n                # Required by row parallel\n                raise ValueError(\n                    f\"The input_size of down's weight = \"\n                    f\"{intermediate_size_per_partition} is not divisible by \"\n                    f\"weight quantization block_k = {block_k}.\"\n                )\n\n        w13_up_dim, w2_down_dim, weight_padded = get_moe_weight_sizes(\n            intermediate_size_per_partition,\n            is_aiter_moe=_use_aiter,\n            is_concat=True,\n            is_packed=False,\n        )\n\n        extra_weight_attrs.update(\n            {\"weight_padded\": weight_padded},\n        )\n\n        # WEIGHTS\n        w13_weight = torch.nn.Parameter(","sourceCodeStart":101,"sourceCodeEnd":137,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_fp8_moe.py#L101-L137","documentation":"Companion check to the block_n error: with TP > 1, the down projection's input size per partition (intermediate_size_per_partition) must be divisible by block_k for row-parallel FP8 block-quantized MoE weights to keep scale alignment.","triggerScenarios":"Block-quantized FP8 MoE with tp_size > 1 and intermediate_size_per_partition % block_k != 0 in create_weights.","commonSituations":"Same as 4185: unusual TP degrees or intermediate sizes vs the checkpoint's weight_block_size.","solutions":["Adjust --tp so intermediate_size / tp is divisible by block_k","Re-quantize with a block size compatible with the sharding plan","Run with TP=1 if feasible"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"block_k = cfg[\"quantization_config\"][\"weights\"][\"block_size\"][0]\nassert tp_size == 1 or (2*inter//tp) % block_k == 0","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check both block_n and block_k divisibility when planning TP for block-quant MoE"],"tags":["quantization","fp8","moe","block-quantization","tensor-parallel","shape-mismatch"],"backgroundTag":"tensor-parallel-shape-misaligned","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}