{"record":{"id":"8795f89a4d802520","repo":"sgl-project/sglang","slug":"the-input-size-of-down-s-weight-intermediate-si-8795f8","errorCode":null,"errorMessage":"The input_size of down's weight = {intermediate_size_per_partition} is not divisible by weight quantization block_k = {block_k}.","messagePattern":"The input_size of down's weight = (.+?) is not divisible by weight quantization block_k = (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/fp8.py","lineNumber":1178,"sourceCode":"            block_n, block_k = (\n                quant_config.weight_block_size[0],\n                quant_config.weight_block_size[1],\n            )\n\n            padding_size = get_moe_padding_size(_use_aiter)\n            if not (_use_aiter and padding_size == block_n == block_k):\n                # NOTE(HandH1998): To ensure proper alignment of the block-wise quantization scales, the output_size of the weights for both the gate and up layers must be divisible by block_n.\n                # Required by column parallel or enabling merged weights\n                if intermediate_size_per_partition % block_n != 0:\n                    raise ValueError(\n                        f\"The output_size of gate's and up's weight = \"\n                        f\"{intermediate_size_per_partition} is not divisible by \"\n                        f\"weight quantization block_n = {block_n}.\"\n                    )\n                if tp_size > 1:\n                    # Required by row parallel\n                    if intermediate_size_per_partition % block_k != 0:\n                        raise ValueError(\n                            f\"The input_size of down's weight = \"\n                            f\"{intermediate_size_per_partition} is not divisible by \"\n                            f\"weight quantization block_k = {block_k}.\"\n                        )\n\n        # WEIGHTS\n        if is_fp4_expert:\n            w13_weight = torch.nn.Parameter(\n                torch.empty(\n                    num_experts,\n                    w13_num_shards * intermediate_size_per_partition,\n                    hidden_size // 2,\n                    dtype=torch.int8,\n                ),\n                requires_grad=False,\n            )\n            w2_weight = torch.nn.Parameter(\n                torch.empty(","sourceCodeStart":1160,"sourceCodeEnd":1196,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/fp8.py#L1160-L1196","documentation":"Companion check for MoE down-projection: with TP>1 the down layer's input size (= sharded intermediate size) must be divisible by block_k so FP8 block scales align on the K axis. Raised in create_fp8_moe_weight_ at model init.","triggerScenarios":"FP8 block-quant MoE with tp_size>1 and moe_intermediate_size/tp not a multiple of block_k (128); typically the same misconfigured TP size that trips the gate/up check.","commonSituations":"Odd TP sizes on FP8 MoE models; models whose moe_intermediate_size has few valid TP factorizations.","solutions":["Use a TP size making moe_intermediate_size/tp divisible by 128 (compute before launch)","Fall back to TP=1 (may need more memory) or a compatible TP","Choose a checkpoint without block quantization if flexible"],"exampleFix":"# before\n--tp 5  # down input partition misaligned\n# after\n--tp 4","handlingStrategy":"validation","validationCode":"inter = model_config.moe_intermediate_size\nassert args.tp == 1 or (inter // args.tp) % 128 == 0, \"down-proj K partition misaligned; adjust --tp-size\"","typeGuard":"def down_proj_tp_ok(inter_size: int, tp: int, block_k: int = 128) -> bool:\n    return tp == 1 or (inter_size // tp) % block_k == 0","tryCatchPattern":null,"preventionTips":["One divisibility check covers both gate/up and down constraints — script it","Document valid TP sizes per model"],"tags":["quantization","fp8","moe","tensor-parallel","shape-validation"],"backgroundTag":"tensor-parallel-shape-misaligned","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}