{"record":{"id":"20a66a422e31e9bd","repo":"sgl-project/sglang","slug":"weight-output-partition-size-output-partition-s-20a66a","errorCode":null,"errorMessage":"Weight output_partition_size = {output_partition_size} is not divisible by weight quantization block_n = {block_n}.","messagePattern":"Weight output_partition_size = (.+?) is not divisible by weight quantization block_n = (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/fp8.py","lineNumber":519,"sourceCode":"                \"Skipping block quantization checks for weight partition.\"\n            )\n        else:\n            tp_size = get_parallel().tp_size\n            # Required by row parallel\n            if tp_size > 1 and input_size // input_size_per_partition == tp_size:\n                if input_size_per_partition % block_k != 0:\n                    raise ValueError(\n                        f\"Weight input_size_per_partition = \"\n                        f\"{input_size_per_partition} is not divisible by \"\n                        f\"weight quantization block_k = {block_k}.\"\n                    )\n            # Required by column parallel or enabling merged weights\n            if (\n                tp_size > 1 and output_size // output_size_per_partition == tp_size\n            ) or len(output_partition_sizes) > 1:\n                for output_partition_size in output_partition_sizes:\n                    if output_partition_size % block_n != 0:\n                        raise ValueError(\n                            f\"Weight output_partition_size = \"\n                            f\"{output_partition_size} is not divisible by \"\n                            f\"weight quantization block_n = {block_n}.\"\n                        )\n\n    @staticmethod\n    def create_fp8_weight_(\n        layer: torch.nn.Module,\n        block_quant: bool,\n        quant_config,\n        use_mxfp8: bool,\n        output_size_per_partition: int,\n        input_size_per_partition: int,\n        output_partition_sizes: List[int],\n        input_size: int,\n        output_size: int,\n        params_dtype: torch.dtype,\n        weight_loader,","sourceCodeStart":501,"sourceCodeEnd":537,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/fp8.py#L501-L537","documentation":"For column-parallel or merged-weight FP8 block-quant layers, each output partition size must be divisible by block_n so per-block weight scales align with partition boundaries. validate_block_quant_shapes raises when any output_partition_size % block_n != 0.","triggerScenarios":"Creating merged column-parallel weights (e.g. gate+up QKV fused layers) where an individual partition size (intermediate_size/tp, num_heads*head_dim/tp) is not a multiple of block_n=128 under tp_size>1 or when len(output_partition_sizes)>1.","commonSituations":"MoE intermediate sizes or attention head configurations not divisible into 128-multiples after TP sharding; unusual TP sizes; models with head_dim like 80 or intermediate sizes like 1536 with TP>1.","solutions":["Choose a TP size such that every output partition is a multiple of 128 (check head counts and intermediate sizes)","Use TP=1 or a power-of-two TP that evenly divides the model's dimensions","Verify the checkpoint's weight_block_size; some models use [128,128] and need dimensions divisible accordingly"],"exampleFix":"# before\n--tp 3  # head partition = 4096/3 not multiple of 128\n# after\n--tp 2  # each partition divisible by block_n","handlingStrategy":"validation","validationCode":"block_n = 128\nfor part in output_partition_sizes:  # e.g. [num_heads/tp*head_dim, ...]\n    assert part % block_n == 0, f\"partition {part} not divisible by {block_n}; change --tp-size\"","typeGuard":"def partitions_ok(sizes: list[int], block_n: int = 128) -> bool:\n    return all(s % block_n == 0 for s in sizes)","tryCatchPattern":null,"preventionTips":["Check head-count and intermediate-size sharding divisibility by 128","Automate TP validation in launch scripts"],"tags":["quantization","fp8","tensor-parallel","shape-validation"],"backgroundTag":"tensor-parallel-shape-misaligned","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}