{"record":{"id":"616156ed99789503","repo":"sgl-project/sglang","slug":"weight-input-size-per-partition-input-size-per-616156","errorCode":null,"errorMessage":"Weight input_size_per_partition = {input_size_per_partition} is not divisible by weight quantization block_k = {block_k}.","messagePattern":"Weight input_size_per_partition = (.+?) is not divisible by weight quantization block_k = (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/fp8.py","lineNumber":508,"sourceCode":"        output_size_per_partition: int,\n        output_partition_sizes: List[int],\n        skip_block_quant_check: bool = False,\n    ):\n        block_n, block_k = (\n            quant_config.weight_block_size[0],\n            quant_config.weight_block_size[1],\n        )\n\n        if skip_block_quant_check:\n            print_warning_once(\n                \"Skipping block quantization checks for weight partition.\"\n            )\n        else:\n            tp_size = get_parallel().tp_size\n            # Required by row parallel\n            if tp_size > 1 and input_size // input_size_per_partition == tp_size:\n                if input_size_per_partition % block_k != 0:\n                    raise ValueError(\n                        f\"Weight input_size_per_partition = \"\n                        f\"{input_size_per_partition} is not divisible by \"\n                        f\"weight quantization block_k = {block_k}.\"\n                    )\n            # Required by column parallel or enabling merged weights\n            if (\n                tp_size > 1 and output_size // output_size_per_partition == tp_size\n            ) or len(output_partition_sizes) > 1:\n                for output_partition_size in output_partition_sizes:\n                    if output_partition_size % block_n != 0:\n                        raise ValueError(\n                            f\"Weight output_partition_size = \"\n                            f\"{output_partition_size} is not divisible by \"\n                            f\"weight quantization block_n = {block_n}.\"\n                        )\n\n    @staticmethod\n    def create_fp8_weight_(","sourceCodeStart":490,"sourceCodeEnd":526,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/fp8.py#L490-L526","documentation":"validate_block_quant_shapes enforces that for row-parallel (input-sharded) linear layers under tensor parallelism, the per-partition input dimension must be divisible by the weight quantization block_k (e.g. 128). Otherwise FP8 block scale tiles would straddle partition boundaries and produce wrong results.","triggerScenarios":"Creating an FP8 block-quant linear layer with tp_size>1 where the K dimension divided by TP is not a multiple of block_k — e.g. hidden_size 4096 with TP=3 giving input_size_per_partition 1366 with block_k=128, or small models whose hidden size isn't TP-divisible into 128-multiples.","commonSituations":"Odd tensor-parallel sizes (3, 5, 6) on block-quant FP8 models; small custom models with hidden_size not divisible by TP*128; DeepSeek FP8 with unusual --tp-size.","solutions":["Use a tensor-parallel size that divides the hidden/input size into a multiple of block_k (128), e.g. TP in powers of two for standard models","Reduce --tp-size (e.g. to 2 or 1) so input_size/tp is a multiple of 128","If the model architecture allows, pad or choose a checkpoint variant whose K dim is TP-friendly"],"exampleFix":"# before\npython -m sglang.launch_server --model deepseek-ai/DeepSeek-V3 --tp 6  # K partition % 128 != 0\n# after\npython -m sglang.launch_server --model deepseek-ai/DeepSeek-V3 --tp 8","handlingStrategy":"validation","validationCode":"tp = 4; block_k = 128; hidden = model_config.hidden_size\nassert tp == 1 or (hidden // tp) % block_k == 0, f\"hidden/tp={hidden//tp} not divisible by {block_k}; pick another --tp-size\"","typeGuard":"def tp_ok_for_block_k(input_size: int, tp: int, block_k: int = 128) -> bool:\n    return tp == 1 or (input_size // tp) % block_k == 0","tryCatchPattern":null,"preventionTips":["Compute divisibility of hidden_size/tp by 128 before launching FP8 block-quant models","Prefer power-of-two TP sizes","Keep a per-model table of validated TP sizes"],"tags":["quantization","fp8","tensor-parallel","shape-validation"],"backgroundTag":"tensor-parallel-shape-misaligned","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}