{"record":{"id":"4bb9be2ceb5ed710","repo":"sgl-project/sglang","slug":"weight-output-partition-size-output-partition-s","errorCode":null,"errorMessage":"Weight output_partition_size = {output_partition_size} is not divisible by weight quantization block_n = {block_n}.","messagePattern":"Weight output_partition_size = (.+?) is not divisible by weight quantization block_n = (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/fp8.py","lineNumber":169,"sourceCode":"            block_n, block_k = (\n                self.quant_config.weight_block_size[0],\n                self.quant_config.weight_block_size[1],\n            )\n            # Required by row parallel\n            if tp_size > 1 and input_size // input_size_per_partition == tp_size:\n                if input_size_per_partition % block_k != 0:\n                    raise ValueError(\n                        f\"Weight input_size_per_partition = \"\n                        f\"{input_size_per_partition} is not divisible by \"\n                        f\"weight quantization block_k = {block_k}.\"\n                    )\n            # Required by column parallel or enabling merged weights\n            if (\n                tp_size > 1 and output_size // output_size_per_partition == tp_size\n            ) or len(output_partition_sizes) > 1:\n                for output_partition_size in output_partition_sizes:\n                    if output_partition_size % block_n != 0:\n                        raise ValueError(\n                            f\"Weight output_partition_size = \"\n                            f\"{output_partition_size} is not divisible by \"\n                            f\"weight quantization block_n = {block_n}.\"\n                        )\n\n        layer.logical_widths = output_partition_sizes\n        layer.input_size_per_partition = input_size_per_partition\n        layer.output_size_per_partition = output_size_per_partition\n        layer.orig_dtype = params_dtype\n\n        # WEIGHT\n        weight_dtype = (\n            torch.float8_e4m3fn\n            if self.quant_config.is_checkpoint_fp8_serialized\n            else params_dtype\n        )\n\n        weight = ModelWeightParameter(","sourceCodeStart":151,"sourceCodeEnd":187,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/fp8.py#L151-L187","documentation":"Thrown when building FP8 block-quantized weights for a column-parallel or merged linear layer whose per-partition output size is not divisible by the quantization block_n size. The weight layout for block FP8 requires each shard's output dimension to be a multiple of the block size. It fires in create_weights before any tensor is allocated.","triggerScenarios":"Using FP8 block quantization (block_n, e.g. 128) with tensor parallelism > 1 on a column-parallel layer, or a merged layer (e.g. QKV projection) where any individual output_partition_size % block_n != 0.","commonSituations":"Model hidden sizes or intermediate dims not multiples of 128/16 (e.g. odd fused QKV splits), running TP>1 on small models, or overriding quantization config block sizes that mismatch the model architecture.","solutions":["Use a tensor-parallel size that divides the output dimensions so each shard is a multiple of block_n","Check the model's per-head/fused output sizes (e.g. QKV partition sizes) and pick block_n from the checkpoint quant config that divides them","Avoid merging or use a config where each partition size is divisible by block_n","Requantize the checkpoint with block sizes compatible with the model dims"],"exampleFix":"# before\npython -m sglang.launch_server --model m --tp 3 --quantization fp8  # shard 3381 % 128 != 0\n# after\npython -m sglang.launch_server --model m --tp 2 --quantization fp8  # shards divisible by 128","handlingStrategy":"validation","validationCode":"block_n = quant_cfg.weight_block_size[1]\nassert all(ps % block_n == 0 for ps in output_partition_sizes), f'partitions {output_partition_sizes} not aligned to block_n={block_n}'","typeGuard":null,"tryCatchPattern":"try:\n    layer.create_weights(...)\nexcept ValueError as e:\n    if 'not divisible by' in str(e):\n        raise SystemExit(f'TP/layout incompatible with FP8 block_n: {e}') from e\n    raise","preventionTips":["Assert output dims % block_n == 0 per shard before enabling TP","Prefer power-of-two TP degrees","Validate quant block size against model fused-projection sizes at startup"],"tags":["fp8","quantization","tensor-parallel","shape-mismatch"],"backgroundTag":"quantization-block-alignment","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}