{"record":{"id":"a9931dd97ff9cd5d","repo":"sgl-project/sglang","slug":"weight-output-size-per-partition-output-size-pe-a9931d","errorCode":null,"errorMessage":"Weight output_size_per_partition = {output_size_per_partition} is not divisible by pack_factor = {self.quant_config.pack_factor}.","messagePattern":"Weight output_size_per_partition = (.+?) is not divisible by pack_factor = (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/marlin_utils.py","lineNumber":768,"sourceCode":"    ):\n        del output_size  # Unused.\n        weight_loader = extra_weight_attrs[\"weight_loader\"]\n\n        if params_dtype != torch.float16:\n            raise ValueError(\n                f\"The params dtype must be float16, but got {params_dtype}\"\n            )\n\n        # Validate output_size_per_partition\n        output_size_per_partition = sum(output_partition_sizes)\n        if output_size_per_partition % self.quant_config.min_n_threads != 0:\n            raise ValueError(\n                f\"Weight output_size_per_partition = \"\n                f\"{output_size_per_partition} is not divisible by \"\n                f\"min_n_threads = {self.quant_config.min_n_threads}.\"\n            )\n        if output_size_per_partition % self.quant_config.pack_factor != 0:\n            raise ValueError(\n                f\"Weight output_size_per_partition = \"\n                f\"{output_size_per_partition} is not divisible by \"\n                f\"pack_factor = {self.quant_config.pack_factor}.\"\n            )\n\n        # Validate input_size_per_partition\n        if input_size_per_partition % self.quant_config.min_k_threads != 0:\n            raise ValueError(\n                f\"Weight input_size_per_partition = \"\n                f\"{input_size_per_partition} is not divisible by \"\n                f\"min_k_threads = {self.quant_config.min_k_threads}.\"\n            )\n        if (\n            self.quant_config.group_size != -1\n            and input_size_per_partition % self.quant_config.group_size != 0\n        ):\n            raise ValueError(\n                f\"Weight input_size_per_partition = \"","sourceCodeStart":750,"sourceCodeEnd":786,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/marlin_utils.py#L750-L786","documentation":"After the min_n_threads check, create_weights also requires output_size_per_partition divisible by the config's pack_factor (int32 words pack multiple sub-byte weights, so local N must align to the packing width). A TP shard that passed the thread check but breaks packing alignment raises this ValueError.","triggerScenarios":"TP sharding where local N is divisible by min_n_threads but not by pack_factor (e.g. pack_factor 8 with local N ≡ 4 mod 8); mis-computed output_partition_sizes for fused/moe layers.","commonSituations":"Odd TP degrees again; custom layers passing concatenated partition sizes that don't individually align.","solutions":["Use a power-of-two tensor_parallel_size dividing the output dimension cleanly","Check each output_partition_size individually for pack_factor alignment in custom layers","Fall back to --quantization gptq if the shard layout can't change"],"exampleFix":"# before\n--tensor-parallel-size 3  # local N 4096/... misaligned to pack_factor\n# after\n--tensor-parallel-size 4","handlingStrategy":"validation","validationCode":"assert output_size_per_partition % pack_factor == 0, \"local N breaks Marlin packing\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Ensure each output_partition_size is individually pack-aligned in fused layers","Prefer power-of-two TP"],"tags":["marlin","tensor-parallel","pack-factor","shape-validation"],"backgroundTag":"tensor-parallel-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}