{"record":{"id":"798ace11d70a48d7","repo":"sgl-project/sglang","slug":"weight-output-size-per-partition-output-size-pe","errorCode":null,"errorMessage":"Weight output_size_per_partition = {output_size_per_partition} is not divisible by min_n_threads = {self.quant_config.min_n_threads}.","messagePattern":"Weight output_size_per_partition = (.+?) is not divisible by min_n_threads = (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/marlin_utils.py","lineNumber":762,"sourceCode":"        input_size_per_partition: int,\n        output_partition_sizes: list[int],\n        input_size: int,\n        output_size: int,\n        params_dtype: torch.dtype,\n        **extra_weight_attrs,\n    ):\n        del output_size  # Unused.\n        weight_loader = extra_weight_attrs[\"weight_loader\"]\n\n        if params_dtype != torch.float16:\n            raise ValueError(\n                f\"The params dtype must be float16, but got {params_dtype}\"\n            )\n\n        # Validate output_size_per_partition\n        output_size_per_partition = sum(output_partition_sizes)\n        if output_size_per_partition % self.quant_config.min_n_threads != 0:\n            raise ValueError(\n                f\"Weight output_size_per_partition = \"\n                f\"{output_size_per_partition} is not divisible by \"\n                f\"min_n_threads = {self.quant_config.min_n_threads}.\"\n            )\n        if output_size_per_partition % self.quant_config.pack_factor != 0:\n            raise ValueError(\n                f\"Weight output_size_per_partition = \"\n                f\"{output_size_per_partition} is not divisible by \"\n                f\"pack_factor = {self.quant_config.pack_factor}.\"\n            )\n\n        # Validate input_size_per_partition\n        if input_size_per_partition % self.quant_config.min_k_threads != 0:\n            raise ValueError(\n                f\"Weight input_size_per_partition = \"\n                f\"{input_size_per_partition} is not divisible by \"\n                f\"min_k_threads = {self.quant_config.min_k_threads}.\"\n            )","sourceCodeStart":744,"sourceCodeEnd":780,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/marlin_utils.py#L744-L780","documentation":"Marlin tiles the output (N) dimension across thread groups; the per-partition output size (sum of output_partition_sizes, i.e. local N after TP sharding) must be divisible by the config's min_n_threads. If not, this ValueError fires at create_weights before tensors are allocated.","triggerScenarios":"A tensor_parallel_size that leaves output_size_per_partition not divisible by min_n_threads (commonly 64 for Marlin); MoE experts sharded so local expert N breaks the divisibility.","commonSituations":"Non-power-of-two TP degrees (3, 6, 7); small models whose intermediate size barely exceeds min_n_threads.","solutions":["Reduce tensor_parallel_size to the largest power-of-two divisor of the output dim (1, 2, 4, 8)","Fall back to --quantization gptq (non-Marlin) which has weaker shape constraints","Verify intermediate_size / TP is divisible by min_n_threads (typically 64)"],"exampleFix":"# before\n--tensor-parallel-size 6   # 11008/6 not divisible by min_n_threads\n# after\n--tensor-parallel-size 2","handlingStrategy":"validation","validationCode":"min_n = 64  # GPTQ Marlin min_n_threads\nassert (intermediate_size // tp) % min_n == 0, \"local N breaks Marlin thread tiling\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Stick to power-of-two TP degrees with Marlin checkpoints","Probe shapes with check_marlin_supports_shape-style helpers before launch"],"tags":["marlin","tensor-parallel","shape-validation","gptq","awq"],"backgroundTag":"tensor-parallel-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}