{"record":{"id":"5561dcfa42e5eedd","repo":"sgl-project/sglang","slug":"weight-input-size-per-partition-input-size-per-5561dc","errorCode":null,"errorMessage":"Weight input_size_per_partition = {input_size_per_partition} is not divisible by min_k_threads = {self.quant_config.min_k_threads}.","messagePattern":"Weight input_size_per_partition = (.+?) is not divisible by min_k_threads = (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/marlin_utils.py","lineNumber":776,"sourceCode":"\n        # Validate output_size_per_partition\n        output_size_per_partition = sum(output_partition_sizes)\n        if output_size_per_partition % self.quant_config.min_n_threads != 0:\n            raise ValueError(\n                f\"Weight output_size_per_partition = \"\n                f\"{output_size_per_partition} is not divisible by \"\n                f\"min_n_threads = {self.quant_config.min_n_threads}.\"\n            )\n        if output_size_per_partition % self.quant_config.pack_factor != 0:\n            raise ValueError(\n                f\"Weight output_size_per_partition = \"\n                f\"{output_size_per_partition} is not divisible by \"\n                f\"pack_factor = {self.quant_config.pack_factor}.\"\n            )\n\n        # Validate input_size_per_partition\n        if input_size_per_partition % self.quant_config.min_k_threads != 0:\n            raise ValueError(\n                f\"Weight input_size_per_partition = \"\n                f\"{input_size_per_partition} is not divisible by \"\n                f\"min_k_threads = {self.quant_config.min_k_threads}.\"\n            )\n        if (\n            self.quant_config.group_size != -1\n            and input_size_per_partition % self.quant_config.group_size != 0\n        ):\n            raise ValueError(\n                f\"Weight input_size_per_partition = \"\n                f\"{input_size_per_partition} is not divisible by \"\n                f\"group_size = {self.quant_config.group_size}.\"\n            )\n\n        # Check that we have at least 4 tiles horizontally in the shard\n        num_tiles_per_perm = self.quant_config.perm_len // (\n            self.quant_config.tile_size**2\n        )","sourceCodeStart":758,"sourceCodeEnd":794,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/marlin_utils.py#L758-L794","documentation":"The input (K) dimension per partition must be divisible by the Marlin config's min_k_threads because the kernel processes K in fixed thread-group tiles. create_weights raises this when the TP-sharded local K is misaligned.","triggerScenarios":"hidden_size or local K after TP/MoE sharding not divisible by min_k_threads (commonly 256 for Marlin); embedding/lm-head layers with unusual dims.","commonSituations":"Small models (tiny hidden sizes) with Marlin checkpoints; TP degrees like 3 or 6 breaking power-of-two alignment.","solutions":["Reduce tensor_parallel_size to a power-of-two divisor of the input dimension","Use --quantization gptq fallback for exotic dims","Confirm hidden_size / TP >= and divisible by min_k_threads before launch"],"exampleFix":"# before\n--tensor-parallel-size 6  # 4096/6 not divisible by min_k_threads\n# after\n--tensor-parallel-size 2","handlingStrategy":"validation","validationCode":"min_k = 256  # GPTQ Marlin min_k_threads\nassert (hidden_size // tp) % min_k == 0, \"local K breaks Marlin K tiling\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Avoid very high TP degrees on small models","Pre-check hidden_size // TP against min_k_threads before launch"],"tags":["marlin","tensor-parallel","shape-validation","gptq","awq"],"backgroundTag":"tensor-parallel-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}