{"record":{"id":"173dddc198fa5cc9","repo":"sgl-project/sglang","slug":"weight-output-size-per-partition-output-size-pe-173ddd","errorCode":null,"errorMessage":"Weight output_size_per_partition = {output_size_per_partition} is not divisible by  min_thread_n = {GPTQ_MARLIN_MIN_THREAD_N}. Consider reducing tensor_parallel_size or running with --quantization gptq.","messagePattern":"Weight output_size_per_partition = (.+?) is not divisible by  min_thread_n = (.+?)\\. Consider reducing tensor_parallel_size or running with --quantization gptq\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/marlin_utils.py","lineNumber":187,"sourceCode":"def verify_marlin_supported(\n    quant_type: ScalarType, group_size: int, has_zp: bool = False\n) -> None:\n    cond, err_msg = _check_marlin_supported(quant_type, group_size, has_zp)\n    if not cond:\n        assert err_msg is not None\n        raise ValueError(err_msg)\n\n\ndef verify_marlin_supports_shape(\n    output_size_per_partition: int,\n    input_size_per_partition: int,\n    input_size: int,\n    group_size: int,\n) -> None:\n\n    # Validate output_size_per_partition\n    if output_size_per_partition % GPTQ_MARLIN_MIN_THREAD_N != 0:\n        raise ValueError(\n            f\"Weight output_size_per_partition = \"\n            f\"{output_size_per_partition} is not divisible by \"\n            f\" min_thread_n = {GPTQ_MARLIN_MIN_THREAD_N}. \"\n            \"Consider reducing tensor_parallel_size or running \"\n            \"with --quantization gptq.\"\n        )\n\n    # Validate input_size_per_partition\n    if input_size_per_partition % GPTQ_MARLIN_MIN_THREAD_K != 0:\n        raise ValueError(\n            f\"Weight input_size_per_partition = \"\n            f\"{input_size_per_partition} is not divisible \"\n            f\"by min_thread_k = {GPTQ_MARLIN_MIN_THREAD_K}. \"\n            \"Consider reducing tensor_parallel_size or running \"\n            \"with --quantization gptq.\"\n        )\n\n    if group_size < input_size and input_size_per_partition % group_size != 0:","sourceCodeStart":169,"sourceCodeEnd":205,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/marlin_utils.py#L169-L205","documentation":"verify_marlin_supports_shape checks that the TP-local output dimension is divisible by GPTQ_MARLIN_MIN_THREAD_N (the kernel's minimum N tiling, commonly 64). The error message itself suggests the two remedies: reduce tensor_parallel_size, or run with --quantization gptq to use the non-Marlin GEMM path. Called from create_weights and the pre-flight check_marlin_supports_shape.","triggerScenarios":"GPTQ Marlin serving with a TP degree making output_size_per_partition < or not divisible by GPTQ_MARLIN_MIN_THREAD_N; small models or heavily sharded MoE experts.","commonSituations":"Testing quantized small models on many GPUs; TP=6/7 style layouts; CI matrix runs sweeping TP sizes.","solutions":["Reduce tensor_parallel_size (e.g. to 1, 2, 4) so local N is divisible by GPTQ_MARLIN_MIN_THREAD_N","Run with --quantization gptq to bypass Marlin","Use check_marlin_supports_shape in a pre-launch probe to select TP automatically"],"exampleFix":"# before\n--tensor-parallel-size 8   # local N 512 not divisible by 64*... per kernel constraint\n# after\n--tensor-parallel-size 4 --quantization marlin   # or: --quantization gptq","handlingStrategy":"fallback","validationCode":"from sglang.srt.layers.quantization.marlin_utils import check_marlin_supports_shape\ncheck_marlin_supports_shape(output_size // tp, input_size // tp, group_size)  # raises early with guidance","typeGuard":null,"tryCatchPattern":"try:\n    check_marlin_supports_shape(out_pp, in_pp, gs)\nexcept ValueError:\n    server_args.quantization = \"gptq\"  # fall back to non-Marlin path","preventionTips":["Run check_marlin_supports_shape before launching Marlin models","Auto-select the largest power-of-two TP that satisfies thread-N/K divisibility"],"tags":["marlin","gptq","tensor-parallel","shape-validation"],"backgroundTag":"tensor-parallel-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}