{"record":{"id":"9199a8b4deddf545","repo":"sgl-project/sglang","slug":"comfy-w4a4-layer-prefix-r-has-input-size-logica","errorCode":null,"errorMessage":"Comfy W4A4 layer {prefix!r} has input size {logical_input_size}, incompatible with quant_group_size=64 and convrot_groupsize={convrot_group_size}","messagePattern":"Comfy W4A4 layer (.+?) has input size (.+?), incompatible with quant_group_size=64 and convrot_groupsize=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/quantization_utils.py","lineNumber":244,"sourceCode":"            if weight_dtype != \"I8\" or scale_dtype != \"F32\":\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} needs I8 packed weights and \"\n                    f\"F32 scales, got {weight_dtype} and {scale_dtype}\"\n                )\n            if len(weight_shape) != 2 or scale_shape != (weight_shape[0],):\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} has incompatible weight/scale \"\n                    f\"shapes: {weight_shape} and {scale_shape}\"\n                )\n            logical_input_size = weight_shape[1] * 2\n            convrot_group_size = int(marker.get(\"convrot_groupsize\", 256))\n            if convrot_group_size not in (16, 64, 256):\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} has unsupported \"\n                    f\"convrot_groupsize={convrot_group_size}\"\n                )\n            if logical_input_size % 64 or logical_input_size % convrot_group_size:\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} has input size \"\n                    f\"{logical_input_size}, incompatible with quant_group_size=64 \"\n                    f\"and convrot_groupsize={convrot_group_size}\"\n                )\n            continue\n        if marker_format == \"nvfp4\":\n            weight_dtype, weight_shape = checkpoint_meta[f\"{prefix}.weight\"]\n            scale_dtype, scale_shape = checkpoint_meta[f\"{prefix}.weight_scale\"]\n            scale_2_dtype, scale_2_shape = checkpoint_meta[f\"{prefix}.weight_scale_2\"]\n            if weight_dtype != \"U8\" or scale_dtype != \"F8_E4M3\":\n                raise ValueError(\n                    f\"Comfy NVFP4 layer {prefix!r} needs U8 packed weights and \"\n                    f\"FP8 block scales, got {weight_dtype} and {scale_dtype}\"\n                )\n            if scale_2_dtype != \"F32\" or scale_2_shape not in ((), (1,)):\n                raise ValueError(\n                    f\"Comfy NVFP4 layer {prefix!r} needs a scalar F32 \"\n                    f\"weight_scale_2, got {scale_2_dtype}{scale_2_shape}\"","sourceCodeStart":226,"sourceCodeEnd":262,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/quantization_utils.py#L226-L262","documentation":"For each `convrot_w4a4` layer, the checker computes logical_input_size = weight_shape[1] * 2 (two 4-bit values packed per I8 byte) and requires it to be divisible by both 64 (the W4A4 quant group size) and the layer's convrot_groupsize. If either modulus is nonzero, the packed weight layout cannot be evenly divided into quantization groups, so the layer is rejected.","triggerScenarios":"`inspect_comfy_quant_markers` runs on a convrot_w4a4 layer whose packed weight second dimension times 2 is not a multiple of 64 or of convrot_groupsize (16/64/256). Typical when the layer's true input features are, e.g., 48, 96, or any non-multiple of the group size after 2x unpacking.","commonSituations":"Quantizing a model whose convolution/linear input channels aren't a multiple of the group size; a mismatched convrot_groupsize marker vs the actual weight layout; partially exported or truncated weight tensors.","solutions":["Print weight_shape for the failing prefix and verify logical_input_size = shape[1]*2; if it's not a multiple of the group size the model architecture is incompatible with this format","Re-quantize with convrot_groupsize=16 (the most permissive divisor) if the input size is a multiple of 16 but not 64/256","Re-export the checkpoint ensuring the exporter pads or packs weights so the logical input size is group-aligned","If the marker's groupsize is wrong, fix it to match how the weights were actually packed"],"exampleFix":"# weight_shape = (out, 24) -> logical_input_size = 48, fails % 64\n# re-export with groupsize 16:\n{\"format\": \"convrot_w4a4\", \"convrot_groupsize\": 16}","handlingStrategy":"validation","validationCode":"w_shape = checkpoint_meta[f\"{prefix}.weight\"][1]\nlogical_in = w_shape[1] * 2\ngs = int(marker.get(\"convrot_groupsize\", 256))\nassert logical_in % 64 == 0 and logical_in % gs == 0, (\n    f\"{prefix}: logical_input_size={logical_in} incompatible with groups 64/{gs}\"\n)","typeGuard":"def convrot_input_size_ok(weight_shape, convrot_group_size: int) -> bool:\n    logical_in = weight_shape[1] * 2\n    return logical_in % 64 == 0 and logical_in % convrot_group_size == 0","tryCatchPattern":null,"preventionTips":["Pre-check packed weight second-dim alignment (x2 must be multiple of 64 and group size) before loading","When quantizing, prefer architectures whose layer input channels are multiples of 256","Run the marker inspector on a small probe checkpoint right after export, not only at serve time"],"tags":["quantization","comfy","w4a4","shape-mismatch","checkpoint-validation"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}