{"record":{"id":"46876479b6326b90","repo":"sgl-project/sglang","slug":"comfy-w4a4-layer-prefix-r-has-incompatible-weigh","errorCode":null,"errorMessage":"Comfy W4A4 layer {prefix!r} has incompatible weight/scale shapes: {weight_shape} and {scale_shape}","messagePattern":"Comfy W4A4 layer (.+?) has incompatible weight/scale shapes: (.+?) and (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/quantization_utils.py","lineNumber":232,"sourceCode":"            )\n            if marker[\"_has_correction\"] and checkpoint_meta[correction_key] != (\n                \"F32\",\n                expected_correction,\n            ):\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} has an incompatible correction tensor\"\n                )\n            continue\n        if marker_format == \"convrot_w4a4\":\n            weight_dtype, weight_shape = checkpoint_meta[f\"{prefix}.weight\"]\n            scale_dtype, scale_shape = checkpoint_meta[f\"{prefix}.weight_scale\"]\n            if weight_dtype != \"I8\" or scale_dtype != \"F32\":\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} needs I8 packed weights and \"\n                    f\"F32 scales, got {weight_dtype} and {scale_dtype}\"\n                )\n            if len(weight_shape) != 2 or scale_shape != (weight_shape[0],):\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} has incompatible weight/scale \"\n                    f\"shapes: {weight_shape} and {scale_shape}\"\n                )\n            logical_input_size = weight_shape[1] * 2\n            convrot_group_size = int(marker.get(\"convrot_groupsize\", 256))\n            if convrot_group_size not in (16, 64, 256):\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} has unsupported \"\n                    f\"convrot_groupsize={convrot_group_size}\"\n                )\n            if logical_input_size % 64 or logical_input_size % convrot_group_size:\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} has input size \"\n                    f\"{logical_input_size}, incompatible with quant_group_size=64 \"\n                    f\"and convrot_groupsize={convrot_group_size}\"\n                )\n            continue\n        if marker_format == \"nvfp4\":","sourceCodeStart":214,"sourceCodeEnd":250,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/quantization_utils.py#L214-L250","documentation":"A Comfy W4A4 layer requires a 2D packed weight (two 4-bit values per int8) and a per-output-channel F32 scale of shape (weight_shape[0],). This error reports the stored weight is not 2D or the scale length doesn't equal the number of output channels.","triggerScenarios":"inspect_comfy_quant_markers processes a convrot_w4a4 layer where len(weight_shape) != 2 or scale_shape != (weight_shape[0],) — e.g. a 4D conv kernel routed through this path or per-group scales (2D) saved instead of per-channel scales.","commonSituations":"Applying the linear-layer W4A4 marker to a conv layer (4D weight), exporter writing group scales under the channel-scale tensor name, or checkpoint where weight was saved transposed with an extra dim.","solutions":["Check weight rank: it must be 2D (out_channels, in_features//2); use the conv-specific quant format for conv layers","Confirm {prefix}.weight_scale is 1D with length equal to weight_shape[0]","Re-export the checkpoint with the correct Comfy W4A4 recipe"],"exampleFix":"# before: weight 4D conv kernel, scale 2D -> raises\n# after:  weight 2D (O, I//2) I8, scale 1D F32 (O,)\nscale = scale_per_channel.view(-1)  # shape (out_channels,)","handlingStrategy":"validation","validationCode":"_, ws = meta[f\"{prefix}.weight\"]; _, ss = meta[f\"{prefix}.weight_scale\"]\nif len(ws) != 2 or ss != (ws[0],):\n    raise ValueError(f\"bad W4A4 shapes: weight {ws}, scale {ss}\")","typeGuard":"def w4a4_shapes_valid(meta: dict, prefix: str) -> bool:\n    w = meta.get(f\"{prefix}.weight\"); s = meta.get(f\"{prefix}.weight_scale\")\n    return w is not None and s is not None and len(w[1]) == 2 and s[1] == (w[1][0],)","tryCatchPattern":null,"preventionTips":["Use conv-specific quant formats for conv layers","Validate per-channel scale length equals out_features at export time"],"tags":["quantization","shape-mismatch","w4a4","scales"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}