{"record":{"id":"ff972b6617561670","repo":"sgl-project/sglang","slug":"comfy-w4a8-layer-prefix-r-has-incompatible-weigh","errorCode":null,"errorMessage":"Comfy W4A8 layer {prefix!r} has incompatible weight/scale shapes: {weight_shape}, {scale_shape}, and {channel_shape}","messagePattern":"Comfy W4A8 layer (.+?) has incompatible weight/scale shapes: (.+?), (.+?), and (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/quantization_utils.py","lineNumber":196,"sourceCode":"                    f\"Comfy W4A8 layer {prefix!r} needs I8 weights and FP8 \"\n                    f\"group scales, got {weight_dtype} and {scale_dtype}\"\n                )\n            if channel_dtype != \"F32\":\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs F32 channel scales, \"\n                    f\"got {channel_dtype}\"\n                )\n            if len(weight_shape) != 2:\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs a 2D packed weight, \"\n                    f\"got {weight_shape}\"\n                )\n            logical_input_size = weight_shape[1] * 2\n            expected_scale_shape = (weight_shape[0], logical_input_size // group_size)\n            if scale_shape != expected_scale_shape or channel_shape != (\n                weight_shape[0],\n            ):\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} has incompatible weight/scale \"\n                    f\"shapes: {weight_shape}, {scale_shape}, and {channel_shape}\"\n                )\n            codebook_key = f\"{prefix}.weight_codebook\"\n            correction_key = f\"{prefix}.weight_correction\"\n            marker[\"_has_codebook\"] = codebook_key in checkpoint_meta\n            marker[\"_has_correction\"] = correction_key in checkpoint_meta\n            if marker[\"_has_codebook\"] and checkpoint_meta[codebook_key] != (\n                \"F32\",\n                (16,),\n            ):\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs an F32[16] codebook\"\n                )\n            expected_correction = (\n                logical_input_size // group_size,\n                weight_shape[0],\n            )","sourceCodeStart":178,"sourceCodeEnd":214,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/quantization_utils.py#L178-L214","documentation":"For a Comfy W4A8 layer with packed weight shape (O, I/2), the group scales must be exactly (O, I/group_size) and the channel scales (O,). This error reports that the stored scale shapes are inconsistent with the weight and marker group_size, meaning dequantization would read out of bounds or produce wrong results.","triggerScenarios":"inspect_comfy_quant_markers computes logical_input_size = weight_shape[1] * 2 and expected scale shapes, then finds scale_shape or channel_shape deviates — typically because group_size in the marker disagrees with the one used at export time, or scales were saved for a different layer width.","commonSituations":"Marker's group_size edited after export, checkpoint sliced/fused (e.g. fused QKV changing out features) without regenerating scales, or mixing tensors from two checkpoint versions.","solutions":["Verify the marker's group_size matches the value used when scales were created (expected scale shape is (O, I/group_size))","Re-export the checkpoint so weight and scales are generated together","Check that out_features (weight_shape[0]) matches the channel scale length and no layer fusion changed shapes after quantization"],"exampleFix":"# before: group_size=16 in marker, scales exported with group_size=64\n# after: marker group_size matches exporter (64), scales (O, I/64)\nmarker = {\"format\": \"asym_w4a8_int8\", \"group_size\": 64}","handlingStrategy":"validation","validationCode":"wdt, ws = meta[f\"{prefix}.weight\"]; _, ss = meta[f\"{prefix}.weight_s_rel\"]; _, cs = meta[f\"{prefix}.weight_s_channel\"]\ngs = int(marker.get(\"group_size\", 16))\nlogical_in = ws[1] * 2\nassert ss == (ws[0], logical_in // gs) and cs == (ws[0],), \"scale/weight mismatch\"","typeGuard":"def shapes_are_consistent(ws, ss, cs, group_size) -> bool:\n    logical_in = ws[1] * 2\n    return ss == (ws[0], logical_in // group_size) and cs == (ws[0],)","tryCatchPattern":null,"preventionTips":["Freeze group_size in one place (exporter config) and reuse it in markers","Never quantize then fuse layers; fuse then quantize","Add a post-export self-test that dequantizes one layer and checks shapes"],"tags":["quantization","shape-mismatch","w4a8","group-size"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}