{"record":{"id":"853701f61a45c6dc","repo":"sgl-project/sglang","slug":"comfy-w4a8-layer-prefix-r-has-an-incompatible-co","errorCode":null,"errorMessage":"Comfy W4A8 layer {prefix!r} has an incompatible correction tensor","messagePattern":"Comfy W4A8 layer (.+?) has an incompatible correction tensor","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/quantization_utils.py","lineNumber":219,"sourceCode":"            correction_key = f\"{prefix}.weight_correction\"\n            marker[\"_has_codebook\"] = codebook_key in checkpoint_meta\n            marker[\"_has_correction\"] = correction_key in checkpoint_meta\n            if marker[\"_has_codebook\"] and checkpoint_meta[codebook_key] != (\n                \"F32\",\n                (16,),\n            ):\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs an F32[16] codebook\"\n                )\n            expected_correction = (\n                logical_input_size // group_size,\n                weight_shape[0],\n            )\n            if marker[\"_has_correction\"] and checkpoint_meta[correction_key] != (\n                \"F32\",\n                expected_correction,\n            ):\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} has an incompatible correction tensor\"\n                )\n            continue\n        if marker_format == \"convrot_w4a4\":\n            weight_dtype, weight_shape = checkpoint_meta[f\"{prefix}.weight\"]\n            scale_dtype, scale_shape = checkpoint_meta[f\"{prefix}.weight_scale\"]\n            if weight_dtype != \"I8\" or scale_dtype != \"F32\":\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} needs I8 packed weights and \"\n                    f\"F32 scales, got {weight_dtype} and {scale_dtype}\"\n                )\n            if len(weight_shape) != 2 or scale_shape != (weight_shape[0],):\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} has incompatible weight/scale \"\n                    f\"shapes: {weight_shape} and {scale_shape}\"\n                )\n            logical_input_size = weight_shape[1] * 2\n            convrot_group_size = int(marker.get(\"convrot_groupsize\", 256))","sourceCodeStart":201,"sourceCodeEnd":237,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/quantization_utils.py#L201-L237","documentation":"An optional weight correction tensor ({prefix}.weight_correction) for a Comfy W4A8 layer must be F32 with shape (logical_input_size // group_size, out_features) — transposed relative to the group scales. This error fires when the correction tensor exists but its metadata doesn't match that exact dtype/shape.","triggerScenarios":"inspect_comfy_quant_markers finds {prefix}.weight_correction present but checkpoint_meta[correction_key] != ('F32', (I // group_size, O)) — e.g. wrong dtype, stored in (O, groups) orientation, or generated with a different group_size.","commonSituations":"Exporter change transposing the correction layout, correction computed for a different group_size than the marker declares, or a downcast re-save changing F32 to BF16.","solutions":["Compare the stored correction shape against (logical_input_size // group_size, weight_shape[0]) and transpose/re-export if it's stored the other way","Ensure correction dtype is F32","Regenerate the checkpoint with one consistent export so group_size, scales, and corrections all agree"],"exampleFix":"# before: correction shape (O, groups) -> raises\n# after:  correction shape (groups, O) i.e. (I//group_size, out_features)\ncorrection = correction.T.contiguous().float()","handlingStrategy":"validation","validationCode":"corr = meta.get(f\"{prefix}.weight_correction\")\nexpected = (logical_input_size // group_size, out_features)\nif corr is not None and corr != (\"F32\", expected):\n    raise ValueError(f\"bad correction tensor {corr}, expected {expected}\")","typeGuard":"def has_valid_correction(meta, prefix, expected_shape) -> bool:\n    c = meta.get(f\"{prefix}.weight_correction\")\n    return c is None or c == (\"F32\", tuple(expected_shape))","tryCatchPattern":null,"preventionTips":["Regenerate corrections whenever group_size changes","Keep correction tensors in F32","Version-stamp checkpoints so layout changes are detectable"],"tags":["quantization","shape-mismatch","w4a8","correction-tensor"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}