{"record":{"id":"c3b2dea5f30443a8","repo":"sgl-project/sglang","slug":"comfy-w4a8-layer-prefix-r-needs-a-2d-packed-weig","errorCode":null,"errorMessage":"Comfy W4A8 layer {prefix!r} needs a 2D packed weight, got {weight_shape}","messagePattern":"Comfy W4A8 layer (.+?) needs a 2D packed weight, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/quantization_utils.py","lineNumber":187,"sourceCode":"            scale_dtype, scale_shape = checkpoint_meta[f\"{prefix}.weight_s_rel\"]\n            channel_dtype, channel_shape = checkpoint_meta[f\"{prefix}.weight_s_channel\"]\n            group_size = int(marker.get(\"group_size\", 16))\n            if group_size < 4:\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} has invalid group_size={group_size}\"\n                )\n            if weight_dtype != \"I8\" or scale_dtype != \"F8_E4M3\":\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs I8 weights and FP8 \"\n                    f\"group scales, got {weight_dtype} and {scale_dtype}\"\n                )\n            if channel_dtype != \"F32\":\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs F32 channel scales, \"\n                    f\"got {channel_dtype}\"\n                )\n            if len(weight_shape) != 2:\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs a 2D packed weight, \"\n                    f\"got {weight_shape}\"\n                )\n            logical_input_size = weight_shape[1] * 2\n            expected_scale_shape = (weight_shape[0], logical_input_size // group_size)\n            if scale_shape != expected_scale_shape or channel_shape != (\n                weight_shape[0],\n            ):\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} has incompatible weight/scale \"\n                    f\"shapes: {weight_shape}, {scale_shape}, and {channel_shape}\"\n                )\n            codebook_key = f\"{prefix}.weight_codebook\"\n            correction_key = f\"{prefix}.weight_correction\"\n            marker[\"_has_codebook\"] = codebook_key in checkpoint_meta\n            marker[\"_has_correction\"] = correction_key in checkpoint_meta\n            if marker[\"_has_codebook\"] and checkpoint_meta[codebook_key] != (\n                \"F32\",","sourceCodeStart":169,"sourceCodeEnd":205,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/quantization_utils.py#L169-L205","documentation":"The packed int8 weight of a Comfy W4A8 layer must be a 2D tensor of shape (out_channels, in_features/2) since two 4-bit values are packed per int8. This error means the weight tensor is not 2D (e.g. 1D or 3D), so packed-weight arithmetic cannot proceed.","triggerScenarios":"inspect_comfy_quant_markers finds len(weight_shape) != 2 for {prefix}.weight in an asym_w4a8_int8 layer — for instance a conv layer exported with an extra dimension or an unquantized linear weight of unexpected rank.","commonSituations":"Attempting to quantize/store a conv or other non-matmul layer through the W4A8 path (which only supports 2D linear weights), or an exporter reshaping bug.","solutions":["Confirm the failing prefix is a linear (matmul) layer; conv layers need a different quant format","Re-export ensuring packed weights are reshaped to (out_features, in_features // 2)","Check for exporter bugs that saved transposed/stacked weights"],"exampleFix":"# before: weight shape (K, N, 2) -> raises\n# after:  weight.reshape(K, N * 2) saved as 2D I8 tensor\npacked = pack_w4(w).contiguous().view(out_features, in_features // 2)","handlingStrategy":"validation","validationCode":"_, wshape = meta[f\"{prefix}.weight\"]\nif len(wshape) != 2:\n    raise ValueError(f\"skip: non-2D packed weight {wshape}\")","typeGuard":"def is_2d_packed_weight(meta: dict, prefix: str) -> bool:\n    e = meta.get(f\"{prefix}.weight\")\n    return e is not None and len(e[1]) == 2","tryCatchPattern":null,"preventionTips":["Only quantize linear layers through the W4A8 path","Check weight rank in the exporter before saving"],"tags":["quantization","shape-mismatch","w4a8","safetensors"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}