{"record":{"id":"6b6824bff5680a5e","repo":"sgl-project/sglang","slug":"comfy-w4a4-layer-prefix-r-needs-i8-packed-weight","errorCode":null,"errorMessage":"Comfy W4A4 layer {prefix!r} needs I8 packed weights and F32 scales, got {weight_dtype} and {scale_dtype}","messagePattern":"Comfy W4A4 layer (.+?) needs I8 packed weights and F32 scales, got (.+?) and (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/quantization_utils.py","lineNumber":227,"sourceCode":"                    f\"Comfy W4A8 layer {prefix!r} needs an F32[16] codebook\"\n                )\n            expected_correction = (\n                logical_input_size // group_size,\n                weight_shape[0],\n            )\n            if marker[\"_has_correction\"] and checkpoint_meta[correction_key] != (\n                \"F32\",\n                expected_correction,\n            ):\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} has an incompatible correction tensor\"\n                )\n            continue\n        if marker_format == \"convrot_w4a4\":\n            weight_dtype, weight_shape = checkpoint_meta[f\"{prefix}.weight\"]\n            scale_dtype, scale_shape = checkpoint_meta[f\"{prefix}.weight_scale\"]\n            if weight_dtype != \"I8\" or scale_dtype != \"F32\":\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} needs I8 packed weights and \"\n                    f\"F32 scales, got {weight_dtype} and {scale_dtype}\"\n                )\n            if len(weight_shape) != 2 or scale_shape != (weight_shape[0],):\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} has incompatible weight/scale \"\n                    f\"shapes: {weight_shape} and {scale_shape}\"\n                )\n            logical_input_size = weight_shape[1] * 2\n            convrot_group_size = int(marker.get(\"convrot_groupsize\", 256))\n            if convrot_group_size not in (16, 64, 256):\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} has unsupported \"\n                    f\"convrot_groupsize={convrot_group_size}\"\n                )\n            if logical_input_size % 64 or logical_input_size % convrot_group_size:\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} has input size \"","sourceCodeStart":209,"sourceCodeEnd":245,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/quantization_utils.py#L209-L245","documentation":"A Comfy W4A4 (convrot_w4a4) layer must store weights packed into int8 ('I8') and scales in float32 ('F32'). This error means the checkpoint's weight or scale dtype for such a layer deviates, so the W4A4 dequantization kernel cannot be applied.","triggerScenarios":"inspect_comfy_quant_markers reads {prefix}.weight and {prefix}.weight_scale metadata for a convrot_w4a4 layer and finds weight_dtype != 'I8' or scale_dtype != 'F32' (e.g. FP16 weights from a mixed-precision export, or FP8 scales copied from a W4A8 recipe).","commonSituations":"Checkpoint quantized for a different backend (W4A8/FP8) but with markers labeled convrot_w4a4, or a generic half-precision conversion script applied to all tensors including packed weights.","solutions":["Verify dtypes: {prefix}.weight must be I8 and {prefix}.weight_scale must be F32 in safetensors metadata","Re-run the Comfy W4A4 quantization recipe to regenerate a consistent checkpoint","If the checkpoint is actually W4A8/FP8 format, fix the marker format field or route it to the correct loader"],"exampleFix":"# before: weight F16, scale BF16 -> raises\n# after:  weight I8 (2x4bit packed), scale F32\nquantize_comfy_w4a4(model, groupsize=256, out_dir=\"out\")","handlingStrategy":"validation","validationCode":"wdt, _ = meta[f\"{prefix}.weight\"]; sdt, _ = meta[f\"{prefix}.weight_scale\"]\nif wdt != \"I8\" or sdt != \"F32\":\n    raise ValueError(f\"not a Comfy W4A4 layer: {wdt}/{sdt}\")","typeGuard":"def is_comfy_w4a4_layer(meta: dict, prefix: str) -> bool:\n    w = meta.get(f\"{prefix}.weight\"); s = meta.get(f\"{prefix}.weight_scale\")\n    return w is not None and s is not None and w[0] == \"I8\" and s[0] == \"F32\"","tryCatchPattern":null,"preventionTips":["Route checkpoints by tensor-dtype fingerprint, not just marker names","Never run fp16 conversion scripts over packed int8 checkpoints"],"tags":["quantization","dtype-mismatch","w4a4","safetensors"],"backgroundTag":"tensor-dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}