{"record":{"id":"e11f0d740ac140eb","repo":"sgl-project/sglang","slug":"comfy-w4a8-layer-prefix-r-needs-i8-weights-and-f","errorCode":null,"errorMessage":"Comfy W4A8 layer {prefix!r} needs I8 weights and FP8 group scales, got {weight_dtype} and {scale_dtype}","messagePattern":"Comfy W4A8 layer (.+?) needs I8 weights and FP8 group scales, got (.+?) and (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/quantization_utils.py","lineNumber":177,"sourceCode":"                f\"Comfy layer {prefix!r} is missing checkpoint tensors: \"\n                f\"{sorted(missing)}\"\n            )\n        if marker_format == \"float8_e4m3fn\":\n            marker[\"_activation_scheme\"] = (\n                \"static\" if f\"{prefix}.input_scale\" in checkpoint_meta else \"dynamic\"\n            )\n            continue\n        if marker_format == \"asym_w4a8_int8\":\n            weight_dtype, weight_shape = checkpoint_meta[f\"{prefix}.weight\"]\n            scale_dtype, scale_shape = checkpoint_meta[f\"{prefix}.weight_s_rel\"]\n            channel_dtype, channel_shape = checkpoint_meta[f\"{prefix}.weight_s_channel\"]\n            group_size = int(marker.get(\"group_size\", 16))\n            if group_size < 4:\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} has invalid group_size={group_size}\"\n                )\n            if weight_dtype != \"I8\" or scale_dtype != \"F8_E4M3\":\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs I8 weights and FP8 \"\n                    f\"group scales, got {weight_dtype} and {scale_dtype}\"\n                )\n            if channel_dtype != \"F32\":\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs F32 channel scales, \"\n                    f\"got {channel_dtype}\"\n                )\n            if len(weight_shape) != 2:\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs a 2D packed weight, \"\n                    f\"got {weight_shape}\"\n                )\n            logical_input_size = weight_shape[1] * 2\n            expected_scale_shape = (weight_shape[0], logical_input_size // group_size)\n            if scale_shape != expected_scale_shape or channel_shape != (\n                weight_shape[0],\n            ):","sourceCodeStart":159,"sourceCodeEnd":195,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/quantization_utils.py#L159-L195","documentation":"A Comfy W4A8 layer must store its packed weights as int8 (dtype string 'I8' in safetensors metadata) and its per-group scales as FP8 e4m3 ('F8_E4M3'). This error means one of those tensors has a different dtype, indicating the checkpoint was quantized with an incompatible scheme.","triggerScenarios":"inspect_comfy_quant_markers reads {prefix}.weight and {prefix}.weight_s_rel metadata for an asym_w4a8_int8 layer and finds weight_dtype != 'I8' or scale_dtype != 'F8_E4M3' (e.g. F16/BF16 weights or F32 scales).","commonSituations":"Mixing checkpoints from different quant tools (e.g. an AWQ/GPTQ-style checkpoint fed to the Comfy W4A8 path), re-saving safetensors with dtype conversion, or using a model variant exported with a newer/different format version.","solutions":["Verify the checkpoint was produced by the Comfy W4A8 (asym_w4a8_int8) exporter, not another quantizer","Check dtypes: safetensors metadata for {prefix}.weight should read I8 and {prefix}.weight_s_rel should read F8_E4M3","Re-quantize/re-export the model with the correct W4A8 recipe, or load it with the matching quant backend","If the model is genuinely a different format, update the marker format string so it routes to the right handler"],"exampleFix":"# before: weight dtype BF16, scale dtype F32  -> raises\n# after: weight dtype I8 (packed int8), scale dtype F8_E4M3\nquantize_comfy_w4a8(model, out_dir=\"out\")  # exporter writes correct dtypes","handlingStrategy":"validation","validationCode":"wdt, _ = meta[f\"{prefix}.weight\"]; sdt, _ = meta[f\"{prefix}.weight_s_rel\"]\nif wdt != \"I8\" or sdt != \"F8_E4M3\":\n    raise ValueError(f\"not a Comfy W4A8 layer: {wdt}/{sdt}\")","typeGuard":"def is_comfy_w4a8_layer(meta: dict, prefix: str) -> bool:\n    w = meta.get(f\"{prefix}.weight\"); s = meta.get(f\"{prefix}.weight_s_rel\")\n    return w is not None and s is not None and w[0] == \"I8\" and s[0] == \"F8_E4M3\"","tryCatchPattern":null,"preventionTips":["Detect checkpoint format from tensor dtypes before choosing a loader","Don't run dtype-conversion utilities over quantized checkpoints","Keep one quant tool per checkpoint; don't mix exports"],"tags":["quantization","dtype-mismatch","w4a8","safetensors"],"backgroundTag":"tensor-dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}