{"record":{"id":"eacc993d765ef28b","repo":"sgl-project/sglang","slug":"comfy-w4a8-layer-prefix-r-needs-f32-channel-scal","errorCode":null,"errorMessage":"Comfy W4A8 layer {prefix!r} needs F32 channel scales, got {channel_dtype}","messagePattern":"Comfy W4A8 layer (.+?) needs F32 channel scales, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/quantization_utils.py","lineNumber":182,"sourceCode":"                \"static\" if f\"{prefix}.input_scale\" in checkpoint_meta else \"dynamic\"\n            )\n            continue\n        if marker_format == \"asym_w4a8_int8\":\n            weight_dtype, weight_shape = checkpoint_meta[f\"{prefix}.weight\"]\n            scale_dtype, scale_shape = checkpoint_meta[f\"{prefix}.weight_s_rel\"]\n            channel_dtype, channel_shape = checkpoint_meta[f\"{prefix}.weight_s_channel\"]\n            group_size = int(marker.get(\"group_size\", 16))\n            if group_size < 4:\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} has invalid group_size={group_size}\"\n                )\n            if weight_dtype != \"I8\" or scale_dtype != \"F8_E4M3\":\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs I8 weights and FP8 \"\n                    f\"group scales, got {weight_dtype} and {scale_dtype}\"\n                )\n            if channel_dtype != \"F32\":\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs F32 channel scales, \"\n                    f\"got {channel_dtype}\"\n                )\n            if len(weight_shape) != 2:\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs a 2D packed weight, \"\n                    f\"got {weight_shape}\"\n                )\n            logical_input_size = weight_shape[1] * 2\n            expected_scale_shape = (weight_shape[0], logical_input_size // group_size)\n            if scale_shape != expected_scale_shape or channel_shape != (\n                weight_shape[0],\n            ):\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} has incompatible weight/scale \"\n                    f\"shapes: {weight_shape}, {scale_shape}, and {channel_shape}\"\n                )\n            codebook_key = f\"{prefix}.weight_codebook\"","sourceCodeStart":164,"sourceCodeEnd":200,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/quantization_utils.py#L164-L200","documentation":"Comfy W4A8 layers additionally require a per-channel scale tensor ({prefix}.weight_s_channel) stored as float32. This error fires when that channel-scale tensor exists but its safetensors dtype is not 'F32', which would break the channel-wise dequantization path.","triggerScenarios":"inspect_comfy_quant_markers inspects {prefix}.weight_s_channel for an asym_w4a8_int8 layer and its metadata dtype is anything other than F32 (e.g. BF16, F16, or F64).","commonSituations":"A downcasting safetensors re-save step (e.g. convert to bf16 utility applied blindly to all tensors), or an exporter version that emitted channel scales in half precision.","solutions":["Check the dtype of {prefix}.weight_s_channel in the safetensors metadata and confirm it is F32","Re-export the checkpoint without dtype downcasting of scale tensors","If a conversion script halved the scales, restore from the original export"],"exampleFix":"# before: weight_s_channel stored as BF16 -> raises\n# after:  weight_s_channel stored as F32\nsave_file({\"weight_s_channel\": scale_channel.float()}, \"model.safetensors\")","handlingStrategy":"validation","validationCode":"cdt, _ = meta[f\"{prefix}.weight_s_channel\"]\nif cdt != \"F32\":\n    raise ValueError(f\"channel scales must be F32, got {cdt}\")","typeGuard":"def has_f32_channel_scales(meta: dict, prefix: str) -> bool:\n    e = meta.get(f\"{prefix}.weight_s_channel\")\n    return e is not None and e[0] == \"F32\"","tryCatchPattern":null,"preventionTips":["Exclude scale tensors from any half-precision re-save step","Validate scale dtypes right after export"],"tags":["quantization","dtype-mismatch","w4a8","channel-scales"],"backgroundTag":"tensor-dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}