{"record":{"id":"af05f05df894980a","repo":"sgl-project/sglang","slug":"comfy-w4a8-layer-prefix-r-has-invalid-group-size","errorCode":null,"errorMessage":"Comfy W4A8 layer {prefix!r} has invalid group_size={group_size}","messagePattern":"Comfy W4A8 layer (.+?) has invalid group_size=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/quantization_utils.py","lineNumber":173,"sourceCode":"            continue\n        missing = required - checkpoint_meta.keys()\n        if missing:\n            raise ValueError(\n                f\"Comfy layer {prefix!r} is missing checkpoint tensors: \"\n                f\"{sorted(missing)}\"\n            )\n        if marker_format == \"float8_e4m3fn\":\n            marker[\"_activation_scheme\"] = (\n                \"static\" if f\"{prefix}.input_scale\" in checkpoint_meta else \"dynamic\"\n            )\n            continue\n        if marker_format == \"asym_w4a8_int8\":\n            weight_dtype, weight_shape = checkpoint_meta[f\"{prefix}.weight\"]\n            scale_dtype, scale_shape = checkpoint_meta[f\"{prefix}.weight_s_rel\"]\n            channel_dtype, channel_shape = checkpoint_meta[f\"{prefix}.weight_s_channel\"]\n            group_size = int(marker.get(\"group_size\", 16))\n            if group_size < 4:\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} has invalid group_size={group_size}\"\n                )\n            if weight_dtype != \"I8\" or scale_dtype != \"F8_E4M3\":\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs I8 weights and FP8 \"\n                    f\"group scales, got {weight_dtype} and {scale_dtype}\"\n                )\n            if channel_dtype != \"F32\":\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs F32 channel scales, \"\n                    f\"got {channel_dtype}\"\n                )\n            if len(weight_shape) != 2:\n                raise ValueError(\n                    f\"Comfy W4A8 layer {prefix!r} needs a 2D packed weight, \"\n                    f\"got {weight_shape}\"\n                )\n            logical_input_size = weight_shape[1] * 2","sourceCodeStart":155,"sourceCodeEnd":191,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/quantization_utils.py#L155-L191","documentation":"For Comfy W4A8 (asym_w4a8_int8) layers, the marker's group_size field (defaulting to 16) must be >= 4 because weights are packed 2-per-int8 and group scales are computed per group_size elements. A smaller or nonsensical group_size would make the dequantization math invalid, so it is rejected.","triggerScenarios":"inspect_comfy_quant_markers processes a layer whose marker dict has \"format\": \"asym_w4a8_int8\" and group_size set (or defaulted) to a value < 4, e.g. 0, 1, 2, or a negative number.","commonSituations":"Hand-edited or LLM-generated quant marker JSON with a typo'd group_size, an exporter bug writing group_size=0, or a marker missing group_size while an old code version expected a different default.","solutions":["Open the checkpoint's quant marker metadata and fix group_size to a valid value (>= 4, typically 16/32/64/128)","Re-run quantization with a supported group size so markers are written correctly","If the value is garbage from a corrupt export, re-export the whole checkpoint"],"exampleFix":"# before: marker = {\"format\": \"asym_w4a8_int8\", \"group_size\": 2}\n# after:  marker = {\"format\": \"asym_w4a8_int8\", \"group_size\": 16}","handlingStrategy":"validation","validationCode":"group_size = int(marker.get(\"group_size\", 16))\nif group_size < 4:\n    raise ValueError(f\"refusing to load: bad group_size {group_size}\")\ninspect_comfy_quant_markers(...)","typeGuard":"def is_valid_group_size(marker: dict) -> bool:\n    gs = int(marker.get(\"group_size\", 16))\n    return gs >= 4","tryCatchPattern":null,"preventionTips":["Never hand-edit quant marker JSON numbers","Pin the exporter version that produced checkpoints","Add a schema check for markers in your loading pipeline"],"tags":["quantization","w4a8","group-size","validation"],"backgroundTag":"quantization-config-invalid","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}