{"record":{"id":"462a2bc96d2ae2f9","repo":"sgl-project/sglang","slug":"comfy-w4a4-layer-prefix-r-has-unsupported-convro","errorCode":null,"errorMessage":"Comfy W4A4 layer {prefix!r} has unsupported convrot_groupsize={convrot_group_size}","messagePattern":"Comfy W4A4 layer (.+?) has unsupported convrot_groupsize=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/quantization_utils.py","lineNumber":239,"sourceCode":"                )\n            continue\n        if marker_format == \"convrot_w4a4\":\n            weight_dtype, weight_shape = checkpoint_meta[f\"{prefix}.weight\"]\n            scale_dtype, scale_shape = checkpoint_meta[f\"{prefix}.weight_scale\"]\n            if weight_dtype != \"I8\" or scale_dtype != \"F32\":\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} needs I8 packed weights and \"\n                    f\"F32 scales, got {weight_dtype} and {scale_dtype}\"\n                )\n            if len(weight_shape) != 2 or scale_shape != (weight_shape[0],):\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} has incompatible weight/scale \"\n                    f\"shapes: {weight_shape} and {scale_shape}\"\n                )\n            logical_input_size = weight_shape[1] * 2\n            convrot_group_size = int(marker.get(\"convrot_groupsize\", 256))\n            if convrot_group_size not in (16, 64, 256):\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} has unsupported \"\n                    f\"convrot_groupsize={convrot_group_size}\"\n                )\n            if logical_input_size % 64 or logical_input_size % convrot_group_size:\n                raise ValueError(\n                    f\"Comfy W4A4 layer {prefix!r} has input size \"\n                    f\"{logical_input_size}, incompatible with quant_group_size=64 \"\n                    f\"and convrot_groupsize={convrot_group_size}\"\n                )\n            continue\n        if marker_format == \"nvfp4\":\n            weight_dtype, weight_shape = checkpoint_meta[f\"{prefix}.weight\"]\n            scale_dtype, scale_shape = checkpoint_meta[f\"{prefix}.weight_scale\"]\n            scale_2_dtype, scale_2_shape = checkpoint_meta[f\"{prefix}.weight_scale_2\"]\n            if weight_dtype != \"U8\" or scale_dtype != \"F8_E4M3\":\n                raise ValueError(\n                    f\"Comfy NVFP4 layer {prefix!r} needs U8 packed weights and \"\n                    f\"FP8 block scales, got {weight_dtype} and {scale_dtype}\"","sourceCodeStart":221,"sourceCodeEnd":257,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/quantization_utils.py#L221-L257","documentation":"While scanning ComfyUI-format safetensors quantization markers, `inspect_comfy_quant_markers` validates each `convrot_w4a4` layer's `convrot_groupsize` marker field. Only 16, 64, or 256 are supported because the conv-rotation W4A4 kernels are compiled for those group sizes. Any other value (including a missing field defaulting to 256 is fine, but an explicit bad value) raises this ValueError at config-inspection time, before model loading.","triggerScenarios":"Calling `inspect_comfy_quant_markers(checkpoint_meta, ...)` (directly, or indirectly via `_get_encoder_quant_config` / `inspect_minimax_h3_safetensors`) on a checkpoint where a layer marker has format 'convrot_w4a4' and its JSON/attr field `convrot_groupsize` is set to something outside {16, 64, 256}, e.g. 32, 128, or 0.","commonSituations":"Exporting a Comfy W4A4 model with a custom/newer quantizer that emits group sizes this SGLang build doesn't support; hand-editing marker metadata; a checkpoint from a different framework version with changed group-size conventions.","solutions":["Check the checkpoint's marker metadata (e.g. via safetensors header inspection) and confirm the convrot_groupsize value actually present","Re-export/quantize the model with convrot_groupsize set to 16, 64, or 256","Upgrade SGLang — newer builds may support additional group sizes","If the value is genuinely 256-equivalent but was serialized incorrectly, patch the marker metadata to a supported value"],"exampleFix":"# before: exported with groupsize 128\n{\"format\": \"convrot_w4a4\", \"convrot_groupsize\": 128}\n# after\n{\"format\": \"convrot_w4a4\", \"convrot_groupsize\": 64}","handlingStrategy":"validation","validationCode":"import json\nfrom safetensors import safe_open\n\nwith safe_open(\"model.safetensors\", framework=\"pt\") as f:\n    meta = f.metadata() or \"{}\"\nmarkers = json.loads(meta).get(\"quant_markers\", meta)\nfor prefix, m in markers.items():\n    if m.get(\"format\") == \"convrot_w4a4\":\n        gs = int(m.get(\"convrot_groupsize\", 256))\n        assert gs in (16, 64, 256), f\"{prefix}: unsupported convrot_groupsize={gs}\"","typeGuard":"def is_supported_convrot_groupsize(marker: dict) -> bool:\n    return marker.get(\"format\") != \"convrot_w4a4\" or int(\n        marker.get(\"convrot_groupsize\", 256)\n    ) in (16, 64, 256)","tryCatchPattern":null,"preventionTips":["Validate quant marker metadata against supported group sizes before calling inspect_comfy_quant_markers","Use the same exporter version that produced checkpoints this SGLang build was tested with","Add a startup assertion in your model-loading wrapper that fails fast with a clear message on unsupported marker fields"],"tags":["quantization","comfy","w4a4","checkpoint-validation","safetensors"],"backgroundTag":"quantization-config-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}