sgl-project/sglang · error · ValueError

Comfy W4A4 layer {prefix!r} has unsupported convrot_groupsiz

Error message

Comfy W4A4 layer {prefix!r} has unsupported convrot_groupsize={convrot_group_size}

What it means

While scanning ComfyUI-format safetensors quantization markers, `inspect_comfy_quant_markers` validates each `convrot_w4a4` layer's `convrot_groupsize` marker field. Only 16, 64, or 256 are supported because the conv-rotation W4A4 kernels are compiled for those group sizes. Any other value (including a missing field defaulting to 256 is fine, but an explicit bad value) raises this ValueError at config-inspection time, before model loading.

Source

Thrown at python/sglang/multimodal_gen/runtime/utils/quantization_utils.py:239

                )
            continue
        if marker_format == "convrot_w4a4":
            weight_dtype, weight_shape = checkpoint_meta[f"{prefix}.weight"]
            scale_dtype, scale_shape = checkpoint_meta[f"{prefix}.weight_scale"]
            if weight_dtype != "I8" or scale_dtype != "F32":
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} needs I8 packed weights and "
                    f"F32 scales, got {weight_dtype} and {scale_dtype}"
                )
            if len(weight_shape) != 2 or scale_shape != (weight_shape[0],):
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} has incompatible weight/scale "
                    f"shapes: {weight_shape} and {scale_shape}"
                )
            logical_input_size = weight_shape[1] * 2
            convrot_group_size = int(marker.get("convrot_groupsize", 256))
            if convrot_group_size not in (16, 64, 256):
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} has unsupported "
                    f"convrot_groupsize={convrot_group_size}"
                )
            if logical_input_size % 64 or logical_input_size % convrot_group_size:
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} has input size "
                    f"{logical_input_size}, incompatible with quant_group_size=64 "
                    f"and convrot_groupsize={convrot_group_size}"
                )
            continue
        if marker_format == "nvfp4":
            weight_dtype, weight_shape = checkpoint_meta[f"{prefix}.weight"]
            scale_dtype, scale_shape = checkpoint_meta[f"{prefix}.weight_scale"]
            scale_2_dtype, scale_2_shape = checkpoint_meta[f"{prefix}.weight_scale_2"]
            if weight_dtype != "U8" or scale_dtype != "F8_E4M3":
                raise ValueError(
                    f"Comfy NVFP4 layer {prefix!r} needs U8 packed weights and "
                    f"FP8 block scales, got {weight_dtype} and {scale_dtype}"

View on GitHub (pinned to 0132848349)

Solutions

1. Check the checkpoint's marker metadata (e.g. via safetensors header inspection) and confirm the convrot_groupsize value actually present
2. Re-export/quantize the model with convrot_groupsize set to 16, 64, or 256
3. Upgrade SGLang — newer builds may support additional group sizes
4. If the value is genuinely 256-equivalent but was serialized incorrectly, patch the marker metadata to a supported value

Example fix

# before: exported with groupsize 128
{"format": "convrot_w4a4", "convrot_groupsize": 128}
# after
{"format": "convrot_w4a4", "convrot_groupsize": 64}

import json
from safetensors import safe_open

with safe_open("model.safetensors", framework="pt") as f:
    meta = f.metadata() or "{}"
markers = json.loads(meta).get("quant_markers", meta)
for prefix, m in markers.items():
    if m.get("format") == "convrot_w4a4":
        gs = int(m.get("convrot_groupsize", 256))
        assert gs in (16, 64, 256), f"{prefix}: unsupported convrot_groupsize={gs}"

def is_supported_convrot_groupsize(marker: dict) -> bool:
    return marker.get("format") != "convrot_w4a4" or int(
        marker.get("convrot_groupsize", 256)
    ) in (16, 64, 256)

When it happens

Trigger: Calling `inspect_comfy_quant_markers(checkpoint_meta, ...)` (directly, or indirectly via `_get_encoder_quant_config` / `inspect_minimax_h3_safetensors`) on a checkpoint where a layer marker has format 'convrot_w4a4' and its JSON/attr field `convrot_groupsize` is set to something outside {16, 64, 256}, e.g. 32, 128, or 0.

Common situations: Exporting a Comfy W4A4 model with a custom/newer quantizer that emits group sizes this SGLang build doesn't support; hand-editing marker metadata; a checkpoint from a different framework version with changed group-size conventions.

Related errors

• Comfy W4A4 layer {prefix!r} has input size {logical_input_si
• Comfy W4A4 layer {prefix!r} needs I8 packed weights and F32
• Comfy full_precision_matrix_mult does not support fused line
• Unsupported Comfy FP8 layer formats: {unsupported}
• ComfyFp8Config must be constructed from safetensors layer ma

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/462a2bc96d2ae2f9. Report an issue: GitHub.