sgl-project/sglang · error · ValueError

Comfy W4A4 layer {prefix!r} has incompatible weight/scale sh

Error message

Comfy W4A4 layer {prefix!r} has incompatible weight/scale shapes: {weight_shape} and {scale_shape}

What it means

A Comfy W4A4 layer requires a 2D packed weight (two 4-bit values per int8) and a per-output-channel F32 scale of shape (weight_shape[0],). This error reports the stored weight is not 2D or the scale length doesn't equal the number of output channels.

Source

Thrown at python/sglang/multimodal_gen/runtime/utils/quantization_utils.py:232

            )
            if marker["_has_correction"] and checkpoint_meta[correction_key] != (
                "F32",
                expected_correction,
            ):
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} has an incompatible correction tensor"
                )
            continue
        if marker_format == "convrot_w4a4":
            weight_dtype, weight_shape = checkpoint_meta[f"{prefix}.weight"]
            scale_dtype, scale_shape = checkpoint_meta[f"{prefix}.weight_scale"]
            if weight_dtype != "I8" or scale_dtype != "F32":
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} needs I8 packed weights and "
                    f"F32 scales, got {weight_dtype} and {scale_dtype}"
                )
            if len(weight_shape) != 2 or scale_shape != (weight_shape[0],):
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} has incompatible weight/scale "
                    f"shapes: {weight_shape} and {scale_shape}"
                )
            logical_input_size = weight_shape[1] * 2
            convrot_group_size = int(marker.get("convrot_groupsize", 256))
            if convrot_group_size not in (16, 64, 256):
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} has unsupported "
                    f"convrot_groupsize={convrot_group_size}"
                )
            if logical_input_size % 64 or logical_input_size % convrot_group_size:
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} has input size "
                    f"{logical_input_size}, incompatible with quant_group_size=64 "
                    f"and convrot_groupsize={convrot_group_size}"
                )
            continue
        if marker_format == "nvfp4":

View on GitHub (pinned to 0132848349)

Solutions

1. Check weight rank: it must be 2D (out_channels, in_features//2); use the conv-specific quant format for conv layers
2. Confirm {prefix}.weight_scale is 1D with length equal to weight_shape[0]
3. Re-export the checkpoint with the correct Comfy W4A4 recipe

Example fix

# before: weight 4D conv kernel, scale 2D -> raises
# after:  weight 2D (O, I//2) I8, scale 1D F32 (O,)
scale = scale_per_channel.view(-1)  # shape (out_channels,)

_, ws = meta[f"{prefix}.weight"]; _, ss = meta[f"{prefix}.weight_scale"]
if len(ws) != 2 or ss != (ws[0],):
    raise ValueError(f"bad W4A4 shapes: weight {ws}, scale {ss}")

def w4a4_shapes_valid(meta: dict, prefix: str) -> bool:
    w = meta.get(f"{prefix}.weight"); s = meta.get(f"{prefix}.weight_scale")
    return w is not None and s is not None and len(w[1]) == 2 and s[1] == (w[1][0],)

When it happens

Trigger: inspect_comfy_quant_markers processes a convrot_w4a4 layer where len(weight_shape) != 2 or scale_shape != (weight_shape[0],) — e.g. a 4D conv kernel routed through this path or per-group scales (2D) saved instead of per-channel scales.

Common situations: Applying the linear-layer W4A4 marker to a conv layer (4D weight), exporter writing group scales under the channel-scale tensor name, or checkpoint where weight was saved transposed with an extra dim.

Related errors

• Serialized W4A4 layer {prefix!r} has input size {layer.input
• Comfy W4A4 layer {prefix!r} has input size {logical_input_si
• Shape mismatch: A K={k}, B K={b.shape[1] * 2}
• The input size is not aligned with the quantized weight shap
• Serialized kitchen_int8 layer {prefix!r} has input size {lay

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/46876479b6326b90. Report an issue: GitHub.