sgl-project/sglang · error · ValueError

Comfy W4A8 layer {prefix!r} has incompatible weight/scale sh

Error message

Comfy W4A8 layer {prefix!r} has incompatible weight/scale shapes: {weight_shape}, {scale_shape}, and {channel_shape}

What it means

For a Comfy W4A8 layer with packed weight shape (O, I/2), the group scales must be exactly (O, I/group_size) and the channel scales (O,). This error reports that the stored scale shapes are inconsistent with the weight and marker group_size, meaning dequantization would read out of bounds or produce wrong results.

Source

Thrown at python/sglang/multimodal_gen/runtime/utils/quantization_utils.py:196

                    f"Comfy W4A8 layer {prefix!r} needs I8 weights and FP8 "
                    f"group scales, got {weight_dtype} and {scale_dtype}"
                )
            if channel_dtype != "F32":
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} needs F32 channel scales, "
                    f"got {channel_dtype}"
                )
            if len(weight_shape) != 2:
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} needs a 2D packed weight, "
                    f"got {weight_shape}"
                )
            logical_input_size = weight_shape[1] * 2
            expected_scale_shape = (weight_shape[0], logical_input_size // group_size)
            if scale_shape != expected_scale_shape or channel_shape != (
                weight_shape[0],
            ):
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} has incompatible weight/scale "
                    f"shapes: {weight_shape}, {scale_shape}, and {channel_shape}"
                )
            codebook_key = f"{prefix}.weight_codebook"
            correction_key = f"{prefix}.weight_correction"
            marker["_has_codebook"] = codebook_key in checkpoint_meta
            marker["_has_correction"] = correction_key in checkpoint_meta
            if marker["_has_codebook"] and checkpoint_meta[codebook_key] != (
                "F32",
                (16,),
            ):
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} needs an F32[16] codebook"
                )
            expected_correction = (
                logical_input_size // group_size,
                weight_shape[0],
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Verify the marker's group_size matches the value used when scales were created (expected scale shape is (O, I/group_size))
2. Re-export the checkpoint so weight and scales are generated together
3. Check that out_features (weight_shape[0]) matches the channel scale length and no layer fusion changed shapes after quantization

Example fix

# before: group_size=16 in marker, scales exported with group_size=64
# after: marker group_size matches exporter (64), scales (O, I/64)
marker = {"format": "asym_w4a8_int8", "group_size": 64}

wdt, ws = meta[f"{prefix}.weight"]; _, ss = meta[f"{prefix}.weight_s_rel"]; _, cs = meta[f"{prefix}.weight_s_channel"]
gs = int(marker.get("group_size", 16))
logical_in = ws[1] * 2
assert ss == (ws[0], logical_in // gs) and cs == (ws[0],), "scale/weight mismatch"

def shapes_are_consistent(ws, ss, cs, group_size) -> bool:
    logical_in = ws[1] * 2
    return ss == (ws[0], logical_in // group_size) and cs == (ws[0],)

When it happens

Trigger: inspect_comfy_quant_markers computes logical_input_size = weight_shape[1] * 2 and expected scale shapes, then finds scale_shape or channel_shape deviates — typically because group_size in the marker disagrees with the one used at export time, or scales were saved for a different layer width.

Common situations: Marker's group_size edited after export, checkpoint sliced/fused (e.g. fused QKV changing out features) without regenerating scales, or mixing tensors from two checkpoint versions.

Related errors

• Comfy W4A8 layer {prefix!r} has invalid group_size={group_si
• Comfy W4A8 layer {prefix!r} needs a 2D packed weight, got {w
• Comfy W4A8 layer {prefix!r} has an incompatible correction t
• W4AFP8 shape_k = {shape_k} must be divisible by group_size =
• Shape mismatch: A K={k}, B K={b.shape[1] * 2}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ff972b6617561670. Report an issue: GitHub.