sgl-project/sglang · error · ValueError

Comfy W4A8 layer {prefix!r} has an incompatible correction t

Error message

Comfy W4A8 layer {prefix!r} has an incompatible correction tensor

What it means

An optional weight correction tensor ({prefix}.weight_correction) for a Comfy W4A8 layer must be F32 with shape (logical_input_size // group_size, out_features) — transposed relative to the group scales. This error fires when the correction tensor exists but its metadata doesn't match that exact dtype/shape.

Source

Thrown at python/sglang/multimodal_gen/runtime/utils/quantization_utils.py:219

            correction_key = f"{prefix}.weight_correction"
            marker["_has_codebook"] = codebook_key in checkpoint_meta
            marker["_has_correction"] = correction_key in checkpoint_meta
            if marker["_has_codebook"] and checkpoint_meta[codebook_key] != (
                "F32",
                (16,),
            ):
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} needs an F32[16] codebook"
                )
            expected_correction = (
                logical_input_size // group_size,
                weight_shape[0],
            )
            if marker["_has_correction"] and checkpoint_meta[correction_key] != (
                "F32",
                expected_correction,
            ):
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} has an incompatible correction tensor"
                )
            continue
        if marker_format == "convrot_w4a4":
            weight_dtype, weight_shape = checkpoint_meta[f"{prefix}.weight"]
            scale_dtype, scale_shape = checkpoint_meta[f"{prefix}.weight_scale"]
            if weight_dtype != "I8" or scale_dtype != "F32":
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} needs I8 packed weights and "
                    f"F32 scales, got {weight_dtype} and {scale_dtype}"
                )
            if len(weight_shape) != 2 or scale_shape != (weight_shape[0],):
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} has incompatible weight/scale "
                    f"shapes: {weight_shape} and {scale_shape}"
                )
            logical_input_size = weight_shape[1] * 2
            convrot_group_size = int(marker.get("convrot_groupsize", 256))

View on GitHub (pinned to 0132848349)

Solutions

1. Compare the stored correction shape against (logical_input_size // group_size, weight_shape[0]) and transpose/re-export if it's stored the other way
2. Ensure correction dtype is F32
3. Regenerate the checkpoint with one consistent export so group_size, scales, and corrections all agree

Example fix

# before: correction shape (O, groups) -> raises
# after:  correction shape (groups, O) i.e. (I//group_size, out_features)
correction = correction.T.contiguous().float()

corr = meta.get(f"{prefix}.weight_correction")
expected = (logical_input_size // group_size, out_features)
if corr is not None and corr != ("F32", expected):
    raise ValueError(f"bad correction tensor {corr}, expected {expected}")

def has_valid_correction(meta, prefix, expected_shape) -> bool:
    c = meta.get(f"{prefix}.weight_correction")
    return c is None or c == ("F32", tuple(expected_shape))

When it happens

Trigger: inspect_comfy_quant_markers finds {prefix}.weight_correction present but checkpoint_meta[correction_key] != ('F32', (I // group_size, O)) — e.g. wrong dtype, stored in (O, groups) orientation, or generated with a different group_size.

Common situations: Exporter change transposing the correction layout, correction computed for a different group_size than the marker declares, or a downcast re-save changing F32 to BF16.

Related errors

• Comfy W4A8 layer {prefix!r} needs a 2D packed weight, got {w
• Comfy W4A8 layer {prefix!r} has incompatible weight/scale sh
• Shape mismatch: A K={k}, B K={b.shape[1] * 2}
• The input size is not aligned with the quantized weight shap
• Serialized kitchen_int8 layer {prefix!r} has input size {lay

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/853701f61a45c6dc. Report an issue: GitHub.