sgl-project/sglang · error · ValueError

Comfy W4A4 layer {prefix!r} needs I8 packed weights and F32

Error message

Comfy W4A4 layer {prefix!r} needs I8 packed weights and F32 scales, got {weight_dtype} and {scale_dtype}

What it means

A Comfy W4A4 (convrot_w4a4) layer must store weights packed into int8 ('I8') and scales in float32 ('F32'). This error means the checkpoint's weight or scale dtype for such a layer deviates, so the W4A4 dequantization kernel cannot be applied.

Source

Thrown at python/sglang/multimodal_gen/runtime/utils/quantization_utils.py:227

                    f"Comfy W4A8 layer {prefix!r} needs an F32[16] codebook"
                )
            expected_correction = (
                logical_input_size // group_size,
                weight_shape[0],
            )
            if marker["_has_correction"] and checkpoint_meta[correction_key] != (
                "F32",
                expected_correction,
            ):
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} has an incompatible correction tensor"
                )
            continue
        if marker_format == "convrot_w4a4":
            weight_dtype, weight_shape = checkpoint_meta[f"{prefix}.weight"]
            scale_dtype, scale_shape = checkpoint_meta[f"{prefix}.weight_scale"]
            if weight_dtype != "I8" or scale_dtype != "F32":
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} needs I8 packed weights and "
                    f"F32 scales, got {weight_dtype} and {scale_dtype}"
                )
            if len(weight_shape) != 2 or scale_shape != (weight_shape[0],):
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} has incompatible weight/scale "
                    f"shapes: {weight_shape} and {scale_shape}"
                )
            logical_input_size = weight_shape[1] * 2
            convrot_group_size = int(marker.get("convrot_groupsize", 256))
            if convrot_group_size not in (16, 64, 256):
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} has unsupported "
                    f"convrot_groupsize={convrot_group_size}"
                )
            if logical_input_size % 64 or logical_input_size % convrot_group_size:
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} has input size "

View on GitHub (pinned to 0132848349)

Solutions

1. Verify dtypes: {prefix}.weight must be I8 and {prefix}.weight_scale must be F32 in safetensors metadata
2. Re-run the Comfy W4A4 quantization recipe to regenerate a consistent checkpoint
3. If the checkpoint is actually W4A8/FP8 format, fix the marker format field or route it to the correct loader

Example fix

# before: weight F16, scale BF16 -> raises
# after:  weight I8 (2x4bit packed), scale F32
quantize_comfy_w4a4(model, groupsize=256, out_dir="out")

wdt, _ = meta[f"{prefix}.weight"]; sdt, _ = meta[f"{prefix}.weight_scale"]
if wdt != "I8" or sdt != "F32":
    raise ValueError(f"not a Comfy W4A4 layer: {wdt}/{sdt}")

def is_comfy_w4a4_layer(meta: dict, prefix: str) -> bool:
    w = meta.get(f"{prefix}.weight"); s = meta.get(f"{prefix}.weight_scale")
    return w is not None and s is not None and w[0] == "I8" and s[0] == "F32"

When it happens

Trigger: inspect_comfy_quant_markers reads {prefix}.weight and {prefix}.weight_scale metadata for a convrot_w4a4 layer and finds weight_dtype != 'I8' or scale_dtype != 'F32' (e.g. FP16 weights from a mixed-precision export, or FP8 scales copied from a W4A8 recipe).

Common situations: Checkpoint quantized for a different backend (W4A8/FP8) but with markers labeled convrot_w4a4, or a generic half-precision conversion script applied to all tensors including packed weights.

Related errors

• Comfy W4A8 layer {prefix!r} needs I8 weights and FP8 group s
• Comfy W4A4 layer {prefix!r} has unsupported convrot_groupsiz
• Type must match: {self.a_dtype} != {self.b_dtype}
• int32-packed scale buffers require scale_ue8m0=True
• scale_ue8m0=True requires an int32-packed output_s

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6b6824bff5680a5e. Report an issue: GitHub.