sgl-project/sglang · error · ValueError

Comfy W4A8 layer {prefix!r} needs a 2D packed weight, got {w

Error message

Comfy W4A8 layer {prefix!r} needs a 2D packed weight, got {weight_shape}

What it means

The packed int8 weight of a Comfy W4A8 layer must be a 2D tensor of shape (out_channels, in_features/2) since two 4-bit values are packed per int8. This error means the weight tensor is not 2D (e.g. 1D or 3D), so packed-weight arithmetic cannot proceed.

Source

Thrown at python/sglang/multimodal_gen/runtime/utils/quantization_utils.py:187

            scale_dtype, scale_shape = checkpoint_meta[f"{prefix}.weight_s_rel"]
            channel_dtype, channel_shape = checkpoint_meta[f"{prefix}.weight_s_channel"]
            group_size = int(marker.get("group_size", 16))
            if group_size < 4:
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} has invalid group_size={group_size}"
                )
            if weight_dtype != "I8" or scale_dtype != "F8_E4M3":
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} needs I8 weights and FP8 "
                    f"group scales, got {weight_dtype} and {scale_dtype}"
                )
            if channel_dtype != "F32":
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} needs F32 channel scales, "
                    f"got {channel_dtype}"
                )
            if len(weight_shape) != 2:
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} needs a 2D packed weight, "
                    f"got {weight_shape}"
                )
            logical_input_size = weight_shape[1] * 2
            expected_scale_shape = (weight_shape[0], logical_input_size // group_size)
            if scale_shape != expected_scale_shape or channel_shape != (
                weight_shape[0],
            ):
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} has incompatible weight/scale "
                    f"shapes: {weight_shape}, {scale_shape}, and {channel_shape}"
                )
            codebook_key = f"{prefix}.weight_codebook"
            correction_key = f"{prefix}.weight_correction"
            marker["_has_codebook"] = codebook_key in checkpoint_meta
            marker["_has_correction"] = correction_key in checkpoint_meta
            if marker["_has_codebook"] and checkpoint_meta[codebook_key] != (
                "F32",

View on GitHub (pinned to 0132848349)

Solutions

1. Confirm the failing prefix is a linear (matmul) layer; conv layers need a different quant format
2. Re-export ensuring packed weights are reshaped to (out_features, in_features // 2)
3. Check for exporter bugs that saved transposed/stacked weights

Example fix

# before: weight shape (K, N, 2) -> raises
# after:  weight.reshape(K, N * 2) saved as 2D I8 tensor
packed = pack_w4(w).contiguous().view(out_features, in_features // 2)

_, wshape = meta[f"{prefix}.weight"]
if len(wshape) != 2:
    raise ValueError(f"skip: non-2D packed weight {wshape}")

def is_2d_packed_weight(meta: dict, prefix: str) -> bool:
    e = meta.get(f"{prefix}.weight")
    return e is not None and len(e[1]) == 2

When it happens

Trigger: inspect_comfy_quant_markers finds len(weight_shape) != 2 for {prefix}.weight in an asym_w4a8_int8 layer — for instance a conv layer exported with an extra dimension or an unquantized linear weight of unexpected rank.

Common situations: Attempting to quantize/store a conv or other non-matmul layer through the W4A8 path (which only supports 2D linear weights), or an exporter reshaping bug.

Related errors

• Comfy W4A8 layer {prefix!r} needs I8 weights and FP8 group s
• Comfy W4A8 layer {prefix!r} has incompatible weight/scale sh
• Comfy W4A8 layer {prefix!r} has an incompatible correction t
• Shape mismatch: A K={k}, B K={b.shape[1] * 2}
• The input size is not aligned with the quantized weight shap

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c3b2dea5f30443a8. Report an issue: GitHub.