sgl-project/sglang · error · ValueError

Comfy W4A4 layer {prefix!r} has input size {logical_input_si

Error message

Comfy W4A4 layer {prefix!r} has input size {logical_input_size}, incompatible with quant_group_size=64 and convrot_groupsize={convrot_group_size}

What it means

For each `convrot_w4a4` layer, the checker computes logical_input_size = weight_shape[1] * 2 (two 4-bit values packed per I8 byte) and requires it to be divisible by both 64 (the W4A4 quant group size) and the layer's convrot_groupsize. If either modulus is nonzero, the packed weight layout cannot be evenly divided into quantization groups, so the layer is rejected.

Source

Thrown at python/sglang/multimodal_gen/runtime/utils/quantization_utils.py:244

            if weight_dtype != "I8" or scale_dtype != "F32":
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} needs I8 packed weights and "
                    f"F32 scales, got {weight_dtype} and {scale_dtype}"
                )
            if len(weight_shape) != 2 or scale_shape != (weight_shape[0],):
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} has incompatible weight/scale "
                    f"shapes: {weight_shape} and {scale_shape}"
                )
            logical_input_size = weight_shape[1] * 2
            convrot_group_size = int(marker.get("convrot_groupsize", 256))
            if convrot_group_size not in (16, 64, 256):
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} has unsupported "
                    f"convrot_groupsize={convrot_group_size}"
                )
            if logical_input_size % 64 or logical_input_size % convrot_group_size:
                raise ValueError(
                    f"Comfy W4A4 layer {prefix!r} has input size "
                    f"{logical_input_size}, incompatible with quant_group_size=64 "
                    f"and convrot_groupsize={convrot_group_size}"
                )
            continue
        if marker_format == "nvfp4":
            weight_dtype, weight_shape = checkpoint_meta[f"{prefix}.weight"]
            scale_dtype, scale_shape = checkpoint_meta[f"{prefix}.weight_scale"]
            scale_2_dtype, scale_2_shape = checkpoint_meta[f"{prefix}.weight_scale_2"]
            if weight_dtype != "U8" or scale_dtype != "F8_E4M3":
                raise ValueError(
                    f"Comfy NVFP4 layer {prefix!r} needs U8 packed weights and "
                    f"FP8 block scales, got {weight_dtype} and {scale_dtype}"
                )
            if scale_2_dtype != "F32" or scale_2_shape not in ((), (1,)):
                raise ValueError(
                    f"Comfy NVFP4 layer {prefix!r} needs a scalar F32 "
                    f"weight_scale_2, got {scale_2_dtype}{scale_2_shape}"

View on GitHub (pinned to 0132848349)

Solutions

Print weight_shape for the failing prefix and verify logical_input_size = shape[1]*2; if it's not a multiple of the group size the model architecture is incompatible with this format
Re-quantize with convrot_groupsize=16 (the most permissive divisor) if the input size is a multiple of 16 but not 64/256
Re-export the checkpoint ensuring the exporter pads or packs weights so the logical input size is group-aligned
If the marker's groupsize is wrong, fix it to match how the weights were actually packed

Example fix

# weight_shape = (out, 24) -> logical_input_size = 48, fails % 64
# re-export with groupsize 16:
{"format": "convrot_w4a4", "convrot_groupsize": 16}

Defensive patterns

Strategy: validation

Validate before calling

w_shape = checkpoint_meta[f"{prefix}.weight"][1]
logical_in = w_shape[1] * 2
gs = int(marker.get("convrot_groupsize", 256))
assert logical_in % 64 == 0 and logical_in % gs == 0, (
    f"{prefix}: logical_input_size={logical_in} incompatible with groups 64/{gs}"
)

Type guard

def convrot_input_size_ok(weight_shape, convrot_group_size: int) -> bool:
    logical_in = weight_shape[1] * 2
    return logical_in % 64 == 0 and logical_in % convrot_group_size == 0

Prevention

Pre-check packed weight second-dim alignment (x2 must be multiple of 64 and group size) before loading
When quantizing, prefer architectures whose layer input channels are multiples of 256
Run the marker inspector on a small probe checkpoint right after export, not only at serve time

When it happens

Trigger: `inspect_comfy_quant_markers` runs on a convrot_w4a4 layer whose packed weight second dimension times 2 is not a multiple of 64 or of convrot_groupsize (16/64/256). Typical when the layer's true input features are, e.g., 48, 96, or any non-multiple of the group size after 2x unpacking.

Common situations: Quantizing a model whose convolution/linear input channels aren't a multiple of the group size; a mismatched convrot_groupsize marker vs the actual weight layout; partially exported or truncated weight tensors.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9199a8b4deddf545. Report an issue: GitHub.