sgl-project/sglang · error · ValueError

Comfy W4A8 layer {prefix!r} has invalid group_size={group_si

Error message

Comfy W4A8 layer {prefix!r} has invalid group_size={group_size}

What it means

For Comfy W4A8 (asym_w4a8_int8) layers, the marker's group_size field (defaulting to 16) must be >= 4 because weights are packed 2-per-int8 and group scales are computed per group_size elements. A smaller or nonsensical group_size would make the dequantization math invalid, so it is rejected.

Source

Thrown at python/sglang/multimodal_gen/runtime/utils/quantization_utils.py:173

            continue
        missing = required - checkpoint_meta.keys()
        if missing:
            raise ValueError(
                f"Comfy layer {prefix!r} is missing checkpoint tensors: "
                f"{sorted(missing)}"
            )
        if marker_format == "float8_e4m3fn":
            marker["_activation_scheme"] = (
                "static" if f"{prefix}.input_scale" in checkpoint_meta else "dynamic"
            )
            continue
        if marker_format == "asym_w4a8_int8":
            weight_dtype, weight_shape = checkpoint_meta[f"{prefix}.weight"]
            scale_dtype, scale_shape = checkpoint_meta[f"{prefix}.weight_s_rel"]
            channel_dtype, channel_shape = checkpoint_meta[f"{prefix}.weight_s_channel"]
            group_size = int(marker.get("group_size", 16))
            if group_size < 4:
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} has invalid group_size={group_size}"
                )
            if weight_dtype != "I8" or scale_dtype != "F8_E4M3":
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} needs I8 weights and FP8 "
                    f"group scales, got {weight_dtype} and {scale_dtype}"
                )
            if channel_dtype != "F32":
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} needs F32 channel scales, "
                    f"got {channel_dtype}"
                )
            if len(weight_shape) != 2:
                raise ValueError(
                    f"Comfy W4A8 layer {prefix!r} needs a 2D packed weight, "
                    f"got {weight_shape}"
                )
            logical_input_size = weight_shape[1] * 2

View on GitHub (pinned to 0132848349)

Solutions

1. Open the checkpoint's quant marker metadata and fix group_size to a valid value (>= 4, typically 16/32/64/128)
2. Re-run quantization with a supported group size so markers are written correctly
3. If the value is garbage from a corrupt export, re-export the whole checkpoint

Example fix

# before: marker = {"format": "asym_w4a8_int8", "group_size": 2}
# after:  marker = {"format": "asym_w4a8_int8", "group_size": 16}

group_size = int(marker.get("group_size", 16))
if group_size < 4:
    raise ValueError(f"refusing to load: bad group_size {group_size}")
inspect_comfy_quant_markers(...)

def is_valid_group_size(marker: dict) -> bool:
    gs = int(marker.get("group_size", 16))
    return gs >= 4

When it happens

Trigger: inspect_comfy_quant_markers processes a layer whose marker dict has "format": "asym_w4a8_int8" and group_size set (or defaulted) to a value < 4, e.g. 0, 1, 2, or a negative number.

Common situations: Hand-edited or LLM-generated quant marker JSON with a typo'd group_size, an exporter bug writing group_size=0, or a marker missing group_size while an old code version expected a different default.

Related errors

• Serialized W4A4 layer {prefix!r} has unsupported convrot_gro
• Unsupported Comfy W4A8 format for {prefix!r}: {marker_format
• Comfy W4A8 layer {prefix!r} has incompatible weight/scale sh
• kitchen_int8 group_size must be one of {_SUPPORTED_GROUP_SIZ
• Serialized kitchen_int8 layer {prefix!r} must declare convro

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/af05f05df894980a. Report an issue: GitHub.