sgl-project/sglang · critical · ValueError

Serialized W4A4 layer {prefix!r} has unsupported convrot_gro

Error message

Serialized W4A4 layer {prefix!r} has unsupported convrot_groupsize={convrot_group_size}; expected one of {_SUPPORTED_CONVROT_GROUP_SIZES}

What it means

_parse_marker validates convrot_groupsize (default 256) against _SUPPORTED_CONVROT_GROUP_SIZES. Values outside the set cannot be dispatched to a compiled kernel, so __init__/get_quant_method reject them.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a4_config.py:126

        convrot_group_size, linear_dtype = self._parse_marker(prefix, marker)
        if not self._supports_input_size(layer.input_size, convrot_group_size):
            raise ValueError(
                f"Serialized W4A4 layer {prefix!r} has input size "
                f"{layer.input_size}, incompatible with quant_group_size="
                f"{_QUANT_GROUP_SIZE} and convrot_groupsize={convrot_group_size}"
            )
        self.selected.append(prefix)
        return KitchenW4A4LinearMethod(
            convrot_group_size=convrot_group_size,
            linear_dtype=linear_dtype,
        )

    @staticmethod
    def _parse_marker(prefix: str, marker: dict[str, Any]) -> tuple[int, str]:
        convrot_group_size = int(marker.get("convrot_groupsize", 256))
        if convrot_group_size not in _SUPPORTED_CONVROT_GROUP_SIZES:
            raise ValueError(
                f"Serialized W4A4 layer {prefix!r} has unsupported "
                f"convrot_groupsize={convrot_group_size}; expected one of "
                f"{_SUPPORTED_CONVROT_GROUP_SIZES}"
            )
        linear_dtype = str(marker.get("linear_dtype", "int4"))
        if linear_dtype not in _SUPPORTED_LINEAR_DTYPES:
            raise ValueError(
                f"Serialized W4A4 layer {prefix!r} has unsupported "
                f"linear_dtype={linear_dtype!r}; expected one of "
                f"{_SUPPORTED_LINEAR_DTYPES}"
            )
        return convrot_group_size, linear_dtype

    @staticmethod
    def _supports_input_size(input_size: int, convrot_group_size: int) -> bool:
        return (
            input_size % _QUANT_GROUP_SIZE == 0 and input_size % convrot_group_size == 0
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Use a value from _SUPPORTED_CONVROT_GROUP_SIZES in kitchen_w4a4_config.py
2. Upgrade/downgrade the exporter so emitted group sizes match the runtime

Example fix

// before
{"convrot_groupsize": 64}
// after
{"convrot_groupsize": 256}

from ...kitchen_w4a4_config import _SUPPORTED_CONVROT_GROUP_SIZES
assert int(marker.get("convrot_groupsize", 256)) in _SUPPORTED_CONVROT_GROUP_SIZES

def valid_w4a4_convrot_groupsize(m: dict) -> bool:
    return int(m.get("convrot_groupsize", 256)) in _SUPPORTED_CONVROT_GROUP_SIZES

When it happens

Trigger: A W4A4 marker whose 'convrot_groupsize' is missing (defaulting to 256, if 256 is unsupported in the build) or set to an unsupported value like 64 or 512.

Common situations: Exporter and runtime version skew changing the supported set; hand-edited markers.

Related errors

• Serialized kitchen_int8 layer {prefix!r} must declare convro
• Comfy W4A8 layer {prefix!r} has invalid group_size={group_si
• kitchen_int8 group_size must be one of {_SUPPORTED_GROUP_SIZ
• Serialized kitchen_int8 layer {prefix!r} must set convrot=tr
• Serialized kitchen_int8 layer {prefix!r} has input size {lay

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c731ef186cce96a5. Report an issue: GitHub.