sgl-project/sglang · critical · ValueError

Serialized kitchen_int8 layer {prefix!r} must declare convro

Error message

Serialized kitchen_int8 layer {prefix!r} must declare convrot_groupsize in {_SUPPORTED_GROUP_SIZES}, got {marker_group_size!r}

What it means

Each serialized kitchen_int8 marker must declare convrot_groupsize from the supported set. The group size determines the ConvRot kernel layout, so an unknown or missing value cannot be handled.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_int8_config.py:60

        self.layer_markers = layer_markers
        self.is_checkpoint_int8_serialized = layer_markers is not None
        self.checkpoint_uses_native_qkv_layout = self.is_checkpoint_int8_serialized
        self._serialized_group_sizes: dict[str, int] = {}
        if layer_markers is not None:
            for prefix, marker in layer_markers.items():
                if marker.get("format") != "int8_tensorwise":
                    raise ValueError(
                        f"Unsupported Comfy INT8 format for {prefix!r}: "
                        f"{marker.get('format')!r}"
                    )
                if marker.get("convrot") is not True:
                    raise ValueError(
                        f"Serialized kitchen_int8 layer {prefix!r} must set "
                        "convrot=true"
                    )
                marker_group_size = marker.get("convrot_groupsize")
                if marker_group_size not in _SUPPORTED_GROUP_SIZES:
                    raise ValueError(
                        f"Serialized kitchen_int8 layer {prefix!r} must declare "
                        f"convrot_groupsize in {_SUPPORTED_GROUP_SIZES}, got "
                        f"{marker_group_size!r}"
                    )
                self._serialized_group_sizes[prefix] = marker_group_size
        # Which layers actually got quantized is worth stating plainly in the
        # log: a silent fallback to BF16 looks exactly like a slow kernel.
        self.selected: list[str] = []
        self.skipped: list[str] = []
        self._processed = 0
        self._quantized_bytes = 0

    @classmethod
    def get_name(cls) -> str:
        return "kitchen_int8"

    @classmethod
    def get_supported_act_dtypes(cls) -> list[torch.dtype]:

View on GitHub (pinned to 0132848349)

Solutions

1. Set convrot_groupsize to a value from _SUPPORTED_GROUP_SIZES matching how the weights were quantized
2. Re-export with the matching exporter version

Example fix

// before
{"format": "int8_tensorwise", "convrot": true, "convrot_groupsize": 100}
// after
{"format": "int8_tensorwise", "convrot": true, "convrot_groupsize": 128}

from ...kitchen_int8_config import _SUPPORTED_GROUP_SIZES
for prefix, m in layer_markers.items():
    assert m.get("convrot_groupsize") in _SUPPORTED_GROUP_SIZES, (prefix, m)

def valid_convrot_groupsize(m: dict) -> bool:
    return m.get("convrot_groupsize") in _SUPPORTED_GROUP_SIZES

When it happens

Trigger: A marker whose 'convrot_groupsize' is missing, None, or not in _SUPPORTED_GROUP_SIZES.

Common situations: Typos in convrot_groupsize; exporter version mismatch emitting a different key name (e.g. 'group_size').

Related errors

• Serialized kitchen_int8 layer {prefix!r} must set convrot=tr
• Serialized W4A4 layer {prefix!r} has unsupported convrot_gro
• Serialized W4A8 layer {prefix!r} must set convrot=true
• SGLang diffusion currently supports AutoRound auto_gptq chec
• AutoRound fused module {target!r} has inconsistent shard con

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7fee841efbfadd3d. Report an issue: GitHub.