sgl-project/sglang · error · ValueError

kitchen_int8 group_size must be one of {_SUPPORTED_GROUP_SIZ

Error message

kitchen_int8 group_size must be one of {_SUPPORTED_GROUP_SIZES}, got {group_size}

What it means

KitchenInt8Config.__init__ validates that group_size is in _SUPPORTED_GROUP_SIZES and raises immediately otherwise. The ConvRot INT8 kernels only work for specific group sizes, so an out-of-set value cannot be silently coerced.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_int8_config.py:35

logger = init_logger(__name__)

_SUPPORTED_GROUP_SIZES = (16, 64, 256)


class KitchenInt8Config(QuantizationConfig):
    """Dispatch online quantization or serialized Comfy ConvRot layers."""

    def __init__(
        self,
        group_size: int = 256,
        ignored_layers: list[str] | None = None,
        packed_modules_mapping: dict[str, list[str]] | None = None,
        layer_markers: dict[str, dict[str, Any]] | None = None,
    ) -> None:
        super().__init__()
        if group_size not in _SUPPORTED_GROUP_SIZES:
            raise ValueError(
                f"kitchen_int8 group_size must be one of {_SUPPORTED_GROUP_SIZES}, "
                f"got {group_size}"
            )
        self.group_size = group_size
        self.ignored_layers = ignored_layers or []
        self.packed_modules_mapping = packed_modules_mapping or {}
        self.layer_markers = layer_markers
        self.is_checkpoint_int8_serialized = layer_markers is not None
        self.checkpoint_uses_native_qkv_layout = self.is_checkpoint_int8_serialized
        self._serialized_group_sizes: dict[str, int] = {}
        if layer_markers is not None:
            for prefix, marker in layer_markers.items():
                if marker.get("format") != "int8_tensorwise":
                    raise ValueError(
                        f"Unsupported Comfy INT8 format for {prefix!r}: "
                        f"{marker.get('format')!r}"
                    )
                if marker.get("convrot") is not True:

View on GitHub (pinned to 0132848349)

Solutions

1. Read _SUPPORTED_GROUP_SIZES in kitchen_int8_config.py and use one of those values
2. If you need a different group size, re-quantize with a supported scheme

Example fix

// before
KitchenInt8Config(group_size=32)
// after
from ...kitchen_int8_config import _SUPPORTED_GROUP_SIZES
KitchenInt8Config(group_size=next(iter(_SUPPORTED_GROUP_SIZES)))

from sglang.multimodal_gen.runtime.layers.quantization.configs.kitchen_int8_config import _SUPPORTED_GROUP_SIZES
assert group_size in _SUPPORTED_GROUP_SIZES, f"pick from {_SUPPORTED_GROUP_SIZES}"

def is_supported_group_size(gs: int) -> bool:
    return gs in _SUPPORTED_GROUP_SIZES

When it happens

Trigger: Constructing KitchenInt8Config (or passing quant config kwargs) with group_size not in the supported set (check the module's _SUPPORTED_GROUP_SIZES constant).

Common situations: Copying a group_size like 32 or 64 from other quant methods (GPTQ/AWQ) into kitchen_int8; typos in config files.

Related errors

• W4AFP8 group_size must be a positive integer, got {group_siz
• Unsupported Comfy INT8 format for {prefix!r}: {marker.get('f
• Serialized kitchen_int8 layer {prefix!r} must declare convro
• Unsupported Comfy W4A4 format for {prefix!r}: {marker_format
• Serialized W4A4 layer {prefix!r} has unsupported convrot_gro

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/243515b08df68ca5. Report an issue: GitHub.