sgl-project/sglang · critical · ValueError

Serialized W4A4 layer {prefix!r} has input size {layer.input

Error message

Serialized W4A4 layer {prefix!r} has input size {layer.input_size}, incompatible with quant_group_size={_QUANT_GROUP_SIZE} and convrot_groupsize={convrot_group_size}

What it means

get_quant_method verifies via _supports_input_size that the linear's input_size is compatible with both the fixed W4A4 quant group size and the marker's convrot group size. A mismatched geometry cannot be tiled by the fused kernel.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a4_config.py:111

        )

    def get_quant_method(
        self, layer: torch.nn.Module, prefix: str
    ) -> QuantizeMethodBase | None:
        if not isinstance(layer, LinearBase):
            return None
        marker = self.layer_markers.get(prefix)
        if marker is None:
            return UnquantizedLinearMethod()
        if marker.get("format") == "int8_tensorwise":
            assert self._int8_config is not None
            method = self._int8_config.get_quant_method(layer, prefix)
            self.selected.append(prefix)
            return method

        convrot_group_size, linear_dtype = self._parse_marker(prefix, marker)
        if not self._supports_input_size(layer.input_size, convrot_group_size):
            raise ValueError(
                f"Serialized W4A4 layer {prefix!r} has input size "
                f"{layer.input_size}, incompatible with quant_group_size="
                f"{_QUANT_GROUP_SIZE} and convrot_groupsize={convrot_group_size}"
            )
        self.selected.append(prefix)
        return KitchenW4A4LinearMethod(
            convrot_group_size=convrot_group_size,
            linear_dtype=linear_dtype,
        )

    @staticmethod
    def _parse_marker(prefix: str, marker: dict[str, Any]) -> tuple[int, str]:
        convrot_group_size = int(marker.get("convrot_groupsize", 256))
        if convrot_group_size not in _SUPPORTED_CONVROT_GROUP_SIZES:
            raise ValueError(
                f"Serialized W4A4 layer {prefix!r} has unsupported "
                f"convrot_groupsize={convrot_group_size}; expected one of "
                f"{_SUPPORTED_CONVROT_GROUP_SIZES}"

View on GitHub (pinned to 0132848349)

Solutions

1. Choose a supported convrot_groupsize that divides input_size (see _SUPPORTED_CONVROT_GROUP_SIZES)
2. Re-quantize the model with layer-geometry-aware group sizes

Example fix

// before
{"convrot_groupsize": 256}  # input_size=1000
// after
{"convrot_groupsize": 125}  # or re-export with compatible geometry

from ...kitchen_w4a4_config import _QUANT_GROUP_SIZE
cgs = marker.get("convrot_groupsize", 256)
assert layer.input_size % _QUANT_GROUP_SIZE == 0 and layer.input_size % cgs == 0

def w4a4_input_ok(input_size: int, cgs: int, qgs: int) -> bool:
    return input_size % qgs == 0 and input_size % cgs == 0

When it happens

Trigger: Loading a serialized W4A4 layer whose input_size fails _supports_input_size(input_size, convrot_group_size) — typically not divisible by quant_group_size and/or convrot_groupsize.

Common situations: Custom architectures with unusual hidden sizes; markers with convrot_groupsize=256 applied to a layer whose input is not a multiple of it.

Related errors

• Comfy W4A4 layer {prefix!r} has incompatible weight/scale sh
• Comfy W4A4 layer {prefix!r} has input size {logical_input_si
• Shape mismatch: A K={k}, B K={b.shape[1] * 2}
• The input size is not aligned with the quantized weight shap
• Serialized kitchen_int8 layer {prefix!r} has input size {lay

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b70216f651fce913. Report an issue: GitHub.