{"record":{"id":"b70216f651fce913","repo":"sgl-project/sglang","slug":"serialized-w4a4-layer-prefix-r-has-input-size-l","errorCode":null,"errorMessage":"Serialized W4A4 layer {prefix!r} has input size {layer.input_size}, incompatible with quant_group_size={_QUANT_GROUP_SIZE} and convrot_groupsize={convrot_group_size}","messagePattern":"Serialized W4A4 layer (.+?) has input size (.+?), incompatible with quant_group_size=(.+?) and convrot_groupsize=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a4_config.py","lineNumber":111,"sourceCode":"        )\n\n    def get_quant_method(\n        self, layer: torch.nn.Module, prefix: str\n    ) -> QuantizeMethodBase | None:\n        if not isinstance(layer, LinearBase):\n            return None\n        marker = self.layer_markers.get(prefix)\n        if marker is None:\n            return UnquantizedLinearMethod()\n        if marker.get(\"format\") == \"int8_tensorwise\":\n            assert self._int8_config is not None\n            method = self._int8_config.get_quant_method(layer, prefix)\n            self.selected.append(prefix)\n            return method\n\n        convrot_group_size, linear_dtype = self._parse_marker(prefix, marker)\n        if not self._supports_input_size(layer.input_size, convrot_group_size):\n            raise ValueError(\n                f\"Serialized W4A4 layer {prefix!r} has input size \"\n                f\"{layer.input_size}, incompatible with quant_group_size=\"\n                f\"{_QUANT_GROUP_SIZE} and convrot_groupsize={convrot_group_size}\"\n            )\n        self.selected.append(prefix)\n        return KitchenW4A4LinearMethod(\n            convrot_group_size=convrot_group_size,\n            linear_dtype=linear_dtype,\n        )\n\n    @staticmethod\n    def _parse_marker(prefix: str, marker: dict[str, Any]) -> tuple[int, str]:\n        convrot_group_size = int(marker.get(\"convrot_groupsize\", 256))\n        if convrot_group_size not in _SUPPORTED_CONVROT_GROUP_SIZES:\n            raise ValueError(\n                f\"Serialized W4A4 layer {prefix!r} has unsupported \"\n                f\"convrot_groupsize={convrot_group_size}; expected one of \"\n                f\"{_SUPPORTED_CONVROT_GROUP_SIZES}\"","sourceCodeStart":93,"sourceCodeEnd":129,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a4_config.py#L93-L129","documentation":"get_quant_method verifies via _supports_input_size that the linear's input_size is compatible with both the fixed W4A4 quant group size and the marker's convrot group size. A mismatched geometry cannot be tiled by the fused kernel.","triggerScenarios":"Loading a serialized W4A4 layer whose input_size fails _supports_input_size(input_size, convrot_group_size) — typically not divisible by quant_group_size and/or convrot_groupsize.","commonSituations":"Custom architectures with unusual hidden sizes; markers with convrot_groupsize=256 applied to a layer whose input is not a multiple of it.","solutions":["Choose a supported convrot_groupsize that divides input_size (see _SUPPORTED_CONVROT_GROUP_SIZES)","Re-quantize the model with layer-geometry-aware group sizes"],"exampleFix":"// before\n{\"convrot_groupsize\": 256}  # input_size=1000\n// after\n{\"convrot_groupsize\": 125}  # or re-export with compatible geometry","handlingStrategy":"validation","validationCode":"from ...kitchen_w4a4_config import _QUANT_GROUP_SIZE\ncgs = marker.get(\"convrot_groupsize\", 256)\nassert layer.input_size % _QUANT_GROUP_SIZE == 0 and layer.input_size % cgs == 0","typeGuard":"def w4a4_input_ok(input_size: int, cgs: int, qgs: int) -> bool:\n    return input_size % qgs == 0 and input_size % cgs == 0","tryCatchPattern":null,"preventionTips":["Run a divisibility preflight over (input_size, group sizes) for every serialized linear"],"tags":["quantization","shape-mismatch","w4a4"],"backgroundTag":"shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}