{"record":{"id":"2e6d53c74f0c8da1","repo":"sgl-project/sglang","slug":"serialized-kitchen-int8-layer-prefix-r-has-input","errorCode":null,"errorMessage":"Serialized kitchen_int8 layer {prefix!r} has input size {layer.input_size}, which is not divisible by its ConvRot group size {marker_group_size}","messagePattern":"Serialized kitchen_int8 layer (.+?) has input size (.+?), which is not divisible by its ConvRot group size (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_int8_config.py","lineNumber":112,"sourceCode":"            ignored_layers=cls.get_from_keys_or(config, [\"ignored_layers\"], None),\n        )\n\n    def get_quant_method(\n        self, layer: torch.nn.Module, prefix: str\n    ) -> QuantizeMethodBase | None:\n        from sglang.multimodal_gen.runtime.layers.linear import LinearBase\n        from sglang.multimodal_gen.runtime.layers.quantization.kitchen_int8 import (\n            KitchenInt8LinearMethod,\n        )\n\n        if not isinstance(layer, LinearBase):\n            return None\n        if self.layer_markers is not None:\n            marker_group_size = self._serialized_group_sizes.get(prefix)\n            if marker_group_size is None:\n                return UnquantizedLinearMethod()\n            if layer.input_size % marker_group_size:\n                raise ValueError(\n                    f\"Serialized kitchen_int8 layer {prefix!r} has input size \"\n                    f\"{layer.input_size}, which is not divisible by its \"\n                    f\"ConvRot group size {marker_group_size}\"\n                )\n            self.selected.append(prefix)\n            return KitchenInt8LinearMethod(\n                self,\n                group_size=marker_group_size,\n                is_checkpoint_serialized=True,\n            )\n        if is_layer_skipped(\n            prefix, self.ignored_layers, fused_mapping=self.packed_modules_mapping\n        ):\n            self.skipped.append(prefix)\n            return UnquantizedLinearMethod()\n        # The rotation partitions the input dim into fixed-size groups, so a\n        # layer whose input does not divide evenly simply stays in BF16 rather\n        # than failing the whole model. H3's adaln projections (in=2688) are","sourceCodeStart":94,"sourceCodeEnd":130,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_int8_config.py#L94-L130","documentation":"KitchenInt8Config.get_quant_method checks that the linear layer's input_size is divisible by the marker's convrot_group_size; the ConvRot INT8 kernel processes weights in groups, so a non-divisible size has no valid tiling.","triggerScenarios":"Loading a serialized kitchen_int8 layer whose input_size % convrot_groupsize != 0, e.g. input_size=1000 with group size 128.","commonSituations":"Non-standard hidden sizes in custom architectures; markers copied from a different layer with a larger group size.","solutions":["Use a smaller supported convrot_groupsize that divides input_size (e.g. 128 -> a divisor)","Re-quantize with padding or a group size matching the layer geometry","Leave the layer unserialized (remove its marker) so it uses UnquantizedLinearMethod"],"exampleFix":"// before\n{\"convrot\": true, \"convrot_groupsize\": 256}  # input_size=1536? ok; input_size=1000? fails\n// after\n{\"convrot\": true, \"convrot_groupsize\": 100}  # divides 1000 (if supported)","handlingStrategy":"validation","validationCode":"gs = marker[\"convrot_groupsize\"]\nassert layer.input_size % gs == 0, f\"{layer.input_size} not divisible by {gs}\"","typeGuard":"def input_size_ok(input_size: int, gs: int) -> bool:\n    return input_size % gs == 0","tryCatchPattern":null,"preventionTips":["Choose convrot group sizes as divisors of each layer's input size during quantization"],"tags":["quantization","shape-mismatch","convrot"],"backgroundTag":"shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}