{"record":{"id":"3003fe7e57f59ea8","repo":"sgl-project/sglang","slug":"serialized-w4a8-layer-prefix-r-has-input-size-l","errorCode":null,"errorMessage":"Serialized W4A8 layer {prefix!r} has input size {layer.input_size}, incompatible with group_size={group_size} and convrot_groupsize={convrot_group_size}","messagePattern":"Serialized W4A8 layer (.+?) has input size (.+?), incompatible with group_size=(.+?) and convrot_groupsize=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a8_config.py","lineNumber":116,"sourceCode":"            ):\n                raise ValueError(\n                    f\"Unsupported quantized embedding marker for {prefix!r}: {marker}\"\n                )\n            self.selected.append(prefix)\n            return KitchenInt8EmbeddingMethod()\n        if not isinstance(layer, LinearBase):\n            return None\n        if marker is None:\n            return UnquantizedLinearMethod()\n        if marker.get(\"format\") != \"asym_w4a8_int8\":\n            raise ValueError(f\"Unsupported quantized linear marker for {prefix!r}\")\n\n        group_size = int(marker.get(\"group_size\", 16))\n        convrot_group_size = int(marker.get(\"convrot_groupsize\", 256))\n        if not self._supports_input_size(\n            layer.input_size, group_size, convrot_group_size\n        ):\n            raise ValueError(\n                f\"Serialized W4A8 layer {prefix!r} has input size \"\n                f\"{layer.input_size}, incompatible with group_size={group_size} \"\n                f\"and convrot_groupsize={convrot_group_size}\"\n            )\n        self.selected.append(prefix)\n        return KitchenW4A8LinearMethod(\n            group_size=group_size,\n            convrot_group_size=convrot_group_size,\n            has_codebook=bool(marker.get(\"_has_codebook\")),\n            has_correction=bool(marker.get(\"_has_correction\")),\n        )\n\n    @staticmethod\n    def _supports_input_size(\n        input_size: int, group_size: int, convrot_group_size: int\n    ) -> bool:\n        return (\n            group_size >= 4","sourceCodeStart":98,"sourceCodeEnd":134,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a8_config.py#L98-L134","documentation":"Thrown when a serialized W4A8 layer's input_size is not divisible compatible with the group_size and convrot_group_size from its marker (checked via _supports_input_size). W4A8 packing requires the input dimension to align with the weight grouping and the conv-rotation group size, otherwise dequantization kernels would read out of bounds.","triggerScenarios":"get_quant_method on a LinearBase whose layer.input_size is not a multiple of (or otherwise incompatible with) group_size (default 16) and convrot_groupsize (default 256), e.g. input_size=1000 with group_size=16 and convrot_groupsize=256.","commonSituations":"Loading a checkpoint quantized for a different architecture width; non-standard hidden sizes (e.g. 3584, 5504) that are not multiples of 256; a marker with a custom convrot_groupsize that doesn't divide the input size.","solutions":["Check layer.input_size % group_size == 0 and input_size % convrot_groupsize == 0 (per _supports_input_size logic)","Re-quantize the model with a convrot_groupsize that divides the input size (e.g. 128 or the input size itself)","Leave that layer unquantized (remove its marker) and let it fall back to UnquantizedLinearMethod"],"exampleFix":"# before\nmarker = {\"format\": \"asym_w4a8_int8\", \"group_size\": 16, \"convrot_groupsize\": 256}\n# layer.input_size = 5504 -> not a multiple of 256 -> ValueError\n\n# after\nmarker = {\"format\": \"asym_w4a8_int8\", \"group_size\": 16, \"convrot_groupsize\": 172}\n# 5504 % 172 == 0 -> passes","handlingStrategy":"validation","validationCode":"def supports_input_size(input_size: int, group_size: int = 16, convrot: int = 256) -> bool:\n    return input_size % group_size == 0 and input_size % convrot == 0\n\nif not supports_input_size(layer.input_size, marker.get(\"group_size\", 16), marker.get(\"convrot_groupsize\", 256)):\n    layer.marker = None  # fall back to unquantized","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check input_size divisibility before assigning markers","Prefer convrot_groupsize that divides every quantized layer's input size"],"tags":["quantization","dimension-mismatch","checkpoint"],"backgroundTag":"shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}