{"record":{"id":"8f44252250ccc479","repo":"sgl-project/sglang","slug":"unsupported-quantized-linear-marker-for-prefix-r-8f4425","errorCode":null,"errorMessage":"Unsupported quantized linear marker for {prefix!r}","messagePattern":"Unsupported quantized linear marker for (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a8_config.py","lineNumber":109,"sourceCode":"    ) -> QuantizeMethodBase | None:\n        marker = self.layer_markers.get(prefix)\n        if isinstance(layer, VocabParallelEmbedding):\n            if marker is None:\n                return None\n            if marker.get(\"format\") != \"int8_tensorwise\" or not marker.get(\n                \"_is_tensorwise_scalar\"\n            ):\n                raise ValueError(\n                    f\"Unsupported quantized embedding marker for {prefix!r}: {marker}\"\n                )\n            self.selected.append(prefix)\n            return KitchenInt8EmbeddingMethod()\n        if not isinstance(layer, LinearBase):\n            return None\n        if marker is None:\n            return UnquantizedLinearMethod()\n        if marker.get(\"format\") != \"asym_w4a8_int8\":\n            raise ValueError(f\"Unsupported quantized linear marker for {prefix!r}\")\n\n        group_size = int(marker.get(\"group_size\", 16))\n        convrot_group_size = int(marker.get(\"convrot_groupsize\", 256))\n        if not self._supports_input_size(\n            layer.input_size, group_size, convrot_group_size\n        ):\n            raise ValueError(\n                f\"Serialized W4A8 layer {prefix!r} has input size \"\n                f\"{layer.input_size}, incompatible with group_size={group_size} \"\n                f\"and convrot_groupsize={convrot_group_size}\"\n            )\n        self.selected.append(prefix)\n        return KitchenW4A8LinearMethod(\n            group_size=group_size,\n            convrot_group_size=convrot_group_size,\n            has_codebook=bool(marker.get(\"_has_codebook\")),\n            has_correction=bool(marker.get(\"_has_correction\")),\n        )","sourceCodeStart":91,"sourceCodeEnd":127,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a8_config.py#L91-L127","documentation":"Thrown by KitchenW4A8Config.get_quant_method when a quantization marker exists on a linear layer but its 'format' field is not 'asym_w4a8_int8'. The config only supports asymmetric W4A8 int8 weights, so any other serialized format is rejected at layer dispatch time.","triggerScenarios":"Calling get_quant_method(layer, prefix) where the layer's marker dict has marker['format'] != 'asym_w4a8_int8' (e.g. 'sym_w4a8', 'w8a8', or a typo), typically while loading a Kitchen-quantized checkpoint whose markers were written by a different quantization recipe.","commonSituations":"Checkpoint quantized with a newer/older kitchen format string; hand-edited marker dicts; mixing checkpoints serialized with symmetric vs asymmetric W4A8 schemes.","solutions":["Inspect the checkpoint's marker metadata to confirm the actual format string","Re-quantize/export the checkpoint with format 'asym_w4a8_int8'","Extend get_quant_method to handle the new format and raise on a narrower set","Fall back to UnquantizedLinearMethod only if unquantized weights are available"],"exampleFix":"# before\nmarker = {\"format\": \"sym_w4a8_int8\", \"group_size\": 16}\nmethod = config.get_quant_method(layer, prefix)  # ValueError\n\n# after\nmarker = {\"format\": \"asym_w4a8_int8\", \"group_size\": 16}\nmethod = config.get_quant_method(layer, prefix)","handlingStrategy":"validation","validationCode":"marker = getattr(layer, \"marker\", None)\nif marker is not None and marker.get(\"format\") != \"asym_w4a8_int8\":\n    raise SystemExit(f\"checkpoint uses format {marker.get('format')!r}; re-export as asym_w4a8_int8\")","typeGuard":"def is_asym_w4a8_marker(marker: object) -> bool:\n    return (\n        isinstance(marker, dict)\n        and marker.get(\"format\") == \"asym_w4a8_int8\"\n        and isinstance(marker.get(\"group_size\", 16), int)\n    )","tryCatchPattern":null,"preventionTips":["Verify the marker format string before loading weights","Pin the quantization export tool version that writes asym_w4a8_int8 markers"],"tags":["quantization","checkpoint","config-validation"],"backgroundTag":"unsupported-checkpoint-format","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}