{"record":{"id":"759ae49f3627bba7","repo":"sgl-project/sglang","slug":"unsupported-quantized-embedding-marker-for-prefix-759ae4","errorCode":null,"errorMessage":"Unsupported quantized embedding marker for {prefix!r}: {marker}","messagePattern":"Unsupported quantized embedding marker for (.+?): (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a8_config.py","lineNumber":99,"sourceCode":"\n    @classmethod\n    def from_config(cls, config: dict[str, Any]) -> KitchenW4A8Config:\n        raise ValueError(\n            \"kitchen_w4a8 is inferred from per-layer checkpoint metadata; \"\n            \"it is not an online quantization method\"\n        )\n\n    def get_quant_method(\n        self, layer: torch.nn.Module, prefix: str\n    ) -> QuantizeMethodBase | None:\n        marker = self.layer_markers.get(prefix)\n        if isinstance(layer, VocabParallelEmbedding):\n            if marker is None:\n                return None\n            if marker.get(\"format\") != \"int8_tensorwise\" or not marker.get(\n                \"_is_tensorwise_scalar\"\n            ):\n                raise ValueError(\n                    f\"Unsupported quantized embedding marker for {prefix!r}: {marker}\"\n                )\n            self.selected.append(prefix)\n            return KitchenInt8EmbeddingMethod()\n        if not isinstance(layer, LinearBase):\n            return None\n        if marker is None:\n            return UnquantizedLinearMethod()\n        if marker.get(\"format\") != \"asym_w4a8_int8\":\n            raise ValueError(f\"Unsupported quantized linear marker for {prefix!r}\")\n\n        group_size = int(marker.get(\"group_size\", 16))\n        convrot_group_size = int(marker.get(\"convrot_groupsize\", 256))\n        if not self._supports_input_size(\n            layer.input_size, group_size, convrot_group_size\n        ):\n            raise ValueError(\n                f\"Serialized W4A8 layer {prefix!r} has input size \"","sourceCodeStart":81,"sourceCodeEnd":117,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a8_config.py#L81-L117","documentation":"KitchenW4A8Config.get_quant_method accepts a VocabParallelEmbedding only if its marker is format 'int8_tensorwise' AND _is_tensorwise_scalar is set. Other embedding markers (e.g. the rowwise variant used by comfy_nvfp4) are rejected.","triggerScenarios":"Loading an embedding whose marker has _is_rowwise instead of _is_tensorwise_scalar, or a non-int8 format, under kitchen_w4a8.","commonSituations":"Reusing markers produced for a different quant config (comfy_nvfp4 rowwise embeddings) with the W4A8 loader.","solutions":["Re-quantize embeddings as tensorwise-scalar int8 (format 'int8_tensorwise', _is_tensorwise_scalar: true)","Drop the embedding marker so it stays unquantized","Load under comfy_nvfp4 if the rowwise int8 embedding is intentional"],"exampleFix":"// before\n{\"format\": \"int8_tensorwise\", \"_is_rowwise\": true}\n// after\n{\"format\": \"int8_tensorwise\", \"_is_tensorwise_scalar\": true}","handlingStrategy":"validation","validationCode":"m = layer_markers.get(prefix)\nif isinstance(layer, VocabParallelEmbedding) and m is not None:\n    assert m.get(\"format\") == \"int8_tensorwise\" and m.get(\"_is_tensorwise_scalar\"), m","typeGuard":"def is_w4a8_embedding_marker(m: dict) -> bool:\n    return m.get(\"format\") == \"int8_tensorwise\" and bool(m.get(\"_is_tensorwise_scalar\"))","tryCatchPattern":null,"preventionTips":["Verify embedding markers match the quant config variant (rowwise vs tensorwise-scalar) before load"],"tags":["quantization","embedding","checkpoint","w4a8"],"backgroundTag":"unsupported-quantization-config","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}