{"record":{"id":"756f56ac565f43d8","repo":"sgl-project/sglang","slug":"kitchen-w4a8-is-inferred-from-per-layer-checkpoint","errorCode":null,"errorMessage":"kitchen_w4a8 is inferred from per-layer checkpoint metadata; it is not an online quantization method","messagePattern":"kitchen_w4a8 is inferred from per-layer checkpoint metadata; it is not an online quantization method","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a8_config.py","lineNumber":84,"sourceCode":"    @classmethod\n    def get_name(cls) -> str:\n        return \"kitchen_w4a8\"\n\n    @classmethod\n    def get_supported_act_dtypes(cls) -> list[torch.dtype]:\n        return [torch.bfloat16, torch.float16]\n\n    @classmethod\n    def get_min_capability(cls) -> int:\n        return 80\n\n    @classmethod\n    def get_config_filenames(cls) -> list[str]:\n        return []\n\n    @classmethod\n    def from_config(cls, config: dict[str, Any]) -> KitchenW4A8Config:\n        raise ValueError(\n            \"kitchen_w4a8 is inferred from per-layer checkpoint metadata; \"\n            \"it is not an online quantization method\"\n        )\n\n    def get_quant_method(\n        self, layer: torch.nn.Module, prefix: str\n    ) -> QuantizeMethodBase | None:\n        marker = self.layer_markers.get(prefix)\n        if isinstance(layer, VocabParallelEmbedding):\n            if marker is None:\n                return None\n            if marker.get(\"format\") != \"int8_tensorwise\" or not marker.get(\n                \"_is_tensorwise_scalar\"\n            ):\n                raise ValueError(\n                    f\"Unsupported quantized embedding marker for {prefix!r}: {marker}\"\n                )\n            self.selected.append(prefix)","sourceCodeStart":66,"sourceCodeEnd":102,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a8_config.py#L66-L102","documentation":"KitchenW4A8Config.from_config always raises: like W4A4, this method is metadata-driven (per-layer markers) and has no config file representation. Calling from_config is unsupported API usage.","triggerScenarios":"A generic loader resolving the quant method by name and calling from_config on the class.","commonSituations":"Adding kitchen_w4a8 to a JSON quantization_config; registry code that assumes from_config exists for all methods.","solutions":["Construct KitchenW4A8Config(layer_markers=...) directly from checkpoint metadata","Exclude kitchen_w4a8 from config-file-driven loading paths"],"exampleFix":"// before\nKitchenW4A8Config.from_config(cfg)\n// after\nKitchenW4A8Config(layer_markers=markers)","handlingStrategy":"type-guard","validationCode":"if method_name == \"kitchen_w4a8\":\n    cfg = KitchenW4A8Config(layer_markers=markers)  # never from_config","typeGuard":"def supports_from_config(cls) -> bool:\n    return len(cls.get_config_filenames()) > 0","tryCatchPattern":null,"preventionTips":["Branch metadata-driven quant methods away from config-file loading paths"],"tags":["quantization","api-misuse","w4a8"],"backgroundTag":"unsupported-operation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}