{"record":{"id":"44ec3833e97d8f08","repo":"sgl-project/sglang","slug":"kitchen-w4a4-is-inferred-from-per-layer-checkpoint","errorCode":null,"errorMessage":"kitchen_w4a4 is inferred from per-layer checkpoint metadata; it is not an online quantization method","messagePattern":"kitchen_w4a4 is inferred from per-layer checkpoint metadata; it is not an online quantization method","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a4_config.py","lineNumber":90,"sourceCode":"    @classmethod\n    def get_name(cls) -> str:\n        return \"kitchen_w4a4\"\n\n    @classmethod\n    def get_supported_act_dtypes(cls) -> list[torch.dtype]:\n        return [torch.bfloat16, torch.float16]\n\n    @classmethod\n    def get_min_capability(cls) -> int:\n        return 75\n\n    @classmethod\n    def get_config_filenames(cls) -> list[str]:\n        return []\n\n    @classmethod\n    def from_config(cls, config: dict[str, Any]) -> KitchenW4A4Config:\n        raise ValueError(\n            \"kitchen_w4a4 is inferred from per-layer checkpoint metadata; \"\n            \"it is not an online quantization method\"\n        )\n\n    def get_quant_method(\n        self, layer: torch.nn.Module, prefix: str\n    ) -> QuantizeMethodBase | None:\n        if not isinstance(layer, LinearBase):\n            return None\n        marker = self.layer_markers.get(prefix)\n        if marker is None:\n            return UnquantizedLinearMethod()\n        if marker.get(\"format\") == \"int8_tensorwise\":\n            assert self._int8_config is not None\n            method = self._int8_config.get_quant_method(layer, prefix)\n            self.selected.append(prefix)\n            return method\n","sourceCodeStart":72,"sourceCodeEnd":108,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a4_config.py#L72-L108","documentation":"KitchenW4A4Config.from_config always raises: the config has no serialized config file (get_config_filenames returns []) and is constructed purely from per-layer checkpoint metadata. Calling from_config is an API misuse.","triggerScenarios":"Any code path that reads a quant config JSON and calls KitchenW4A4Config.from_config(config), e.g. a generic loader keyed by quant method name.","commonSituations":"Generic quantization-registry code assuming every method implements from_config; adding 'kitchen_w4a4' to a hf quantization_config map.","solutions":["Construct KitchenW4A4Config directly with layer_markers extracted from the checkpoint","Do not list kitchen_w4a4 in config-file-driven quantization maps"],"exampleFix":"// before\nKitchenW4A4Config.from_config({})\n// after\nKitchenW4A4Config(layer_markers=extract_layer_markers(checkpoint))","handlingStrategy":"type-guard","validationCode":"if method_name == \"kitchen_w4a4\":\n    cfg = KitchenW4A4Config(layer_markers=markers)  # never from_config","typeGuard":"def supports_from_config(cls) -> bool:\n    return len(cls.get_config_filenames()) > 0","tryCatchPattern":null,"preventionTips":["Treat empty get_config_filenames() as a signal that from_config is unavailable"],"tags":["quantization","api-misuse","config"],"backgroundTag":"unsupported-operation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}