{"record":{"id":"04157c10c930db3c","repo":"sgl-project/sglang","slug":"serialized-w4a4-checkpoints-are-not-supported-on-m","errorCode":null,"errorMessage":"Serialized W4A4 checkpoints are not supported on MPS","messagePattern":"Serialized W4A4 checkpoints are not supported on MPS","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a4_config.py","lineNumber":37,"sourceCode":"    KitchenInt8Config,\n)\nfrom sglang.multimodal_gen.runtime.layers.quantization.kitchen_w4a4 import (\n    KitchenW4A4LinearMethod,\n)\nfrom sglang.multimodal_gen.runtime.platforms import current_platform\n\n_QUANT_GROUP_SIZE = 64\n_SUPPORTED_CONVROT_GROUP_SIZES = (16, 64, 256)\n_SUPPORTED_LINEAR_DTYPES = (\"int4\", \"int8\")\n\n\nclass KitchenW4A4Config(QuantizationConfig):\n    \"\"\"Dispatch serialized W4A4 linears and their optional INT8 companions.\"\"\"\n\n    def __init__(self, layer_markers: dict[str, dict[str, Any]]) -> None:\n        super().__init__()\n        if current_platform.is_mps():\n            raise ValueError(\"Serialized W4A4 checkpoints are not supported on MPS\")\n        if current_platform.is_cuda():\n            capability = current_platform.get_device_capability()\n            if (\n                capability is not None\n                and capability.to_int() < self.get_min_capability()\n            ):\n                raise ValueError(\n                    \"Serialized W4A4 checkpoints require CUDA compute capability \"\n                    f\">= {self.get_min_capability() / 10:.1f}; got \"\n                    f\"{capability.to_int() / 10:.1f}\"\n                )\n        self.layer_markers = layer_markers\n        self.checkpoint_uses_native_qkv_layout = True\n        self.selected: list[str] = []\n        int8_markers = {\n            prefix: marker\n            for prefix, marker in layer_markers.items()\n            if marker.get(\"format\") == \"int8_tensorwise\"","sourceCodeStart":19,"sourceCodeEnd":55,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a4_config.py#L19-L55","documentation":"KitchenW4A4Config.__init__ hard-fails on MPS because the W4A4 dequant/matmul kernels have no Metal implementation. This check runs before any layers are built, so load fails immediately on Apple Silicon.","triggerScenarios":"Instantiating KitchenW4A4Config on a machine where current_platform.is_mps() is true (torch device type 'mps', Apple Silicon GPU).","commonSituations":"Developing on an M-series Mac and loading a W4A4-serialized Comfy checkpoint; CI running on macOS runners.","solutions":["Run on a CUDA GPU (Linux/NVIDIA) instead","Load an unquantized or int8 (MPS-supported) variant of the checkpoint","Force CPU execution path if supported by the surrounding runtime"],"exampleFix":"// before: run on macOS GPU\n/device mps\n// after\n/device cuda  # or use an fp16/int8 checkpoint","handlingStrategy":"type-guard","validationCode":"import torch\nif torch.backends.mps.is_available() and device == \"mps\":\n    raise SystemExit(\"W4A4 checkpoints need CUDA; use an int8/fp16 export on MPS\")","typeGuard":"def w4a4_supported_here() -> bool:\n    import torch\n    return torch.cuda.is_available()","tryCatchPattern":null,"preventionTips":["Gate quantized-checkpoint selection on device availability in your loader"],"tags":["quantization","mps","platform-support","apple-silicon"],"backgroundTag":"unsupported-platform-operation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}