{"record":{"id":"ed2d6621336f6d56","repo":"sgl-project/sglang","slug":"serialized-w4a4-checkpoints-require-cuda-compute-c","errorCode":null,"errorMessage":"Serialized W4A4 checkpoints require CUDA compute capability >= {self.get_min_capability() / 10:.1f}; got {capability.to_int() / 10:.1f}","messagePattern":"Serialized W4A4 checkpoints require CUDA compute capability >= (.+?); got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a4_config.py","lineNumber":44,"sourceCode":"_QUANT_GROUP_SIZE = 64\n_SUPPORTED_CONVROT_GROUP_SIZES = (16, 64, 256)\n_SUPPORTED_LINEAR_DTYPES = (\"int4\", \"int8\")\n\n\nclass KitchenW4A4Config(QuantizationConfig):\n    \"\"\"Dispatch serialized W4A4 linears and their optional INT8 companions.\"\"\"\n\n    def __init__(self, layer_markers: dict[str, dict[str, Any]]) -> None:\n        super().__init__()\n        if current_platform.is_mps():\n            raise ValueError(\"Serialized W4A4 checkpoints are not supported on MPS\")\n        if current_platform.is_cuda():\n            capability = current_platform.get_device_capability()\n            if (\n                capability is not None\n                and capability.to_int() < self.get_min_capability()\n            ):\n                raise ValueError(\n                    \"Serialized W4A4 checkpoints require CUDA compute capability \"\n                    f\">= {self.get_min_capability() / 10:.1f}; got \"\n                    f\"{capability.to_int() / 10:.1f}\"\n                )\n        self.layer_markers = layer_markers\n        self.checkpoint_uses_native_qkv_layout = True\n        self.selected: list[str] = []\n        int8_markers = {\n            prefix: marker\n            for prefix, marker in layer_markers.items()\n            if marker.get(\"format\") == \"int8_tensorwise\"\n        }\n        self._int8_config = (\n            KitchenInt8Config(layer_markers=int8_markers) if int8_markers else None\n        )\n\n        for prefix, marker in layer_markers.items():\n            marker_format = marker.get(\"format\")","sourceCodeStart":26,"sourceCodeEnd":62,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a4_config.py#L26-L62","documentation":"KitchenW4A4Config.__init__ enforces a minimum CUDA compute capability (get_min_capability()). W4A4 kernels need newer GPU instructions (typically sm_80+/sm_100 depending on the build), so older cards are rejected at load.","triggerScenarios":"Loading a W4A4 checkpoint on a CUDA GPU whose capability.to_int() is below get_min_capability(), e.g. a Turing (7.5) or Pascal card.","commonSituations":"Running on older datacenter GPUs (T4, V100) or older consumer cards; Docker images with older torch that misreport capability.","solutions":["Run on a GPU meeting the minimum capability (check the class's get_min_capability)","Use a kitchen_int8 or unquantized export of the model"],"exampleFix":"// before\n--device cuda  # on T4 (7.5)\n// after\n--device cuda  # on H100 (9.0), or load int8 checkpoint","handlingStrategy":"type-guard","validationCode":"if torch.cuda.is_available():\n    cap = torch.cuda.get_device_capability()\n    assert cap[0] * 10 + cap[1] >= KitchenW4A4Config.get_min_capability()","typeGuard":"def gpu_meets_w4a4() -> bool:\n    import torch\n    if not torch.cuda.is_available():\n        return False\n    major, minor = torch.cuda.get_device_capability()\n    return (major * 10 + minor) >= KitchenW4A4Config.get_min_capability()","tryCatchPattern":null,"preventionTips":["Check torch.cuda.get_device_capability() before selecting W4A4 checkpoints"],"tags":["quantization","cuda","compute-capability","gpu-hardware"],"backgroundTag":"gpu-compute-capability-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}