{"record":{"id":"0c439f8fcc7c268e","repo":"sgl-project/sglang","slug":"serialized-w4a8-checkpoints-require-cuda-compute-c","errorCode":null,"errorMessage":"Serialized W4A8 checkpoints require CUDA compute capability >= {self.get_min_capability() / 10:.1f}; got {capability.to_int() / 10:.1f}","messagePattern":"Serialized W4A8 checkpoints require CUDA compute capability >= (.+?); got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a8_config.py","lineNumber":41,"sourceCode":"    VocabParallelEmbedding,\n)\nfrom sglang.multimodal_gen.runtime.platforms import current_platform\n\n\nclass KitchenW4A8Config(QuantizationConfig):\n    \"\"\"Dispatch each linear from its serialized ``asym_w4a8_int8`` marker.\"\"\"\n\n    def __init__(self, layer_markers: dict[str, dict[str, Any]]) -> None:\n        super().__init__()\n        if current_platform.is_mps():\n            raise ValueError(\"Serialized W4A8 checkpoints are not supported on MPS\")\n        if current_platform.is_cuda():\n            capability = current_platform.get_device_capability()\n            if (\n                capability is not None\n                and capability.to_int() < self.get_min_capability()\n            ):\n                raise ValueError(\n                    \"Serialized W4A8 checkpoints require CUDA compute capability \"\n                    f\">= {self.get_min_capability() / 10:.1f}; got \"\n                    f\"{capability.to_int() / 10:.1f}\"\n                )\n        self.layer_markers = layer_markers\n        self.checkpoint_uses_native_qkv_layout = True\n        self.selected: list[str] = []\n\n        for prefix, marker in layer_markers.items():\n            marker_format = marker.get(\"format\")\n            if marker_format == \"int8_tensorwise\" and marker.get(\n                \"_is_tensorwise_scalar\"\n            ):\n                continue\n            if marker_format != \"asym_w4a8_int8\":\n                raise ValueError(\n                    f\"Unsupported Comfy W4A8 format for {prefix!r}: \"\n                    f\"{marker_format!r}\"","sourceCodeStart":23,"sourceCodeEnd":59,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_w4a8_config.py#L23-L59","documentation":"KitchenW4A8Config.__init__ enforces get_min_capability() on CUDA; W4A8 dequant kernels require newer architectures and older GPUs are rejected at load with the actual vs required capability printed.","triggerScenarios":"Loading a W4A8 checkpoint on a CUDA GPU with capability below the minimum (e.g. sm_75 T4).","commonSituations":"Older GPU fleets; misconfigured containers hiding the real GPU model.","solutions":["Move inference to a GPU meeting the minimum capability","Fall back to a kitchen_int8 or fp16 export"],"exampleFix":"// before: T4 (7.5)\n// after: A100/H100 (8.0/9.0) or int8 checkpoint","handlingStrategy":"type-guard","validationCode":"if torch.cuda.is_available():\n    major, minor = torch.cuda.get_device_capability()\n    assert major * 10 + minor >= KitchenW4A8Config.get_min_capability()","typeGuard":"def gpu_meets_w4a8() -> bool:\n    import torch\n    if not torch.cuda.is_available():\n        return False\n    major, minor = torch.cuda.get_device_capability()\n    return (major * 10 + minor) >= KitchenW4A8Config.get_min_capability()","tryCatchPattern":null,"preventionTips":["Print device capability at startup and gate checkpoint choice on it"],"tags":["quantization","cuda","compute-capability","w4a8"],"backgroundTag":"gpu-compute-capability-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}