{"record":{"id":"1f1ccd99bc58252f","repo":"sgl-project/sglang","slug":"nunchaku-svdquant-is-currently-only-supported-on-a","errorCode":null,"errorMessage":"Nunchaku SVDQuant is currently only supported on Ampere (SM8x) or SM12x GPUs; Unsupported devices: {', '.join(unsupported)}. Disable it with --enable-svdquant false.","messagePattern":"Nunchaku SVDQuant is currently only supported on Ampere \\(SM8x\\) or SM12x GPUs; Unsupported devices: (.+?)\\. Disable it with --enable-svdquant false\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/configs/quantization/nunchaku.py","lineNumber":123,"sourceCode":"\n        if not current_platform.is_cuda():\n            raise ValueError(\n                \"Nunchaku SVDQuant is only supported on NVIDIA CUDA GPUs \"\n                \"(Ampere SM8x or SM12x).\"\n            )\n\n        device_count = torch.cuda.device_count()\n\n        unsupported: list[str] = []\n        for i in range(device_count):\n            major, minor = torch.cuda.get_device_capability(i)\n            if major == 9:\n                unsupported.append(f\"cuda:{i} (SM{major}{minor}, Hopper)\")\n            elif major not in (8, 12):\n                unsupported.append(f\"cuda:{i} (SM{major}{minor})\")\n\n        if unsupported:\n            raise ValueError(\n                \"Nunchaku SVDQuant is currently only supported on Ampere (SM8x) or SM12x GPUs; \"\n                f\"Unsupported devices: {', '.join(unsupported)}. \"\n                \"Disable it with --enable-svdquant false.\"\n            )\n\n        if not self.transformer_weights_path:\n            raise ValueError(\n                \"--enable-svdquant requires --transformer-weights-path to be set\"\n            )\n\n        if not is_nunchaku_available():\n            raise ValueError(\n                \"Nunchaku is enabled, but not installed. Please refer to https://nunchaku.tech/docs/nunchaku/installation/installation.html for detailed installation methods.\"\n            )\n\n        if self.quantization_precision not in (\"int4\", \"nvfp4\"):\n            raise ValueError(\n                f\"Invalid --quantization-precision: {self.quantization_precision}. \"","sourceCodeStart":105,"sourceCodeEnd":141,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/configs/quantization/nunchaku.py#L105-L141","documentation":"Even on CUDA, Nunchaku SVDQuant kernels only support Ampere (SM8x) and SM12x GPUs. The validator enumerates all visible devices and raises with the list of unsupported ones — Hopper SM9x is explicitly unsupported, as is any other major version outside 8 and 12.","triggerScenarios":"Starting with --enable-svdquant on a machine whose visible devices include H100/H200 (SM9x, listed as 'Hopper') or pre-Ampere (e.g. SM7x Turing) or Blackwell-non-SM12x GPUs; resolve_runtime_config -> _validate raises listing each offending cuda:i.","commonSituations":"H100/H200 clusters where the config was authored on A100; mixed-GPU nodes where one device is unsupported; CUDA_VISIBLE_ORDER/visibility changes exposing an unexpected device; older Turing boxes.","solutions":["Disable the feature: --enable-svdquant false","Restrict visibility to supported GPUs, e.g. CUDA_VISIBLE_DEVICES pointing only at A100/Ampere or SM12x devices","Move the workload to an Ampere (A100/30xx) or SM12x machine","Check torch.cuda.get_device_capability(i) per device to confirm the SM major/minor before enabling"],"exampleFix":"# before\n# all GPUs visible, includes H100 (SM90)\ncfg.enable_svdquant = True\n\n# after\n# run on / restrict to Ampere devices\n# CUDA_VISIBLE_DEVICES=0,1  (A100s only)\ncfg.enable_svdquant = True","handlingStrategy":"validation","validationCode":"import torch\nunsupported = [i for i in range(torch.cuda.device_count())\n               if torch.cuda.get_device_capability(i)[0] not in (8, 12)]\nassert not unsupported, f\"unsupported GPUs: {unsupported}\"","typeGuard":"def all_gpus_svdquant_capable() -> bool:\n    import torch\n    return torch.cuda.is_available() and all(\n        torch.cuda.get_device_capability(i)[0] in (8, 12)\n        for i in range(torch.cuda.device_count())\n    )","tryCatchPattern":null,"preventionTips":["Check torch.cuda.get_device_capability before enabling svdquant","Set CUDA_VISIBLE_DEVICES to only Ampere/SM12x devices on mixed nodes","Note H100/H200 (SM9x) are explicitly unsupported by nunchaku SVDQuant"],"tags":["nunchaku","svdquant","gpu-compatibility","ampere","hopper","quantization"],"backgroundTag":"unsupported-gpu-architecture","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}