{"record":{"id":"84a4ec892737aa30","repo":"sgl-project/sglang","slug":"enable-svdquant-requires-transformer-weights-p","errorCode":null,"errorMessage":"--enable-svdquant requires --transformer-weights-path to be set","messagePattern":"--enable-svdquant requires --transformer-weights-path to be set","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/configs/quantization/nunchaku.py","lineNumber":130,"sourceCode":"        device_count = torch.cuda.device_count()\n\n        unsupported: list[str] = []\n        for i in range(device_count):\n            major, minor = torch.cuda.get_device_capability(i)\n            if major == 9:\n                unsupported.append(f\"cuda:{i} (SM{major}{minor}, Hopper)\")\n            elif major not in (8, 12):\n                unsupported.append(f\"cuda:{i} (SM{major}{minor})\")\n\n        if unsupported:\n            raise ValueError(\n                \"Nunchaku SVDQuant is currently only supported on Ampere (SM8x) or SM12x GPUs; \"\n                f\"Unsupported devices: {', '.join(unsupported)}. \"\n                \"Disable it with --enable-svdquant false.\"\n            )\n\n        if not self.transformer_weights_path:\n            raise ValueError(\n                \"--enable-svdquant requires --transformer-weights-path to be set\"\n            )\n\n        if not is_nunchaku_available():\n            raise ValueError(\n                \"Nunchaku is enabled, but not installed. Please refer to https://nunchaku.tech/docs/nunchaku/installation/installation.html for detailed installation methods.\"\n            )\n\n        if self.quantization_precision not in (\"int4\", \"nvfp4\"):\n            raise ValueError(\n                f\"Invalid --quantization-precision: {self.quantization_precision}. \"\n                \"Must be one of: int4, nvfp4\"\n            )\n\n        if self.quantization_rank <= 0:\n            raise ValueError(\n                f\"Invalid --quantization-rank: {self.quantization_rank}. Must be > 0\"\n            )","sourceCodeStart":112,"sourceCodeEnd":148,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/configs/quantization/nunchaku.py#L112-L148","documentation":"SVDQuant via Nunchaku does not fetch quantized weights itself; you must point it at the quantized transformer weights directory with --transformer-weights-path. The validator raises when enable_svdquant is set but transformer_weights_path is empty.","triggerScenarios":"Starting with --enable-svdquant true without also passing --transformer-weights-path; raised from _validate during resolve_runtime_config at startup, before any model loads.","commonSituations":"Assuming the flag alone switches quantization and the hub checkpoint is enough; typos in the weights-path argument leaving it None; config templates that enable svdquant but omit the weights path field.","solutions":["Download/prepare the SVDQuant-quantized transformer weights (e.g. from the nunchaku/hub releases for your model) and pass --transformer-weights-path /path/to/svdquant-weights","Verify the path exists and contains the expected quantized transformer files","If you did not intend to use SVDQuant, drop --enable-svdquant (or set it to false)"],"exampleFix":"# before\nsglang.launch_server ... --enable-svdquant true\n\n# after\nsglang.launch_server ... --enable-svdquant true \\\n  --transformer-weights-path /models/qwen-image-svdquant-int4","handlingStrategy":"validation","validationCode":"import os\nif cfg.enable_svdquant:\n    assert cfg.transformer_weights_path and os.path.isdir(cfg.transformer_weights_path), \\\n        \"--enable-svdquant requires an existing --transformer-weights-path directory\"","typeGuard":"def svdquant_config_complete(cfg) -> bool:\n    import os\n    return (not cfg.enable_svdquant) or bool(cfg.transformer_weights_path and os.path.exists(cfg.transformer_weights_path))","tryCatchPattern":null,"preventionTips":["Always pair --enable-svdquant with --transformer-weights-path","Download the official SVDQuant weight release for your model ahead of time","Add a preflight check that the weights dir is non-empty before launch"],"tags":["nunchaku","svdquant","missing-path","quantization","startup-config"],"backgroundTag":"missing-required-config-path","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}