{"record":{"id":"11193807ed51a401","repo":"sgl-project/sglang","slug":"enable-svdquant-cannot-be-combined-with-a-gguf-t","errorCode":null,"errorMessage":"--enable-svdquant cannot be combined with a GGUF transformer: both supply the transformer weights. Point --transformer-weights-path at either an SVDQuant checkpoint or a .gguf, not one while requesting the other.","messagePattern":"--enable-svdquant cannot be combined with a GGUF transformer: both supply the transformer weights\\. Point --transformer-weights-path at either an SVDQuant checkpoint or a \\.gguf, not one while requesting the other\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/loader/transformer_load_utils.py","lineNumber":539,"sourceCode":"    # either redundant (gguf) or a conflicting request that would otherwise be\n    # dropped without a word.\n    if server_args.quantization == \"gguf\":\n        raise ValueError(\n            \"GGUF is selected by passing the checkpoint itself, not \"\n            \"`--quantization gguf`. Drop the flag; \"\n            \"`--transformer-weights-path <file.gguf>` is what enables it.\"\n        )\n    if server_args.quantization is not None:\n        raise ValueError(\n            f\"--quantization {server_args.quantization} cannot be combined with \"\n            \"a GGUF transformer, whose quantization is fixed by the checkpoint. \"\n            \"Drop the flag, or use an unquantized checkpoint to quantize online.\"\n        )\n    # Nunchaku shares --transformer-weights-path with GGUF, and the GGUF plan is\n    # resolved first, so without this the SVDQuant request would be dropped in\n    # silence rather than refused.\n    if server_args.nunchaku_config is not None:\n        raise ValueError(\n            \"--enable-svdquant cannot be combined with a GGUF transformer: both \"\n            \"supply the transformer weights. Point \"\n            \"--transformer-weights-path at either an SVDQuant checkpoint or a \"\n            \".gguf, not one while requesting the other.\"\n        )\n    if not current_platform.is_cuda():\n        raise ValueError(\n            \"GGUF diffusion checkpoints require CUDA; the GGML kernels have no \"\n            f\"{current_platform.device_type} implementation.\"\n        )\n    uses_fsdp = (\n        server_args.should_use_fsdp_for_component(component_name)\n        if component_name is not None\n        else server_args.use_fsdp_inference\n    )\n    if uses_fsdp:\n        raise ValueError(\n            \"GGUF diffusion checkpoints are incompatible with FSDP inference. \"","sourceCodeStart":521,"sourceCodeEnd":557,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/loader/transformer_load_utils.py#L521-L557","documentation":"Both SVDQuant (Nunchaku) and GGUF supply transformer weights through the shared --transformer-weights-path flag, so requesting both is ambiguous and rejected rather than silently dropping one plan.","triggerScenarios":"Setting server_args.nunchaku_config (via --enable-svdquant) while --transformer-weights-path points at a .gguf file; the GGUF plan resolves first, so the guard raises to avoid dropping the SVDQuant request.","commonSituations":"Switching a server between an SVDQuant checkpoint and a GGUF checkpoint and forgetting to remove the other enabling flag/config.","solutions":["Point --transformer-weights-path at the SVDQuant checkpoint and drop GGUF, or","Point it at the .gguf file and remove --enable-svdquant / nunchaku_config"],"exampleFix":"# before\n--transformer-weights-path model.gguf --enable-svdquant\n# after\n--transformer-weights-path svdquant_checkpoint","handlingStrategy":"validation","validationCode":"if server_args.nunchaku_config is not None and str(server_args.transformer_weights_path or '').endswith('.gguf'):\n    raise SystemExit('SVDQuant and GGUF both use --transformer-weights-path; pick one')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Remember --transformer-weights-path is shared between SVDQuant and GGUF; never enable both"],"tags":["gguf","svdquant","nunchaku","config-conflict"],"backgroundTag":"incompatible-launch-flags","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}