{"record":{"id":"9ddff4518c7532d8","repo":"sgl-project/sglang","slug":"quantization-server-args-quantization-cannot-b","errorCode":null,"errorMessage":"--quantization {server_args.quantization} cannot be combined with a GGUF transformer, whose quantization is fixed by the checkpoint. Drop the flag, or use an unquantized checkpoint to quantize online.","messagePattern":"--quantization (.+?) cannot be combined with a GGUF transformer, whose quantization is fixed by the checkpoint\\. Drop the flag, or use an unquantized checkpoint to quantize online\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/loader/transformer_load_utils.py","lineNumber":530,"sourceCode":"    Called before the checkpoint is downloaded or read, so an unsupported\n    combination costs a second rather than a multi-gigabyte fetch.\n\n    ``component_name`` selects the FSDP decision to check. FSDP is resolved per\n    component, so a globally enabled ``--use-fsdp-inference`` does not shard a\n    transformer that is offloaded; only the component actually holding the\n    packed weights matters.\n    \"\"\"\n    # The quantization comes from the file, so an explicit --quantization is\n    # either redundant (gguf) or a conflicting request that would otherwise be\n    # dropped without a word.\n    if server_args.quantization == \"gguf\":\n        raise ValueError(\n            \"GGUF is selected by passing the checkpoint itself, not \"\n            \"`--quantization gguf`. Drop the flag; \"\n            \"`--transformer-weights-path <file.gguf>` is what enables it.\"\n        )\n    if server_args.quantization is not None:\n        raise ValueError(\n            f\"--quantization {server_args.quantization} cannot be combined with \"\n            \"a GGUF transformer, whose quantization is fixed by the checkpoint. \"\n            \"Drop the flag, or use an unquantized checkpoint to quantize online.\"\n        )\n    # Nunchaku shares --transformer-weights-path with GGUF, and the GGUF plan is\n    # resolved first, so without this the SVDQuant request would be dropped in\n    # silence rather than refused.\n    if server_args.nunchaku_config is not None:\n        raise ValueError(\n            \"--enable-svdquant cannot be combined with a GGUF transformer: both \"\n            \"supply the transformer weights. Point \"\n            \"--transformer-weights-path at either an SVDQuant checkpoint or a \"\n            \".gguf, not one while requesting the other.\"\n        )\n    if not current_platform.is_cuda():\n        raise ValueError(\n            \"GGUF diffusion checkpoints require CUDA; the GGML kernels have no \"\n            f\"{current_platform.device_type} implementation.\"","sourceCodeStart":512,"sourceCodeEnd":548,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/loader/transformer_load_utils.py#L512-L548","documentation":"The loader refuses to load a GGUF transformer checkpoint when --quantization is also set, because a GGUF file's quantization format is baked into the checkpoint itself and cannot be changed or supplemented by an online quantization method.","triggerScenarios":"Passing server_args.quantization (e.g. --quantization awq) together with a .gguf file via --transformer-weights-path; _validate_gguf_runtime_support raises whenever server_args.quantization is not None on a GGUF load.","commonSituations":"Copy-pasting a launch command from an unquantized/safetensors deployment and just swapping the model path to a .gguf file, leaving the old --quantization flag in place.","solutions":["Remove the --quantization flag; GGUF quantization is fixed by the checkpoint","If you want online quantization with --quantization, use an unquantized (safetensors) checkpoint instead of GGUF"],"exampleFix":"# before\n--transformer-weights-path model.gguf --quantization awq\n# after\n--transformer-weights-path model.gguf","handlingStrategy":"validation","validationCode":"if server_args.quantization is not None and str(server_args.transformer_weights_path or '').endswith('.gguf'):\n    raise SystemExit('Drop --quantization: GGUF quantization is fixed by the checkpoint')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Treat --quantization and GGUF checkpoints as mutually exclusive in launch scripts","Add a config lint step that rejects --quantization when --transformer-weights-path ends in .gguf"],"tags":["gguf","quantization","config-conflict","transformer-loader"],"backgroundTag":"incompatible-launch-flags","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}