{"record":{"id":"5b20d05ad8162ccd","repo":"sgl-project/sglang","slug":"per-layer-checkpoint-quantization-and-nunchaku-are","errorCode":null,"errorMessage":"Per-layer checkpoint quantization and Nunchaku are mutually exclusive","messagePattern":"Per-layer checkpoint quantization and Nunchaku are mutually exclusive","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/loader/transformer_load_utils.py","lineNumber":793,"sourceCode":") -> TransformerQuantLoadSpec:\n    if gguf_file is not None:\n        if checkpoint_quant_config is not None:\n            raise ValueError(\"GGUF and safetensors quantization metadata conflict\")\n        return _resolve_gguf_quant_load_spec(\n            gguf_file=gguf_file,\n            server_args=server_args,\n            model_cls=model_cls,\n            component_name=component_name,\n        )\n\n    if checkpoint_quant_config is not None:\n        if server_args.quantization is not None:\n            raise ValueError(\n                \"Checkpoint quantization is encoded in per-layer metadata; do not \"\n                \"also set --quantization\"\n            )\n        if server_args.nunchaku_config is not None:\n            raise ValueError(\n                \"Per-layer checkpoint quantization and Nunchaku are mutually \"\n                \"exclusive\"\n            )\n        quant_config = checkpoint_quant_config\n    elif getattr(model_cls, \"handles_checkpoint_quantization\", False):\n        quant_config = None\n    else:\n        quant_config = _resolve_quant_config(\n            hf_config=hf_config,\n            server_args=server_args,\n            safetensors_list=safetensors_list,\n            component_model_path=component_model_path,\n        )\n\n    if quant_config is not None:\n        packed = getattr(model_cls, \"packed_modules_mapping\", None)\n        if packed and hasattr(quant_config, \"packed_modules_mapping\"):\n            quant_config.packed_modules_mapping = packed","sourceCodeStart":775,"sourceCodeEnd":811,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/loader/transformer_load_utils.py#L775-L811","documentation":"Per-layer checkpoint quantization and the Nunchaku backend are implemented as mutually exclusive weight-loading paths; combining them has no defined semantics. The resolver detects both and aborts before any weights are loaded.","triggerScenarios":"resolve_transformer_quant_load_spec / load_customized is invoked with a checkpoint that has quantization metadata (checkpoint_quant_config is not None) while server_args.nunchaku_config is set.","commonSituations":"User switches to a per-layer quantized checkpoint but leaves --nunchaku-config in their launch config; or copies a Nunchaku-serving command line onto a new quantized model.","solutions":["Remove the --nunchaku-config flag so the checkpoint's per-layer quantization is used","Alternatively, use a Nunchaku-compatible (unquantized or Nunchaku-format) checkpoint and drop the per-layer quantized one","Programmatically, assert server_args.nunchaku_config is None before loading a checkpoint with quant metadata"],"exampleFix":"# before\n--nunchaku-config ... --model per-layer-quantized-model\n# after\n--model per-layer-quantized-model","handlingStrategy":"validation","validationCode":"def nunchaku_compatible(server_args, checkpoint_quant_config) -> bool:\n    return checkpoint_quant_config is None or server_args.nunchaku_config is None","typeGuard":null,"tryCatchPattern":"try:\n    resolve_transformer_quant_load_spec(...)\nexcept ValueError as e:\n    if \"Nunchaku are mutually exclusive\" in str(e):\n        server_args.nunchaku_config = None\n        resolve_transformer_quant_load_spec(...)","preventionTips":["Pick one quantization path per deployment: Nunchaku backend OR per-layer quantized checkpoint","Fail fast at config-parse time by rejecting the combination in your own arg validation"],"tags":["nunchaku","quantization","mutually-exclusive","config-conflict"],"backgroundTag":"conflicting-configuration-options","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}