{"record":{"id":"111c7190c71d5c74","repo":"sgl-project/sglang","slug":"gguf-and-safetensors-quantization-metadata-conflic","errorCode":null,"errorMessage":"GGUF and safetensors quantization metadata conflict","messagePattern":"GGUF and safetensors quantization metadata conflict","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/loader/transformer_load_utils.py","lineNumber":778,"sourceCode":"\n    return filtered\n\n\ndef resolve_transformer_quant_load_spec(\n    *,\n    hf_config: dict,\n    server_args: ServerArgs,\n    safetensors_list: list[str],\n    component_model_path: str,\n    model_cls: type[nn.Module],\n    cls_name: str,\n    component_name: str | None = None,\n    gguf_file: str | None = None,\n    checkpoint_quant_config: QuantizationConfig | None = None,\n) -> TransformerQuantLoadSpec:\n    if gguf_file is not None:\n        if checkpoint_quant_config is not None:\n            raise ValueError(\"GGUF and safetensors quantization metadata conflict\")\n        return _resolve_gguf_quant_load_spec(\n            gguf_file=gguf_file,\n            server_args=server_args,\n            model_cls=model_cls,\n            component_name=component_name,\n        )\n\n    if checkpoint_quant_config is not None:\n        if server_args.quantization is not None:\n            raise ValueError(\n                \"Checkpoint quantization is encoded in per-layer metadata; do not \"\n                \"also set --quantization\"\n            )\n        if server_args.nunchaku_config is not None:\n            raise ValueError(\n                \"Per-layer checkpoint quantization and Nunchaku are mutually \"\n                \"exclusive\"\n            )","sourceCodeStart":760,"sourceCodeEnd":796,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/loader/transformer_load_utils.py#L760-L796","documentation":"resolve_transformer_quant_load_spec received both a gguf_file and a checkpoint_quant_config (safetensors-derived quantization metadata); the two sources of quantization truth are mutually exclusive and the conflict is rejected.","triggerScenarios":"Calling resolve_transformer_quant_load_spec with both gguf_file and checkpoint_quant_config non-None, e.g. a component whose config carries quantization_config while a GGUF override is also supplied.","commonSituations":"A mixed pipeline where one component (e.g. text encoder) has quantization metadata in its config and the transformer override resolves to GGUF, producing both metadata sources in one call.","solutions":["Ensure only one quantization source is active: drop the GGUF override or use a checkpoint without quantization_config metadata","Inspect how gguf_file and checkpoint_quant_config are populated for the component to find the misrouted argument"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"if gguf_file is not None and checkpoint_quant_config is not None:\n    raise SystemExit('conflicting quantization metadata: GGUF file plus checkpoint quant config')","typeGuard":null,"tryCatchPattern":"try:\n    spec = resolve_transformer_quant_load_spec(...)\nexcept ValueError as e:\n    if 'conflict' in str(e):\n        # drop one quantization source and retry\n        ...\n    raise","preventionTips":["Ensure a component gets its quantization plan from exactly one source: GGUF override or checkpoint config, never both"],"tags":["gguf","quantization","metadata-conflict","load-spec"],"backgroundTag":"conflicting-quantization-config","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}