{"record":{"id":"319cf1b9431527ea","repo":"sgl-project/sglang","slug":"a-gguf-encoder-checkpoint-cannot-be-combined-with","errorCode":null,"errorMessage":"A GGUF encoder checkpoint cannot be combined with a second quantization declaration","messagePattern":"A GGUF encoder checkpoint cannot be combined with a second quantization declaration","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/loader/component_loaders/text_encoder_loader.py","lineNumber":211,"sourceCode":"\n            def parameter_name_mapper(name: str) -> str:\n                mapped_name, merge_index, _ = mapping_fn(name)\n                if merge_index is not None:\n                    raise ValueError(\n                        \"Serialized quantized component weights cannot use a \"\n                        \"stacked parameter-name mapping\"\n                    )\n                return mapped_name\n\n            def name_mapper(name: str) -> str:\n                # Layer-prefix metadata omits the suffix that many model\n                # mappings use to delimit a parameter name.\n                mapped_name = parameter_name_mapper(f\"{name}.weight\")\n                return mapped_name.removesuffix(\".weight\")\n\n    if names_gguf_checkpoint(component_weights_path):\n        if quant_config is not None:\n            raise ValueError(\n                \"A GGUF encoder checkpoint cannot be combined with a second \"\n                \"quantization declaration\"\n            )\n        tensor_meta = read_gguf_tensor_meta(component_weights_path)\n        dequantize_prefixes = (\n            vars(model_cls).get(\"gguf_dequantize_prefixes\", ())\n            if model_cls is not None\n            else ()\n        )\n        tensor_meta = remap_gguf_tensor_meta(\n            tensor_meta,\n            parameter_name_mapper or (lambda name: name),\n            dequantize_prefixes=dequantize_prefixes,\n        )\n        return GGUFConfig(component_weights_path, tensor_meta)\n\n    if (\n        quant_config is None","sourceCodeStart":193,"sourceCodeEnd":229,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/loader/component_loaders/text_encoder_loader.py#L193-L229","documentation":"Raised when a GGUF-format encoder checkpoint is loaded together with an explicit quantization declaration (e.g. a quant_config in the component config or an online quantization override). GGUF checkpoints carry their own quantization metadata (read via read_gguf_tensor_meta), so the loader refuses a second, conflicting quantization source. It is a plain ValueError from _get_encoder_quant_config.","triggerScenarios":"Calling the text encoder loader with component_weights_path pointing at a .gguf file while quant_config is not None — e.g. a config.json with quantization_config set, or passing an explicit online quantization string for a GGUF-quantized encoder (Llama-3 GGUF text encoders in multimodal pipelines).","commonSituations":"Downloading a GGUF-quantized variant of a text encoder and leaving a stale quantization_config in the model dir; copy-pasting a loader config that worked for safetensors FP8 checkpoints onto a GGUF checkpoint; combining --quantization flags with GGUF weights.","solutions":["Remove the quantization declaration (quantization_config in config.json or the explicit quantization argument) so the GGUF file is the sole quantization source","Point component_weights_path at the original safetensors checkpoint if you need the explicit quantization config instead","If you want dequantization, rely on the model's gguf_dequantize_prefixes handling rather than a second quant config"],"exampleFix":"# before\nloader.load(component_weights_path=\"encoder.Q8_0.gguf\", quant_config=QuantConfig(fp8))\n\n# after\nloader.load(component_weights_path=\"encoder.Q8_0.gguf\", quant_config=None)","handlingStrategy":"validation","validationCode":"from pathlib import Path\nfrom sglang.multimodal_gen.runtime.loader.component_loaders.text_encoder_loader import names_gguf_checkpoint\n\nif names_gguf_checkpoint(Path(weights_path)) and quant_config is not None:\n    raise SystemExit(\"GGUF checkpoint already carries quantization; unset quant_config\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never set quantization_config for GGUF encoder checkpoints","Detect GGUF files by extension/metadata before configuring quantization"],"tags":["gguf","quantization","text-encoder","multimodal","checkpoint"],"backgroundTag":"conflicting-quantization-config","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}