sgl-project/sglang · error · ValueError

A GGUF encoder checkpoint cannot be combined with a second q

Error message

A GGUF encoder checkpoint cannot be combined with a second quantization declaration

What it means

Raised when a GGUF-format encoder checkpoint is loaded together with an explicit quantization declaration (e.g. a quant_config in the component config or an online quantization override). GGUF checkpoints carry their own quantization metadata (read via read_gguf_tensor_meta), so the loader refuses a second, conflicting quantization source. It is a plain ValueError from _get_encoder_quant_config.

Source

Thrown at python/sglang/multimodal_gen/runtime/loader/component_loaders/text_encoder_loader.py:211

            def parameter_name_mapper(name: str) -> str:
                mapped_name, merge_index, _ = mapping_fn(name)
                if merge_index is not None:
                    raise ValueError(
                        "Serialized quantized component weights cannot use a "
                        "stacked parameter-name mapping"
                    )
                return mapped_name

            def name_mapper(name: str) -> str:
                # Layer-prefix metadata omits the suffix that many model
                # mappings use to delimit a parameter name.
                mapped_name = parameter_name_mapper(f"{name}.weight")
                return mapped_name.removesuffix(".weight")

    if names_gguf_checkpoint(component_weights_path):
        if quant_config is not None:
            raise ValueError(
                "A GGUF encoder checkpoint cannot be combined with a second "
                "quantization declaration"
            )
        tensor_meta = read_gguf_tensor_meta(component_weights_path)
        dequantize_prefixes = (
            vars(model_cls).get("gguf_dequantize_prefixes", ())
            if model_cls is not None
            else ()
        )
        tensor_meta = remap_gguf_tensor_meta(
            tensor_meta,
            parameter_name_mapper or (lambda name: name),
            dequantize_prefixes=dequantize_prefixes,
        )
        return GGUFConfig(component_weights_path, tensor_meta)

    if (
        quant_config is None

View on GitHub (pinned to 0132848349)

Solutions

1. Remove the quantization declaration (quantization_config in config.json or the explicit quantization argument) so the GGUF file is the sole quantization source
2. Point component_weights_path at the original safetensors checkpoint if you need the explicit quantization config instead
3. If you want dequantization, rely on the model's gguf_dequantize_prefixes handling rather than a second quant config

Example fix

# before
loader.load(component_weights_path="encoder.Q8_0.gguf", quant_config=QuantConfig(fp8))

# after
loader.load(component_weights_path="encoder.Q8_0.gguf", quant_config=None)

from pathlib import Path
from sglang.multimodal_gen.runtime.loader.component_loaders.text_encoder_loader import names_gguf_checkpoint

if names_gguf_checkpoint(Path(weights_path)) and quant_config is not None:
    raise SystemExit("GGUF checkpoint already carries quantization; unset quant_config")

When it happens

Trigger: Calling the text encoder loader with component_weights_path pointing at a .gguf file while quant_config is not None — e.g. a config.json with quantization_config set, or passing an explicit online quantization string for a GGUF-quantized encoder (Llama-3 GGUF text encoders in multimodal pipelines).

Common situations: Downloading a GGUF-quantized variant of a text encoder and leaving a stale quantization_config in the model dir; copy-pasting a loader config that worked for safetensors FP8 checkpoints onto a GGUF checkpoint; combining --quantization flags with GGUF weights.

Related errors

• Quanto tensor/map prefixes do not match: missing metadata={s
• Cannot parse checkpoint quantization for {component_name!r}:
• A quantized {component_name!r} checkpoint requires an in-tre
• SGLang diffusion currently supports AutoRound auto_gptq chec
• AutoRound fused module {target!r} has inconsistent shard con

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/319cf1b9431527ea. Report an issue: GitHub.