sgl-project/sglang · error · ValueError

GGUF and safetensors quantization metadata conflict

Error message

GGUF and safetensors quantization metadata conflict

What it means

resolve_transformer_quant_load_spec received both a gguf_file and a checkpoint_quant_config (safetensors-derived quantization metadata); the two sources of quantization truth are mutually exclusive and the conflict is rejected.

Source

Thrown at python/sglang/multimodal_gen/runtime/loader/transformer_load_utils.py:778

    return filtered


def resolve_transformer_quant_load_spec(
    *,
    hf_config: dict,
    server_args: ServerArgs,
    safetensors_list: list[str],
    component_model_path: str,
    model_cls: type[nn.Module],
    cls_name: str,
    component_name: str | None = None,
    gguf_file: str | None = None,
    checkpoint_quant_config: QuantizationConfig | None = None,
) -> TransformerQuantLoadSpec:
    if gguf_file is not None:
        if checkpoint_quant_config is not None:
            raise ValueError("GGUF and safetensors quantization metadata conflict")
        return _resolve_gguf_quant_load_spec(
            gguf_file=gguf_file,
            server_args=server_args,
            model_cls=model_cls,
            component_name=component_name,
        )

    if checkpoint_quant_config is not None:
        if server_args.quantization is not None:
            raise ValueError(
                "Checkpoint quantization is encoded in per-layer metadata; do not "
                "also set --quantization"
            )
        if server_args.nunchaku_config is not None:
            raise ValueError(
                "Per-layer checkpoint quantization and Nunchaku are mutually "
                "exclusive"
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure only one quantization source is active: drop the GGUF override or use a checkpoint without quantization_config metadata
2. Inspect how gguf_file and checkpoint_quant_config are populated for the component to find the misrouted argument

if gguf_file is not None and checkpoint_quant_config is not None:
    raise SystemExit('conflicting quantization metadata: GGUF file plus checkpoint quant config')

try:
    spec = resolve_transformer_quant_load_spec(...)
except ValueError as e:
    if 'conflict' in str(e):
        # drop one quantization source and retry
        ...
    raise

When it happens

Trigger: Calling resolve_transformer_quant_load_spec with both gguf_file and checkpoint_quant_config non-None, e.g. a component whose config carries quantization_config while a GGUF override is also supplied.

Common situations: A mixed pipeline where one component (e.g. text encoder) has quantization metadata in its config and the transformer override resolves to GGUF, producing both metadata sources in one call.

Related errors

• GGUFConfig must be constructed from a GGUF checkpoint
• A GGUF encoder checkpoint cannot be combined with a second q
• Cannot parse checkpoint quantization for {component_name!r}:
• GGUF tensor {tensor.name} is quantized, but diffusion GGUF c
• GGUF tensor {tensor.name} has inner dimension {inner_dim}, w

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/111c7190c71d5c74. Report an issue: GitHub.