sgl-project/sglang · error · ValueError

--enable-svdquant cannot be combined with a GGUF transformer

Error message

--enable-svdquant cannot be combined with a GGUF transformer: both supply the transformer weights. Point --transformer-weights-path at either an SVDQuant checkpoint or a .gguf, not one while requesting the other.

What it means

Both SVDQuant (Nunchaku) and GGUF supply transformer weights through the shared --transformer-weights-path flag, so requesting both is ambiguous and rejected rather than silently dropping one plan.

Source

Thrown at python/sglang/multimodal_gen/runtime/loader/transformer_load_utils.py:539

    # either redundant (gguf) or a conflicting request that would otherwise be
    # dropped without a word.
    if server_args.quantization == "gguf":
        raise ValueError(
            "GGUF is selected by passing the checkpoint itself, not "
            "`--quantization gguf`. Drop the flag; "
            "`--transformer-weights-path <file.gguf>` is what enables it."
        )
    if server_args.quantization is not None:
        raise ValueError(
            f"--quantization {server_args.quantization} cannot be combined with "
            "a GGUF transformer, whose quantization is fixed by the checkpoint. "
            "Drop the flag, or use an unquantized checkpoint to quantize online."
        )
    # Nunchaku shares --transformer-weights-path with GGUF, and the GGUF plan is
    # resolved first, so without this the SVDQuant request would be dropped in
    # silence rather than refused.
    if server_args.nunchaku_config is not None:
        raise ValueError(
            "--enable-svdquant cannot be combined with a GGUF transformer: both "
            "supply the transformer weights. Point "
            "--transformer-weights-path at either an SVDQuant checkpoint or a "
            ".gguf, not one while requesting the other."
        )
    if not current_platform.is_cuda():
        raise ValueError(
            "GGUF diffusion checkpoints require CUDA; the GGML kernels have no "
            f"{current_platform.device_type} implementation."
        )
    uses_fsdp = (
        server_args.should_use_fsdp_for_component(component_name)
        if component_name is not None
        else server_args.use_fsdp_inference
    )
    if uses_fsdp:
        raise ValueError(
            "GGUF diffusion checkpoints are incompatible with FSDP inference. "

View on GitHub (pinned to 0132848349)

Solutions

Point --transformer-weights-path at the SVDQuant checkpoint and drop GGUF, or
Point it at the .gguf file and remove --enable-svdquant / nunchaku_config

Example fix

# before
--transformer-weights-path model.gguf --enable-svdquant
# after
--transformer-weights-path svdquant_checkpoint

Defensive patterns

Strategy: validation

Validate before calling

if server_args.nunchaku_config is not None and str(server_args.transformer_weights_path or '').endswith('.gguf'):
    raise SystemExit('SVDQuant and GGUF both use --transformer-weights-path; pick one')

Prevention

Remember --transformer-weights-path is shared between SVDQuant and GGUF; never enable both

When it happens

Trigger: Setting server_args.nunchaku_config (via --enable-svdquant) while --transformer-weights-path points at a .gguf file; the GGUF plan resolves first, so the guard raises to avoid dropping the SVDQuant request.

Common situations: Switching a server between an SVDQuant checkpoint and a GGUF checkpoint and forgetting to remove the other enabling flag/config.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/11193807ed51a401. Report an issue: GitHub.