sgl-project/sglang · error · ValueError

LoRA is not supported on a GGUF transformer: an adapter cann

Error message

LoRA is not supported on a GGUF transformer: an adapter cannot be merged into packed GGML blocks. Use the unquantized checkpoint to serve LoRA.

What it means

LoRA adapters require merging into ordinary weight tensors; GGUF stores packed GGML quantized blocks, so LoRA cannot be applied and the load is rejected.

Source

Thrown at python/sglang/multimodal_gen/runtime/loader/transformer_load_utils.py:562

        )
    if not current_platform.is_cuda():
        raise ValueError(
            "GGUF diffusion checkpoints require CUDA; the GGML kernels have no "
            f"{current_platform.device_type} implementation."
        )
    uses_fsdp = (
        server_args.should_use_fsdp_for_component(component_name)
        if component_name is not None
        else server_args.use_fsdp_inference
    )
    if uses_fsdp:
        raise ValueError(
            "GGUF diffusion checkpoints are incompatible with FSDP inference. "
            "Run without --use-fsdp-inference, or keep this component offloaded "
            "so FSDP does not manage it."
        )
    if server_args.lora_path is not None:
        raise ValueError(
            "LoRA is not supported on a GGUF transformer: an adapter cannot be "
            "merged into packed GGML blocks. Use the unquantized checkpoint to "
            "serve LoRA."
        )
    # H3's AdaLN paths read the transformer's safetensors directly -- the cache
    # builder needs unquantized weights, and the online rebuild is handed the
    # safetensors file list, which is empty for a GGUF load.
    if server_args.minimax_h3_adaln_online:
        raise ValueError(
            "--minimax-h3-adaln-online rebuilds AdaLN outputs from the "
            "safetensors checkpoint and cannot read a GGUF transformer."
        )
    if server_args.minimax_h3_adaln_cache_path is not None:
        raise ValueError(
            "--minimax-h3-adaln-cache-path requires the unquantized "
            "transformer and cannot be combined with a GGUF checkpoint."
        )

View on GitHub (pinned to 0132848349)

Solutions

Drop --lora-path for GGUF serving
Serve LoRA from the unquantized safetensors checkpoint instead

Example fix

# before
--transformer-weights-path model.gguf --lora-path ./adapter
# after
--model-path unquantized_model --lora-path ./adapter

Defensive patterns

Strategy: validation

Validate before calling

if server_args.lora_path is not None and str(server_args.transformer_weights_path or '').endswith('.gguf'):
    raise SystemExit('LoRA requires the unquantized checkpoint')

Prevention

Keep a separate config profile for LoRA serving that uses safetensors base models

When it happens

Trigger: server_args.lora_path is not None while a GGUF transformer is selected via --transformer-weights-path.

Common situations: Reusing a serving config that enables --lora-path and swapping the base model to a GGUF quantized export.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a447027337c66c2f. Report an issue: GitHub.