hiyouga/LlamaFactory · error · ValueError

Quantized models can only be used for the LoRA or OFT tuning

Error message

Quantized models can only be used for the LoRA or OFT tuning.

What it means

Raised in setup_adapter when the model was loaded quantized (quantization_method set) and training is requested (is_trainable) with a finetuning_type other than lora/oft. Quantized weights cannot be updated in place, so only adapter-based methods (QLoRA/OFT) are trainable; this is the QLoRA contract enforced up front.

Source

Thrown at src/llamafactory/model/adapter.py:332

    return model


def init_adapter(
    config: "PretrainedConfig",
    model: "PreTrainedModel",
    model_args: "ModelArguments",
    finetuning_args: "FinetuningArguments",
    is_trainable: bool,
) -> "PreTrainedModel":
    r"""Initialize the adapters.

    Support full-parameter, freeze and LoRA training.

    Note that the trainable parameters must be cast to float32.
    """
    if is_trainable and getattr(model, "quantization_method", None) is not None:
        if finetuning_args.finetuning_type not in ["lora", "oft"]:
            raise ValueError("Quantized models can only be used for the LoRA or OFT tuning.")

        if finetuning_args.pissa_init:
            raise ValueError("Cannot initialize PiSSA adapter on quantized models.")

    # cast trainable parameters to float32 if:
    # 1. is_trainable and not pure_bf16 and not badam and quantization_bit is not None (qlora)
    # 2. is_trainable and not pure_bf16 and not badam and not zero3 (zero3 already in fp32)
    cast_trainable_params_to_fp32 = False
    if not is_trainable:
        pass
    elif finetuning_args.pure_bf16 or finetuning_args.use_badam:
        logger.info_rank0("Pure bf16 / BAdam detected, remaining trainable params in half precision.")
    elif model_args.quantization_bit is None and is_deepspeed_zero3_enabled():
        logger.info_rank0("DeepSpeed ZeRO3 detected, remaining trainable params in float32.")
    else:
        logger.info_rank0("Upcasting trainable params to float32.")
        cast_trainable_params_to_fp32 = True

View on GitHub (pinned to f28afaf635)

Solutions

1. Set finetuning_type: lora (the standard QLoRA setup) or oft on quantized models.
2. If you need full fine-tuning, remove quantization_bit and load the unquantized checkpoint (possibly with DeepSpeed/FSDP offload for memory).

Example fix

# before
quantization_bit: 4
finetuning_type: full

# after
quantization_bit: 4
finetuning_type: lora

is_quantized = cfg["model_args"].get("quantization_bit") is not None or bool(
    {"GPTQ", "AWQ"} & {t for t in cfg["model_args"].get("model_name_or_path", "").upper().split("-")}
)
if is_quantized and do_train:
    assert cfg["finetuning_args"]["finetuning_type"] in ("lora", "oft"), \
        "quantized models are trainable via lora/oft only (QLoRA)"

When it happens

Trigger: quantization_bit set (bnb) or a GPTQ/AWQ model, with do_train and finetuning_type: full or freeze.

Common situations: Leaving finetuning_type: full (the default in some examples) while adding quantization_bit: 4 to fit a big model; freeze-tuning a GPTQ checkpoint.

Related errors

• Quantization is only compatible with the LoRA or OFT method.
• Cannot use device map for quantized models in training.
• Cannot initialize PiSSA adapter on quantized models.
• Only 4-bit quantized model can use fsdp+qlora or auto device
• `use_llama_pro` is only valid for Freeze or LoRA training.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/14cd767477bd0288. Report an issue: GitHub.