hiyouga/LlamaFactory · error · ValueError

DoRA is not compatible with PTQ-quantized models.

Error message

DoRA is not compatible with PTQ-quantized models.

What it means

Raised in _setup_lora_tuning when use_dora is true and the model was loaded with a quantization_method other than BNB (i.e. GPTQ/AWQ PTQ checkpoints). DoRA's weight-decomposed deltas need to modify base weights, which only works on bnb-quantized (QLoRA-style) models in this stack.

Source

Thrown at src/llamafactory/model/adapter.py:244

        logger.info_rank0("Loaded adapter(s): {}".format(",".join(model_args.adapter_name_or_path)))

    if is_trainable and adapter_to_resume is None:  # create new lora weights while training
        if len(finetuning_args.lora_target) == 1 and finetuning_args.lora_target[0] == "all":
            target_modules = find_all_linear_modules(model, finetuning_args.freeze_vision_tower)
        else:
            target_modules = finetuning_args.lora_target

        if finetuning_args.use_llama_pro:
            target_modules = find_expanded_modules(model, target_modules, finetuning_args.freeze_trainable_layers)

        target_modules = patch_target_modules(model, finetuning_args, target_modules)

        if (
            finetuning_args.use_dora
            and getattr(model, "quantization_method", None) is not None
            and getattr(model, "quantization_method", None) != QuantizationMethod.BNB
        ):
            raise ValueError("DoRA is not compatible with PTQ-quantized models.")

        if model_args.resize_vocab and finetuning_args.additional_target is None:
            input_embeddings = model.get_input_embeddings()
            output_embeddings = model.get_output_embeddings()
            module_names = set()
            for name, module in model.named_modules():
                if module in [input_embeddings, output_embeddings]:
                    module_names.add(name.split(".")[-1])

            finetuning_args.additional_target = module_names
            logger.warning_rank0("Vocab has been resized, add {} to trainable params.".format(",".join(module_names)))

        if finetuning_args.finetuning_type == "lora":
            peft_kwargs = {
                "r": finetuning_args.lora_rank,
                "target_modules": target_modules,
                "lora_alpha": finetuning_args.lora_alpha,
                "lora_dropout": finetuning_args.lora_dropout,

View on GitHub (pinned to f28afaf635)

Solutions

1. Set use_dora: false for GPTQ/AWQ models (plain LoRA works).
2. Use an unquantized or bnb-quantized (quantization_bit: 4/8) model if you want DoRA — the check explicitly allows QuantizationMethod.BNB.
3. Switch to a GPTQ/AWQ path without DoRA, or dequantize/export the model first.

Example fix

# before
model_name_or_path: Qwen/Qwen2-7B-GPTQ-Int4
use_dora: true

# after
model_name_or_path: Qwen/Qwen2-7B
quantization_bit: 4
use_dora: true

qm = getattr(model, "quantization_method", None)  # after loading
if use_dora and qm is not None:
    from peft.utils import QuantizationMethod
    assert qm == QuantizationMethod.BNB, \
        "DoRA needs unquantized or bnb-quantized weights; GPTQ/AWQ checkpoints are unsupported"
# before loading, a cheap proxy: assert model path is not a -GPTQ/-AWQ repo when use_dora

When it happens

Trigger: Loading a GPTQ or AWQ model (model.hf_quantizer sets quantization_method) and setting use_dora: true with finetuning_type: lora.

Common situations: Swapping a QLoRA+DoRA config to a GPTQ checkpoint to save memory; enabling DoRA on an -AWQ/-GPTQ repo without realizing the quantizer type matters.

Related errors

• vLLM engine does not support bnb quantization (GPTQ and AWQ
• `use_dora` is only valid for LoRA training.
• Cannot find satisfying example, considering decrease `export
• DeepSpeed ZeRO-3 or FSDP is incompatible with PTQ-quantized
• AutoGPTQ only accepts 2/3/4/8-bit quantization.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/7d8f212d1b7597f8. Report an issue: GitHub.