hiyouga/LlamaFactory · error · ValueError

`use_dora` is only valid for LoRA training.

Error message

`use_dora` is only valid for LoRA training.

What it means

Weight-Decomposed Low-Rank Adaptation (DoRA) is a LoRA variant that decomposes weights into magnitude and direction; it only applies to LoRA adapters. FinetuningArguments.__post_init__ (src/llamafactory/hparams/finetuning_args.py:632) rejects use_dora: true when finetuning_type != lora.

Source

Thrown at src/llamafactory/hparams/finetuning_args.py:632

        if self.finetuning_type == "lora" and (self.use_galore or self.use_apollo or self.use_badam):
            raise ValueError("Cannot use LoRA with GaLore, APOLLO or BAdam together.")

        if int(self.use_galore) + int(self.use_apollo) + (self.use_badam) > 1:
            raise ValueError("Cannot use GaLore, APOLLO or BAdam together.")

        if self.pissa_init and (self.stage in ["ppo", "kto"] or self.use_ref_model):
            raise ValueError("Cannot use PiSSA for current training stage.")

        if self.finetuning_type != "lora":
            if self.loraplus_lr_ratio is not None:
                raise ValueError("`loraplus_lr_ratio` is only valid for LoRA training.")

            if self.use_rslora:
                raise ValueError("`use_rslora` is only valid for LoRA training.")

            if self.use_dora:
                raise ValueError("`use_dora` is only valid for LoRA training.")

            if self.pissa_init:
                raise ValueError("`pissa_init` is only valid for LoRA training.")

    def to_dict(self) -> dict[str, Any]:
        args = asdict(self)
        args = {k: f"<{k.upper()}>" if k.endswith("api_key") else v for k, v in args.items()}
        return args

View on GitHub (pinned to f28afaf635)

Solutions

1. Remove use_dora: true if fine-tuning fully or freezing.
2. Set finetuning_type: lora to use DoRA adapters.
3. Verify your PEFT/transformers version supports DoRA for the target model if you keep LoRA+DoRA.

Example fix

# before (yaml)
finetuning_type: freeze
use_dora: true

# after (yaml)
finetuning_type: lora
use_dora: true

def check_dora(finetuning_type: str, use_dora: bool) -> None:
    if use_dora and finetuning_type != "lora":
        raise ValueError("use_dora requires finetuning_type=lora")

When it happens

Trigger: use_dora: true together with finetuning_type: full or freeze.

Common situations: Switching a DoRA recipe to full fine-tuning without cleaning flags; or assuming DoRA is a general optimizer that can stack on freeze training.

Related errors

• `reward_model_type` cannot be lora for Freeze/Full PPO train
• Cannot use LoRA with GaLore, APOLLO or BAdam together.
• Cannot use PiSSA for current training stage.
• `loraplus_lr_ratio` is only valid for LoRA training.
• `use_rslora` is only valid for LoRA training.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/7d0c6067027f93a8. Report an issue: GitHub.