hiyouga/LlamaFactory · error · ValueError

Cannot use PiSSA for current training stage.

Error message

Cannot use PiSSA for current training stage.

What it means

PiSSA initializes LoRA from the principal singular components of the base weights; the decomposition assumes the adapter base is frozen and the run has no separate reference model. PPO/KTO stages (and any stage with use_ref_model, i.e. DPO with sigmoid/ipo losses) use reference-model scoring, which conflicts. FinetuningArguments.__post_init__ (src/llamafactory/hparams/finetuning_args.py:622) rejects pissa_init in those cases.

Source

Thrown at src/llamafactory/hparams/finetuning_args.py:622

            raise ValueError("`reward_model_type` cannot be lora for Freeze/Full PPO training.")

        if self.stage == "ppo" and self.reward_model_type == "oft" and self.finetuning_type != "oft":
            raise ValueError("`reward_model_type` cannot be oft for Freeze/Full PPO training.")

        if self.stage == "dpo" and self.pref_loss != "sigmoid" and self.dpo_label_smoothing > 1e-6:
            raise ValueError("`dpo_label_smoothing` is only valid for sigmoid loss function.")

        if self.use_llama_pro and self.finetuning_type == "full":
            raise ValueError("`use_llama_pro` is only valid for Freeze or LoRA training.")

        if self.finetuning_type == "lora" and (self.use_galore or self.use_apollo or self.use_badam):
            raise ValueError("Cannot use LoRA with GaLore, APOLLO or BAdam together.")

        if int(self.use_galore) + int(self.use_apollo) + (self.use_badam) > 1:
            raise ValueError("Cannot use GaLore, APOLLO or BAdam together.")

        if self.pissa_init and (self.stage in ["ppo", "kto"] or self.use_ref_model):
            raise ValueError("Cannot use PiSSA for current training stage.")

        if self.finetuning_type != "lora":
            if self.loraplus_lr_ratio is not None:
                raise ValueError("`loraplus_lr_ratio` is only valid for LoRA training.")

            if self.use_rslora:
                raise ValueError("`use_rslora` is only valid for LoRA training.")

            if self.use_dora:
                raise ValueError("`use_dora` is only valid for LoRA training.")

            if self.pissa_init:
                raise ValueError("`pissa_init` is only valid for LoRA training.")

    def to_dict(self) -> dict[str, Any]:
        args = asdict(self)
        args = {k: f"<{k.upper()}>" if k.endswith("api_key") else v for k, v in args.items()}
        return args

View on GitHub (pinned to f28afaf635)

Solutions

1. Remove pissa_init: true if you are doing PPO/KTO/reference-based DPO.
2. Use pissa_init only for SFT/pretrain/RM/ppo-free stages with finetuning_type: lora.
3. For DPO, switch to pref_loss: orpo or simpo (no reference model) if you must keep PiSSA.

Example fix

# before (yaml)
stage: kto
finetuning_type: lora
pissa_init: true

# after (yaml)
stage: kto
finetuning_type: lora
# pissa_init removed

def check_pissa(stage: str, pref_loss: str, pissa_init: bool, use_ref_model: bool) -> None:
    if pissa_init and (stage in ("ppo", "kto") or use_ref_model):
        raise ValueError("pissa_init is incompatible with reference-model stages (ppo/kto/dpo-sigmoid)")

When it happens

Trigger: pissa_init: true together with stage: ppo, stage: kto, or stage: dpo with a pref_loss that needs a reference model (pref_loss not in ['orpo','simpo']).

Common situations: Enabling PiSSA for faster LoRA convergence and then switching the config to KTO/DPO preference training, forgetting the init flag. Note stage: dpo with pref_loss orpo/simpo is allowed because use_ref_model is false there.

Related errors

• `reward_model_type` cannot be lora for Freeze/Full PPO train
• `pissa_init` is only valid for LoRA training.
• KTransformers does not support lora reward model.
• Unsloth does not support lora reward model.
• bf16 and fp16 cannot be both True.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/16acc5862af4bf59. Report an issue: GitHub.