hiyouga/LlamaFactory · error · ValueError

`pissa_init` is only valid for LoRA training.

Error message

`pissa_init` is only valid for LoRA training.

What it means

PiSSA initialization decomposes base weights into principal + residual parts to initialize LoRA A/B; it is meaningless without LoRA adapters. FinetuningArguments.__post_init__ (src/llamafactory/hparams/finetuning_args.py:635) rejects pissa_init: true when finetuning_type != lora.

Source

Thrown at src/llamafactory/hparams/finetuning_args.py:635

        if int(self.use_galore) + int(self.use_apollo) + (self.use_badam) > 1:
            raise ValueError("Cannot use GaLore, APOLLO or BAdam together.")

        if self.pissa_init and (self.stage in ["ppo", "kto"] or self.use_ref_model):
            raise ValueError("Cannot use PiSSA for current training stage.")

        if self.finetuning_type != "lora":
            if self.loraplus_lr_ratio is not None:
                raise ValueError("`loraplus_lr_ratio` is only valid for LoRA training.")

            if self.use_rslora:
                raise ValueError("`use_rslora` is only valid for LoRA training.")

            if self.use_dora:
                raise ValueError("`use_dora` is only valid for LoRA training.")

            if self.pissa_init:
                raise ValueError("`pissa_init` is only valid for LoRA training.")

    def to_dict(self) -> dict[str, Any]:
        args = asdict(self)
        args = {k: f"<{k.upper()}>" if k.endswith("api_key") else v for k, v in args.items()}
        return args

View on GitHub (pinned to f28afaf635)

Solutions

1. Remove pissa_init: true for non-LoRA fine-tuning.
2. Set finetuning_type: lora to use PiSSA initialization.
3. Remember pissa_init also requires a compatible stage (it is rejected for ppo/kto/reference-based dpo, see the earlier check).

Example fix

# before (yaml)
finetuning_type: full
pissa_init: true

# after (yaml)
finetuning_type: full
# pissa_init removed

def check_pissa_lora(finetuning_type: str, pissa_init: bool) -> None:
    if pissa_init and finetuning_type != "lora":
        raise ValueError("pissa_init requires finetuning_type=lora")

When it happens

Trigger: pissa_init: true with finetuning_type: full, freeze, or oft.

Common situations: Leftover flag when converting a PiSSA-LoRA experiment to full training; also users who conflate PiSSA with a general training-method switch.

Related errors

• Cannot use PiSSA for current training stage.
• `reward_model_type` cannot be lora for Freeze/Full PPO train
• Cannot use LoRA with GaLore, APOLLO or BAdam together.
• `loraplus_lr_ratio` is only valid for LoRA training.
• `use_rslora` is only valid for LoRA training.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/df88c765eba9e7e3. Report an issue: GitHub.