{"record":{"id":"16acc5862af4bf59","repo":"hiyouga/LlamaFactory","slug":"cannot-use-pissa-for-current-training-stage","errorCode":null,"errorMessage":"Cannot use PiSSA for current training stage.","messagePattern":"Cannot use PiSSA for current training stage\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/finetuning_args.py","lineNumber":622,"sourceCode":"            raise ValueError(\"`reward_model_type` cannot be lora for Freeze/Full PPO training.\")\n\n        if self.stage == \"ppo\" and self.reward_model_type == \"oft\" and self.finetuning_type != \"oft\":\n            raise ValueError(\"`reward_model_type` cannot be oft for Freeze/Full PPO training.\")\n\n        if self.stage == \"dpo\" and self.pref_loss != \"sigmoid\" and self.dpo_label_smoothing > 1e-6:\n            raise ValueError(\"`dpo_label_smoothing` is only valid for sigmoid loss function.\")\n\n        if self.use_llama_pro and self.finetuning_type == \"full\":\n            raise ValueError(\"`use_llama_pro` is only valid for Freeze or LoRA training.\")\n\n        if self.finetuning_type == \"lora\" and (self.use_galore or self.use_apollo or self.use_badam):\n            raise ValueError(\"Cannot use LoRA with GaLore, APOLLO or BAdam together.\")\n\n        if int(self.use_galore) + int(self.use_apollo) + (self.use_badam) > 1:\n            raise ValueError(\"Cannot use GaLore, APOLLO or BAdam together.\")\n\n        if self.pissa_init and (self.stage in [\"ppo\", \"kto\"] or self.use_ref_model):\n            raise ValueError(\"Cannot use PiSSA for current training stage.\")\n\n        if self.finetuning_type != \"lora\":\n            if self.loraplus_lr_ratio is not None:\n                raise ValueError(\"`loraplus_lr_ratio` is only valid for LoRA training.\")\n\n            if self.use_rslora:\n                raise ValueError(\"`use_rslora` is only valid for LoRA training.\")\n\n            if self.use_dora:\n                raise ValueError(\"`use_dora` is only valid for LoRA training.\")\n\n            if self.pissa_init:\n                raise ValueError(\"`pissa_init` is only valid for LoRA training.\")\n\n    def to_dict(self) -> dict[str, Any]:\n        args = asdict(self)\n        args = {k: f\"<{k.upper()}>\" if k.endswith(\"api_key\") else v for k, v in args.items()}\n        return args","sourceCodeStart":604,"sourceCodeEnd":640,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/finetuning_args.py#L604-L640","documentation":"PiSSA initializes LoRA from the principal singular components of the base weights; the decomposition assumes the adapter base is frozen and the run has no separate reference model. PPO/KTO stages (and any stage with use_ref_model, i.e. DPO with sigmoid/ipo losses) use reference-model scoring, which conflicts. FinetuningArguments.__post_init__ (src/llamafactory/hparams/finetuning_args.py:622) rejects pissa_init in those cases.","triggerScenarios":"pissa_init: true together with stage: ppo, stage: kto, or stage: dpo with a pref_loss that needs a reference model (pref_loss not in ['orpo','simpo']).","commonSituations":"Enabling PiSSA for faster LoRA convergence and then switching the config to KTO/DPO preference training, forgetting the init flag. Note stage: dpo with pref_loss orpo/simpo is allowed because use_ref_model is false there.","solutions":["Remove pissa_init: true if you are doing PPO/KTO/reference-based DPO.","Use pissa_init only for SFT/pretrain/RM/ppo-free stages with finetuning_type: lora.","For DPO, switch to pref_loss: orpo or simpo (no reference model) if you must keep PiSSA."],"exampleFix":"# before (yaml)\nstage: kto\nfinetuning_type: lora\npissa_init: true\n\n# after (yaml)\nstage: kto\nfinetuning_type: lora\n# pissa_init removed","handlingStrategy":"validation","validationCode":"def check_pissa(stage: str, pref_loss: str, pissa_init: bool, use_ref_model: bool) -> None:\n    if pissa_init and (stage in (\"ppo\", \"kto\") or use_ref_model):\n        raise ValueError(\"pissa_init is incompatible with reference-model stages (ppo/kto/dpo-sigmoid)\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Treat pissa_init as SFT/LoRA-only until verified against the current stage.","When moving a LoRA recipe into preference training, strip all adapter-initialization flags."],"tags":["config","pissa","lora","ppo","kto","dpo"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}