{"record":{"id":"df88c765eba9e7e3","repo":"hiyouga/LlamaFactory","slug":"pissa-init-is-only-valid-for-lora-training","errorCode":null,"errorMessage":"`pissa_init` is only valid for LoRA training.","messagePattern":"`pissa_init` is only valid for LoRA training\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/finetuning_args.py","lineNumber":635,"sourceCode":"\n        if int(self.use_galore) + int(self.use_apollo) + (self.use_badam) > 1:\n            raise ValueError(\"Cannot use GaLore, APOLLO or BAdam together.\")\n\n        if self.pissa_init and (self.stage in [\"ppo\", \"kto\"] or self.use_ref_model):\n            raise ValueError(\"Cannot use PiSSA for current training stage.\")\n\n        if self.finetuning_type != \"lora\":\n            if self.loraplus_lr_ratio is not None:\n                raise ValueError(\"`loraplus_lr_ratio` is only valid for LoRA training.\")\n\n            if self.use_rslora:\n                raise ValueError(\"`use_rslora` is only valid for LoRA training.\")\n\n            if self.use_dora:\n                raise ValueError(\"`use_dora` is only valid for LoRA training.\")\n\n            if self.pissa_init:\n                raise ValueError(\"`pissa_init` is only valid for LoRA training.\")\n\n    def to_dict(self) -> dict[str, Any]:\n        args = asdict(self)\n        args = {k: f\"<{k.upper()}>\" if k.endswith(\"api_key\") else v for k, v in args.items()}\n        return args\n","sourceCodeStart":617,"sourceCodeEnd":641,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/finetuning_args.py#L617-L641","documentation":"PiSSA initialization decomposes base weights into principal + residual parts to initialize LoRA A/B; it is meaningless without LoRA adapters. FinetuningArguments.__post_init__ (src/llamafactory/hparams/finetuning_args.py:635) rejects pissa_init: true when finetuning_type != lora.","triggerScenarios":"pissa_init: true with finetuning_type: full, freeze, or oft.","commonSituations":"Leftover flag when converting a PiSSA-LoRA experiment to full training; also users who conflate PiSSA with a general training-method switch.","solutions":["Remove pissa_init: true for non-LoRA fine-tuning.","Set finetuning_type: lora to use PiSSA initialization.","Remember pissa_init also requires a compatible stage (it is rejected for ppo/kto/reference-based dpo, see the earlier check)."],"exampleFix":"# before (yaml)\nfinetuning_type: full\npissa_init: true\n\n# after (yaml)\nfinetuning_type: full\n# pissa_init removed","handlingStrategy":"validation","validationCode":"def check_pissa_lora(finetuning_type: str, pissa_init: bool) -> None:\n    if pissa_init and finetuning_type != \"lora\":\n        raise ValueError(\"pissa_init requires finetuning_type=lora\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["PiSSA decomposes base weights to seed LoRA A/B — meaningless without adapters.","Note the stage restriction too: ppo/kto/reference-dpo are rejected in a separate check."],"tags":["config","lora","pissa"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}