{"record":{"id":"28817687eddd8f4b","repo":"hiyouga/LlamaFactory","slug":"use-rslora-is-only-valid-for-lora-training","errorCode":null,"errorMessage":"`use_rslora` is only valid for LoRA training.","messagePattern":"`use_rslora` is only valid for LoRA training\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/finetuning_args.py","lineNumber":629,"sourceCode":"\n        if self.use_llama_pro and self.finetuning_type == \"full\":\n            raise ValueError(\"`use_llama_pro` is only valid for Freeze or LoRA training.\")\n\n        if self.finetuning_type == \"lora\" and (self.use_galore or self.use_apollo or self.use_badam):\n            raise ValueError(\"Cannot use LoRA with GaLore, APOLLO or BAdam together.\")\n\n        if int(self.use_galore) + int(self.use_apollo) + (self.use_badam) > 1:\n            raise ValueError(\"Cannot use GaLore, APOLLO or BAdam together.\")\n\n        if self.pissa_init and (self.stage in [\"ppo\", \"kto\"] or self.use_ref_model):\n            raise ValueError(\"Cannot use PiSSA for current training stage.\")\n\n        if self.finetuning_type != \"lora\":\n            if self.loraplus_lr_ratio is not None:\n                raise ValueError(\"`loraplus_lr_ratio` is only valid for LoRA training.\")\n\n            if self.use_rslora:\n                raise ValueError(\"`use_rslora` is only valid for LoRA training.\")\n\n            if self.use_dora:\n                raise ValueError(\"`use_dora` is only valid for LoRA training.\")\n\n            if self.pissa_init:\n                raise ValueError(\"`pissa_init` is only valid for LoRA training.\")\n\n    def to_dict(self) -> dict[str, Any]:\n        args = asdict(self)\n        args = {k: f\"<{k.upper()}>\" if k.endswith(\"api_key\") else v for k, v in args.items()}\n        return args\n","sourceCodeStart":611,"sourceCodeEnd":641,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/finetuning_args.py#L611-L641","documentation":"Rank-Stabilized LoRA (rsLoRA) rescales the adapter update by sqrt(rank) instead of 1/rank; the scaling only applies to LoRA adapters. FinetuningArguments.__post_init__ (src/llamafactory/hparams/finetuning_args.py:629) rejects use_rslora: true when finetuning_type is not lora.","triggerScenarios":"use_rslora: true with finetuning_type: full or freeze.","commonSituations":"Users migrating a LoRA recipe (with rsLoRA for high-rank stability) to full fine-tuning and leaving the flag; or combining rsLoRA with GaLore-style full training.","solutions":["Remove use_rslora: true for full/freeze training.","Set finetuning_type: lora to keep using rsLoRA.","If high-rank adaptation was the goal, use LoRA with a larger lora_rank plus use_rslora."],"exampleFix":"# before (yaml)\nfinetuning_type: full\nuse_rslora: true\n\n# after (yaml)\nfinetuning_type: full\n# use_rslora removed","handlingStrategy":"validation","validationCode":"def check_rslora(finetuning_type: str, use_rslora: bool) -> None:\n    if use_rslora and finetuning_type != \"lora\":\n        raise ValueError(\"use_rslora requires finetuning_type=lora\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["rsLoRA, DoRA, LoRA+, PiSSA are all LoRA-exclusive: audit them together when changing finetuning_type.","Keep a shared 'lora-extras' YAML snippet you include only for LoRA runs."],"tags":["config","lora","rslora"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}