{"record":{"id":"21765114ea8d82ca","repo":"hiyouga/LlamaFactory","slug":"when-adapter-name-or-path-is-provided-for-traini","errorCode":null,"errorMessage":"When `adapter_name_or_path` is provided for training, only a single LoRA adapter is supported. Training will continue on the specified adapter. Please merge multiple adapters before starting a new LoRA adapter.","messagePattern":"When `adapter_name_or_path` is provided for training, only a single LoRA adapter is supported\\. Training will continue on the specified adapter\\. Please merge multiple adapters before starting a new LoRA adapter\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/plugins/model_plugins/peft.py","lineNumber":129,"sourceCode":"\n\ndef load_adapter(model: HFModel, adapter_name_or_path: list[str] | str, is_train: bool) -> HFModel:\n    r\"\"\"Loads adapter(s) into the model.\n\n    Determine adapter usage based on mode:\n    - Training: Load the single adapter for continued training.\n    - Inference: Merge all adapters to clean up the model.\n    - Unmergeable: Keep the single adapter active without merging.\n    \"\"\"\n    if not isinstance(adapter_name_or_path, list):\n        adapter_name_or_path = [adapter_name_or_path]\n\n    # TODO\n    # Adapters fix for deepspeed and quant\n    # Adapters fix for vision\n\n    if is_train and len(adapter_name_or_path) > 1:\n        raise ValueError(\n            \"When `adapter_name_or_path` is provided for training, only a single LoRA adapter is supported. \"\n            \"Training will continue on the specified adapter. \"\n            \"Please merge multiple adapters before starting a new LoRA adapter.\"\n        )\n\n    if is_train:\n        adapter_to_merge = []\n        adapter_to_resume = adapter_name_or_path[0]\n    else:\n        adapter_to_merge = adapter_name_or_path\n        adapter_to_resume = None\n\n    if adapter_to_merge:\n        model = merge_adapters(model, adapter_to_merge)\n\n    if adapter_to_resume is not None:\n        model = PeftModel.from_pretrained(model, adapter_to_resume, is_trainable=is_train)\n        if is_train:","sourceCodeStart":111,"sourceCodeEnd":147,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/plugins/model_plugins/peft.py#L111-L147","documentation":"When resuming LoRA training with adapter_name_or_path, the PEFT plugin only supports exactly one adapter for continued training; PEFT cannot train on top of multiple simultaneously loaded adapters. If is_train is true and more than one adapter path is given, it raises ValueError telling you to merge adapters first.","triggerScenarios":"Passing adapter_name_or_path as a list with 2+ entries (e.g. [\"lora/v1\",\"lora/v2\"]) together with a training stage (is_train). Multi-adapter lists are only valid for inference, where they get merged.","commonSituations":"Copying an inference-style config (multiple adapters for merging/evaluation) into a train config; attempting to stack a new LoRA on several previous adapters; YAML lists under adapter_name_or_path for continued fine-tuning.","solutions":["Keep only the single adapter you want to continue training in adapter_name_or_path","Merge the other adapters first (llamafactory-cli export / merge_and_unload) and point training at the merged model, or chain adapters by training sequentially","For multi-adapter inference, ensure the stage is inference so is_train is false and merging applies"],"exampleFix":"# before\nadapter_name_or_path:\n  - saves/lora_v1\n  - saves/lora_v2\nstage: sft\n\n# after\nadapter_name_or_path: saves/lora_v2\nstage: sft\n# (or merge lora_v1 into the base model and train from the merged checkpoint)","handlingStrategy":"validation","validationCode":"if stage in {\"sft\",\"dpo\",\"rm\",\"kto\",\"pt\"} and isinstance(adapter_name_or_path, list) and len(adapter_name_or_path) > 1:\n    raise ValueError(\"training supports a single adapter; merge extras first\")","typeGuard":"def is_single_adapter_for_training(paths, is_train: bool) -> bool:\n    return not is_train or (isinstance(paths, str) or len(paths) == 1)","tryCatchPattern":null,"preventionTips":["Use a single adapter path in train configs; reserve multi-adapter lists for inference","Merge finished adapters (export/merge_and_unload) before stacking new LoRA training"],"tags":["configuration","lora","peft","training"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}