{"record":{"id":"ed1871d07934701d","repo":"hiyouga/LlamaFactory","slug":"quantized-model-only-accepts-a-single-adapter-mer","errorCode":null,"errorMessage":"Quantized model only accepts a single adapter. Merge them first.","messagePattern":"Quantized model only accepts a single adapter\\. Merge them first\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/parser.py","lineNumber":249,"sourceCode":") -> None:\n    if model_args.adapter_name_or_path is not None and finetuning_args.finetuning_type != \"lora\":\n        raise ValueError(\"Adapter is only valid for the LoRA method.\")\n\n    if model_args.quantization_bit is not None:\n        if finetuning_args.finetuning_type not in [\"lora\", \"oft\"]:\n            raise ValueError(\"Quantization is only compatible with the LoRA or OFT method.\")\n\n        if finetuning_args.pissa_init:\n            raise ValueError(\"Please use scripts/pissa_init.py to initialize PiSSA for a quantized model.\")\n\n        if model_args.resize_vocab:\n            raise ValueError(\"Cannot resize embedding layers of a quantized model.\")\n\n        if model_args.adapter_name_or_path is not None and finetuning_args.create_new_adapter:\n            raise ValueError(\"Cannot create new adapter upon a quantized model.\")\n\n        if model_args.adapter_name_or_path is not None and len(model_args.adapter_name_or_path) != 1:\n            raise ValueError(\"Quantized model only accepts a single adapter. Merge them first.\")\n\n\ndef _check_extra_dependencies(\n    model_args: \"ModelArguments\",\n    finetuning_args: \"FinetuningArguments\",\n    training_args: Optional[\"TrainingArguments\"] = None,\n) -> None:\n    if model_args.use_kt:\n        check_version(\"kt-kernel\", mandatory=True)\n        check_version(\"transformers-kt\", mandatory=True)\n        check_version(\"accelerate-kt\", mandatory=True)\n\n    if model_args.use_unsloth:\n        check_version(\"unsloth\", mandatory=True)\n\n    if model_args.enable_liger_kernel:\n        check_version(\"liger-kernel\", mandatory=True)\n","sourceCodeStart":231,"sourceCodeEnd":267,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/parser.py#L231-L267","documentation":"When a model is loaded quantized (quantization_bit set), LlamaFactory allows at most one adapter to be attached. adapter_name_or_path accepts a list for multi-adapter (merge/average) scenarios, but applying multiple adapters to a quantized base is unsupported, so the parser rejects len(adapter_name_or_path) != 1. The intended workflow is to merge adapters into the base model first, then quantize.","triggerScenarios":"A train/inference config with quantization_bit: 4 (or 8) plus adapter_name_or_path given as a list of two or more paths, e.g. adapter_name_or_path: [saves/adapter1, saves/adapter2]. Raised during argument checking in get_train_args() before training starts.","commonSituations":"Users doing model merging experiments (passing several LoRA checkpoint dirs to blend) while also keeping quantization_bit set from a memory-saving QLoRA config. Also happens when a comma-separated adapter string is parsed into a multi-element list.","solutions":["Reduce adapter_name_or_path to a single adapter path and remove the extra ones.","Merge the adapters into the base model first via llamafactory-cli export (export with the adapters applied), then point model_name_or_path at the merged model.","Drop quantization_bit if multi-adapter loading is essential and you can afford full-precision memory."],"exampleFix":"# before\nmodel_name_or_path: meta-llama/Llama-3-8B\nquantization_bit: 4\nadapter_name_or_path:\n  - saves/adapter1\n  - saves/adapter2\n\n# after (single adapter on the quantized model)\nmodel_name_or_path: meta-llama/Llama-3-8B\nquantization_bit: 4\nadapter_name_or_path: saves/adapter1","handlingStrategy":"validation","validationCode":"adapters = cfg.get(\"adapter_name_or_path\") or []\nif isinstance(adapters, str):\n    adapters = [adapters]\nif cfg.get(\"quantization_bit\") and len(adapters) > 1:\n    raise SystemExit(\"Quantized model accepts a single adapter; merge extras via export first\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Normalize adapter_name_or_path to a list in your config loader and assert len == 1 whenever quantization_bit is set.","Maintain a merge-then-quantize pipeline: export merged models, then quantize, instead of stacking adapters on QLoRA."],"tags":["config","qlora","adapter","merging","validation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}