{"record":{"id":"bfb01082b345306e","repo":"hiyouga/LlamaFactory","slug":"cannot-create-new-adapter-upon-a-quantized-model","errorCode":null,"errorMessage":"Cannot create new adapter upon a quantized model.","messagePattern":"Cannot create new adapter upon a quantized model\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/parser.py","lineNumber":246,"sourceCode":"    model_args: \"ModelArguments\",\n    data_args: \"DataArguments\",\n    finetuning_args: \"FinetuningArguments\",\n) -> None:\n    if model_args.adapter_name_or_path is not None and finetuning_args.finetuning_type != \"lora\":\n        raise ValueError(\"Adapter is only valid for the LoRA method.\")\n\n    if model_args.quantization_bit is not None:\n        if finetuning_args.finetuning_type not in [\"lora\", \"oft\"]:\n            raise ValueError(\"Quantization is only compatible with the LoRA or OFT method.\")\n\n        if finetuning_args.pissa_init:\n            raise ValueError(\"Please use scripts/pissa_init.py to initialize PiSSA for a quantized model.\")\n\n        if model_args.resize_vocab:\n            raise ValueError(\"Cannot resize embedding layers of a quantized model.\")\n\n        if model_args.adapter_name_or_path is not None and finetuning_args.create_new_adapter:\n            raise ValueError(\"Cannot create new adapter upon a quantized model.\")\n\n        if model_args.adapter_name_or_path is not None and len(model_args.adapter_name_or_path) != 1:\n            raise ValueError(\"Quantized model only accepts a single adapter. Merge them first.\")\n\n\ndef _check_extra_dependencies(\n    model_args: \"ModelArguments\",\n    finetuning_args: \"FinetuningArguments\",\n    training_args: Optional[\"TrainingArguments\"] = None,\n) -> None:\n    if model_args.use_kt:\n        check_version(\"kt-kernel\", mandatory=True)\n        check_version(\"transformers-kt\", mandatory=True)\n        check_version(\"accelerate-kt\", mandatory=True)\n\n    if model_args.use_unsloth:\n        check_version(\"unsloth\", mandatory=True)\n","sourceCodeStart":228,"sourceCodeEnd":264,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/parser.py#L228-L264","documentation":"LlamaFactory refuses to create a fresh LoRA/OF(adapter on top of an already-quantized model (e.g. 4/8-bit via quantization_bit). The check fires in _check_model_args when all three of quantization_bit, adapter_name_or_path, and finetuning_args.create_new_adapter are set. Quantized weights cannot backprop into a new adapter initialization path safely, so the library blocks it at argument-parsing time, before any model is loaded.","triggerScenarios":"A YAML/JSON train config (or CLI args) that simultaneously sets quantization_bit: 4 (or 8), adapter_name_or_path: <path(s)> (loading existing adapters), and create_new_adapter: true. Calling llamafactory-cli train with such a config raises ValueError immediately in get_train_args().","commonSituations":"Users iteratively training LoRA on a QLoRA setup: they load a previously trained adapter and want to add a second new adapter for another task in the same run. Copying an old QLoRA config and flipping create_new_adapter: true without removing the old adapter path is the typical mistake.","solutions":["Set create_new_adapter: false in the finetuning section so the existing adapter is resumed/trained instead of creating a new one.","Remove adapter_name_or_path from the config if you truly want a brand-new adapter on the quantized base model (create_new_adapter is then irrelevant).","If you need multiple adapters, run on the non-quantized base model, create/train adapters there, then export merged weights and quantize afterwards.","Merge your existing adapters into the base model first (export_model / llamafactory-cli export) and start a new adapter from the merged model."],"exampleFix":"# before (YAML)\nmodel_name_or_path: meta-llama/Llama-3-8B\nquantization_bit: 4\nadapter_name_or_path: saves/llama3_lora\nfinetuning_type: lora\ncreate_new_adapter: true\n\n# after (train a fresh adapter, no old adapter loaded)\nmodel_name_or_path: meta-llama/Llama-3-8B\nquantization_bit: 4\nfinetuning_type: lora\ncreate_new_adapter: true","handlingStrategy":"validation","validationCode":"def check_qlora_new_adapter(cfg: dict) -> None:\n    m, f = cfg.get(\"model\", cfg), cfg.get(\"finetuning\", cfg)\n    if m.get(\"quantization_bit\") and f.get(\"adapter_name_or_path\") and f.get(\"create_new_adapter\"):\n        raise SystemExit(\"QLoRA + existing adapter + create_new_adapter is not allowed; drop adapter_name_or_path or set create_new_adapter: false\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep QLoRA configs minimal: never combine adapter_name_or_path with create_new_adapter.","One adapter task per config file; generate configs from templates that already encode the constraint.","Run a YAML lint step in CI that asserts these mutual-exclusion rules before submitting jobs."],"tags":["config","qlora","adapter","validation","llamafactory"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}