{"record":{"id":"14cd767477bd0288","repo":"hiyouga/LlamaFactory","slug":"quantized-models-can-only-be-used-for-the-lora-or","errorCode":null,"errorMessage":"Quantized models can only be used for the LoRA or OFT tuning.","messagePattern":"Quantized models can only be used for the LoRA or OFT tuning\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/model/adapter.py","lineNumber":332,"sourceCode":"    return model\n\n\ndef init_adapter(\n    config: \"PretrainedConfig\",\n    model: \"PreTrainedModel\",\n    model_args: \"ModelArguments\",\n    finetuning_args: \"FinetuningArguments\",\n    is_trainable: bool,\n) -> \"PreTrainedModel\":\n    r\"\"\"Initialize the adapters.\n\n    Support full-parameter, freeze and LoRA training.\n\n    Note that the trainable parameters must be cast to float32.\n    \"\"\"\n    if is_trainable and getattr(model, \"quantization_method\", None) is not None:\n        if finetuning_args.finetuning_type not in [\"lora\", \"oft\"]:\n            raise ValueError(\"Quantized models can only be used for the LoRA or OFT tuning.\")\n\n        if finetuning_args.pissa_init:\n            raise ValueError(\"Cannot initialize PiSSA adapter on quantized models.\")\n\n    # cast trainable parameters to float32 if:\n    # 1. is_trainable and not pure_bf16 and not badam and quantization_bit is not None (qlora)\n    # 2. is_trainable and not pure_bf16 and not badam and not zero3 (zero3 already in fp32)\n    cast_trainable_params_to_fp32 = False\n    if not is_trainable:\n        pass\n    elif finetuning_args.pure_bf16 or finetuning_args.use_badam:\n        logger.info_rank0(\"Pure bf16 / BAdam detected, remaining trainable params in half precision.\")\n    elif model_args.quantization_bit is None and is_deepspeed_zero3_enabled():\n        logger.info_rank0(\"DeepSpeed ZeRO3 detected, remaining trainable params in float32.\")\n    else:\n        logger.info_rank0(\"Upcasting trainable params to float32.\")\n        cast_trainable_params_to_fp32 = True\n","sourceCodeStart":314,"sourceCodeEnd":350,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/model/adapter.py#L314-L350","documentation":"Raised in setup_adapter when the model was loaded quantized (quantization_method set) and training is requested (is_trainable) with a finetuning_type other than lora/oft. Quantized weights cannot be updated in place, so only adapter-based methods (QLoRA/OFT) are trainable; this is the QLoRA contract enforced up front.","triggerScenarios":"quantization_bit set (bnb) or a GPTQ/AWQ model, with do_train and finetuning_type: full or freeze.","commonSituations":"Leaving finetuning_type: full (the default in some examples) while adding quantization_bit: 4 to fit a big model; freeze-tuning a GPTQ checkpoint.","solutions":["Set finetuning_type: lora (the standard QLoRA setup) or oft on quantized models.","If you need full fine-tuning, remove quantization_bit and load the unquantized checkpoint (possibly with DeepSpeed/FSDP offload for memory)."],"exampleFix":"# before\nquantization_bit: 4\nfinetuning_type: full\n\n# after\nquantization_bit: 4\nfinetuning_type: lora","handlingStrategy":"validation","validationCode":"is_quantized = cfg[\"model_args\"].get(\"quantization_bit\") is not None or bool(\n    {\"GPTQ\", \"AWQ\"} & {t for t in cfg[\"model_args\"].get(\"model_name_or_path\", \"\").upper().split(\"-\")}\n)\nif is_quantized and do_train:\n    assert cfg[\"finetuning_args\"][\"finetuning_type\"] in (\"lora\", \"oft\"), \\\n        \"quantized models are trainable via lora/oft only (QLoRA)\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Whenever quantization_bit is set, default finetuning_type to lora in your templates.","For full tuning, plan for unquantized weights plus offload (DeepSpeed/FSDP) instead of quantization."],"tags":["qlora","quantization","finetuning-type","full-tuning"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}