{"record":{"id":"edb1d2a5e71b0020","repo":"hiyouga/LlamaFactory","slug":"quantization-is-only-compatible-with-the-lora-or-o","errorCode":null,"errorMessage":"Quantization is only compatible with the LoRA or OFT method.","messagePattern":"Quantization is only compatible with the LoRA or OFT method\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/parser.py","lineNumber":237,"sourceCode":"def _set_env_vars() -> None:\n    if is_torch_npu_available():\n        # avoid JIT compile on NPU devices, see https://zhuanlan.zhihu.com/p/660875458\n        torch.npu.set_compile_mode(jit_compile=is_env_enabled(\"NPU_JIT_COMPILE\"))\n        # avoid use fork method on NPU devices, see https://github.com/hiyouga/LLaMA-Factory/issues/7447\n        os.environ[\"VLLM_WORKER_MULTIPROC_METHOD\"] = \"spawn\"\n\n\ndef _verify_model_args(\n    model_args: \"ModelArguments\",\n    data_args: \"DataArguments\",\n    finetuning_args: \"FinetuningArguments\",\n) -> None:\n    if model_args.adapter_name_or_path is not None and finetuning_args.finetuning_type != \"lora\":\n        raise ValueError(\"Adapter is only valid for the LoRA method.\")\n\n    if model_args.quantization_bit is not None:\n        if finetuning_args.finetuning_type not in [\"lora\", \"oft\"]:\n            raise ValueError(\"Quantization is only compatible with the LoRA or OFT method.\")\n\n        if finetuning_args.pissa_init:\n            raise ValueError(\"Please use scripts/pissa_init.py to initialize PiSSA for a quantized model.\")\n\n        if model_args.resize_vocab:\n            raise ValueError(\"Cannot resize embedding layers of a quantized model.\")\n\n        if model_args.adapter_name_or_path is not None and finetuning_args.create_new_adapter:\n            raise ValueError(\"Cannot create new adapter upon a quantized model.\")\n\n        if model_args.adapter_name_or_path is not None and len(model_args.adapter_name_or_path) != 1:\n            raise ValueError(\"Quantized model only accepts a single adapter. Merge them first.\")\n\n\ndef _check_extra_dependencies(\n    model_args: \"ModelArguments\",\n    finetuning_args: \"FinetuningArguments\",\n    training_args: Optional[\"TrainingArguments\"] = None,","sourceCodeStart":219,"sourceCodeEnd":255,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/parser.py#L219-L255","documentation":"Raised by _verify_model_args when quantization_bit is set and finetuning_type is neither 'lora' nor 'oft'. Quantized (QLoRA-style) weights are frozen INT4/INT8 matrices; only parameter-efficient deltas on top (LoRA/OFT) can train, so full/freeze/pissa training on quantized weights is impossible.","triggerScenarios":"A YAML combining quantization_bit: 8 (or 4) with finetuning_type: full or freeze; the check fires inside get_train_args before any model load.","commonSituations":"Users try full finetuning of a large model with a GPTQ/AWQ-quantized checkpoint to save memory, or leave quantization_bit set from a QLoRA experiment while switching finetuning_type.","solutions":["Switch to `finetuning_type: lora` (classic QLoRA) or `oft` to train on a quantized model.","Or unset `quantization_bit` and load BF16 weights for full finetuning (requires enough GPU memory)."],"exampleFix":"# before (yaml)\nquantization_bit: 4\nfinetuning_type: full\n\n# after (yaml)\nquantization_bit: 4\nfinetuning_type: lora","handlingStrategy":"validation","validationCode":"if cfg.get('quantization_bit') is not None and cfg.get('finetuning_type') not in ('lora', 'oft'):\n    raise SystemExit('quantization requires finetuning_type lora or oft')","typeGuard":"def quantization_compatible(cfg: dict) -> bool:\n    return cfg.get('quantization_bit') is None or cfg.get('finetuning_type') in ('lora', 'oft')","tryCatchPattern":null,"preventionTips":["Remember quantized checkpoints are frozen; plan PEFT for memory-constrained runs."],"tags":["quantization","qlora","config","llamafactory"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}