hiyouga/LlamaFactory · error · ValueError

Quantization is only compatible with the LoRA or OFT method.

Error message

Quantization is only compatible with the LoRA or OFT method.

What it means

Raised by _verify_model_args when quantization_bit is set and finetuning_type is neither 'lora' nor 'oft'. Quantized (QLoRA-style) weights are frozen INT4/INT8 matrices; only parameter-efficient deltas on top (LoRA/OFT) can train, so full/freeze/pissa training on quantized weights is impossible.

Source

Thrown at src/llamafactory/hparams/parser.py:237

def _set_env_vars() -> None:
    if is_torch_npu_available():
        # avoid JIT compile on NPU devices, see https://zhuanlan.zhihu.com/p/660875458
        torch.npu.set_compile_mode(jit_compile=is_env_enabled("NPU_JIT_COMPILE"))
        # avoid use fork method on NPU devices, see https://github.com/hiyouga/LLaMA-Factory/issues/7447
        os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"


def _verify_model_args(
    model_args: "ModelArguments",
    data_args: "DataArguments",
    finetuning_args: "FinetuningArguments",
) -> None:
    if model_args.adapter_name_or_path is not None and finetuning_args.finetuning_type != "lora":
        raise ValueError("Adapter is only valid for the LoRA method.")

    if model_args.quantization_bit is not None:
        if finetuning_args.finetuning_type not in ["lora", "oft"]:
            raise ValueError("Quantization is only compatible with the LoRA or OFT method.")

        if finetuning_args.pissa_init:
            raise ValueError("Please use scripts/pissa_init.py to initialize PiSSA for a quantized model.")

        if model_args.resize_vocab:
            raise ValueError("Cannot resize embedding layers of a quantized model.")

        if model_args.adapter_name_or_path is not None and finetuning_args.create_new_adapter:
            raise ValueError("Cannot create new adapter upon a quantized model.")

        if model_args.adapter_name_or_path is not None and len(model_args.adapter_name_or_path) != 1:
            raise ValueError("Quantized model only accepts a single adapter. Merge them first.")


def _check_extra_dependencies(
    model_args: "ModelArguments",
    finetuning_args: "FinetuningArguments",
    training_args: Optional["TrainingArguments"] = None,

View on GitHub (pinned to f28afaf635)

Solutions

1. Switch to `finetuning_type: lora` (classic QLoRA) or `oft` to train on a quantized model.
2. Or unset `quantization_bit` and load BF16 weights for full finetuning (requires enough GPU memory).

Example fix

# before (yaml)
quantization_bit: 4
finetuning_type: full

# after (yaml)
quantization_bit: 4
finetuning_type: lora

if cfg.get('quantization_bit') is not None and cfg.get('finetuning_type') not in ('lora', 'oft'):
    raise SystemExit('quantization requires finetuning_type lora or oft')

def quantization_compatible(cfg: dict) -> bool:
    return cfg.get('quantization_bit') is None or cfg.get('finetuning_type') in ('lora', 'oft')

When it happens

Trigger: A YAML combining quantization_bit: 8 (or 4) with finetuning_type: full or freeze; the check fires inside get_train_args before any model load.

Common situations: Users try full finetuning of a large model with a GPTQ/AWQ-quantized checkpoint to save memory, or leave quantization_bit set from a QLoRA experiment while switching finetuning_type.

Related errors

• Please use scripts/pissa_init.py to initialize PiSSA for a q
• Cannot create new adapter upon a quantized model.
• KTransformers uses LLaMA-Factory's `disable_gradient_checkpo
• KTransformers supplies its checkpoint context; remove `gradi
• Disable FSDP activation checkpointing when using KTransforme

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/edb1d2a5e71b0020. Report an issue: GitHub.