hiyouga/LlamaFactory · error · ValueError

Quantized model only accepts a single adapter. Merge them fi

Error message

Quantized model only accepts a single adapter. Merge them first.

What it means

When a model is loaded quantized (quantization_bit set), LlamaFactory allows at most one adapter to be attached. adapter_name_or_path accepts a list for multi-adapter (merge/average) scenarios, but applying multiple adapters to a quantized base is unsupported, so the parser rejects len(adapter_name_or_path) != 1. The intended workflow is to merge adapters into the base model first, then quantize.

Source

Thrown at src/llamafactory/hparams/parser.py:249

) -> None:
    if model_args.adapter_name_or_path is not None and finetuning_args.finetuning_type != "lora":
        raise ValueError("Adapter is only valid for the LoRA method.")

    if model_args.quantization_bit is not None:
        if finetuning_args.finetuning_type not in ["lora", "oft"]:
            raise ValueError("Quantization is only compatible with the LoRA or OFT method.")

        if finetuning_args.pissa_init:
            raise ValueError("Please use scripts/pissa_init.py to initialize PiSSA for a quantized model.")

        if model_args.resize_vocab:
            raise ValueError("Cannot resize embedding layers of a quantized model.")

        if model_args.adapter_name_or_path is not None and finetuning_args.create_new_adapter:
            raise ValueError("Cannot create new adapter upon a quantized model.")

        if model_args.adapter_name_or_path is not None and len(model_args.adapter_name_or_path) != 1:
            raise ValueError("Quantized model only accepts a single adapter. Merge them first.")


def _check_extra_dependencies(
    model_args: "ModelArguments",
    finetuning_args: "FinetuningArguments",
    training_args: Optional["TrainingArguments"] = None,
) -> None:
    if model_args.use_kt:
        check_version("kt-kernel", mandatory=True)
        check_version("transformers-kt", mandatory=True)
        check_version("accelerate-kt", mandatory=True)

    if model_args.use_unsloth:
        check_version("unsloth", mandatory=True)

    if model_args.enable_liger_kernel:
        check_version("liger-kernel", mandatory=True)

View on GitHub (pinned to f28afaf635)

Solutions

1. Reduce adapter_name_or_path to a single adapter path and remove the extra ones.
2. Merge the adapters into the base model first via llamafactory-cli export (export with the adapters applied), then point model_name_or_path at the merged model.
3. Drop quantization_bit if multi-adapter loading is essential and you can afford full-precision memory.

Example fix

# before
model_name_or_path: meta-llama/Llama-3-8B
quantization_bit: 4
adapter_name_or_path:
  - saves/adapter1
  - saves/adapter2

# after (single adapter on the quantized model)
model_name_or_path: meta-llama/Llama-3-8B
quantization_bit: 4
adapter_name_or_path: saves/adapter1

adapters = cfg.get("adapter_name_or_path") or []
if isinstance(adapters, str):
    adapters = [adapters]
if cfg.get("quantization_bit") and len(adapters) > 1:
    raise SystemExit("Quantized model accepts a single adapter; merge extras via export first")

When it happens

Trigger: A train/inference config with quantization_bit: 4 (or 8) plus adapter_name_or_path given as a list of two or more paths, e.g. adapter_name_or_path: [saves/adapter1, saves/adapter2]. Raised during argument checking in get_train_args() before training starts.

Common situations: Users doing model merging experiments (passing several LoRA checkpoint dirs to blend) while also keeping quantization_bit set from a memory-saving QLoRA config. Also happens when a comma-separated adapter string is parsed into a multi-element list.

Related errors

• Cannot create new adapter upon a quantized model.
• Unknown mixing strategy: {data_args.mix_strategy}.
• Cannot specify `val_size` if `eval_dataset` is not None.
• `kt_model_max_length` must be a positive integer.
• `adapter_folder` must stay inside the KT adapter directory.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/ed1871d07934701d. Report an issue: GitHub.