hiyouga/LlamaFactory · error · ValueError

vLLM only accepts a single adapter. Merge them first.

Error message

vLLM only accepts a single adapter. Merge them first.

What it means

Raised in get_infer_args when infer_backend is vllm and adapter_name_or_path contains more than one adapter path. The vLLM LoRA integration registers a single LoRA adapter; serving several unmerged adapters at once is not supported, so you must merge them into the base model first.

Source

Thrown at src/llamafactory/hparams/parser.py:687

def get_infer_args(args: dict[str, Any] | list[str] | None = None) -> _INFER_CLS:
    model_args, data_args, finetuning_args, generating_args = _parse_infer_args(args)

    # Setup logging
    _set_transformers_logging()

    # Check arguments
    if model_args.infer_backend == "vllm":
        if finetuning_args.stage != "sft":
            raise ValueError("vLLM engine only supports auto-regressive models.")

        if model_args.quantization_bit is not None:
            raise ValueError("vLLM engine does not support bnb quantization (GPTQ and AWQ are supported).")

        if model_args.rope_scaling is not None:
            raise ValueError("vLLM engine does not support RoPE scaling.")

        if model_args.adapter_name_or_path is not None and len(model_args.adapter_name_or_path) != 1:
            raise ValueError("vLLM only accepts a single adapter. Merge them first.")

    _set_env_vars()
    _verify_model_args(model_args, data_args, finetuning_args)
    _check_extra_dependencies(model_args, finetuning_args)

    # Post-process model arguments
    if model_args.export_dir is not None and model_args.export_device == "cpu":
        model_args.device_map = {"": torch.device("cpu")}
        if data_args.cutoff_len != DataArguments().cutoff_len:  # override cutoff_len if it is not default
            model_args.model_max_length = data_args.cutoff_len
    else:
        model_args.device_map = "auto"

    model_args.configure_kt_loading(finetuning_args, data_args.cutoff_len)

    return model_args, data_args, finetuning_args, generating_args

View on GitHub (pinned to f28afaf635)

Solutions

1. Keep only one adapter in adapter_name_or_path.
2. Merge the multiple adapters first with llamafactory-cli export (set adapter_name_or_path to the full list, export_dir to the merged output), then point model_name_or_path at the merged model.
3. Alternatively merge adapters via peft outside LlamaFactory and serve the merged checkpoint.

Example fix

# before
infer_backend: vllm
adapter_name_or_path:
  - saves/lora-base
  - saves/lora-style

# after (after exporting merged model to saves/merged)
infer_backend: vllm
model_name_or_path: saves/merged

adapters = cfg["model_args"].get("adapter_name_or_path") or []
if cfg["model_args"].get("infer_backend") == "vllm":
    assert isinstance(adapters, str) or len(adapters) <= 1, \
        "vLLM accepts one adapter; merge multiples via llamafactory-cli export first"

When it happens

Trigger: model_args with infer_backend: vllm and adapter_name_or_path set to a list of two or more paths, e.g. [saves/lora1, saves/lora2].

Common situations: Stacking a base LoRA with a continuation or style LoRA; iterating experiments and listing every checkpoint adapter in one config.

Related errors

• Please merge adapters before quantizing the model.
• Cannot merge adapters to a quantized model.
• Currently merge and export model function is only supported
• Please set adapter_name_or_path to merge adapters into base
• vLLM not install, you may need to run `pip install vllm` or

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/cf405a5ef5b244b0. Report an issue: GitHub.