hiyouga/LlamaFactory · error · ValueError

Please merge adapters before quantizing the model.

Error message

Please merge adapters before quantizing the model.

What it means

export_model (src/llamafactory/train/tuner.py:183) rejects combining `adapter_name_or_path` (adapters to merge) with `export_quantization_bit` (quantize during export). Quantization tooling consumes plain merged weights, so the two operations must be sequential, not one step.

Source

Thrown at src/llamafactory/train/tuner.py:183

    if "-h" in args or "--help" in args:
        get_train_args(args)

    ray_args = get_ray_args(args)
    callbacks = callbacks or []
    if ray_args.use_ray:
        _ray_training_function(ray_args, config={"args": args, "callbacks": callbacks})
    else:
        _training_function(config={"args": args, "callbacks": callbacks})


def export_model(args: Optional[dict[str, Any]] = None) -> None:
    model_args, data_args, finetuning_args, _ = get_infer_args(args)

    if model_args.export_dir is None:
        raise ValueError("Please specify `export_dir` to save model.")

    if model_args.adapter_name_or_path is not None and model_args.export_quantization_bit is not None:
        raise ValueError("Please merge adapters before quantizing the model.")

    tokenizer_module = load_tokenizer(model_args)
    tokenizer = tokenizer_module["tokenizer"]
    processor = tokenizer_module["processor"]
    template = get_template_and_fix_tokenizer(tokenizer, data_args)
    model = load_model(tokenizer, model_args, finetuning_args)  # must after fixing tokenizer to resize vocab

    if getattr(model, "quantization_method", None) is not None and model_args.adapter_name_or_path is not None:
        raise ValueError("Cannot merge adapters to a quantized model.")

    if not isinstance(model, PreTrainedModel):
        raise ValueError("The model is not a `PreTrainedModel`, export aborted.")

    if getattr(model, "quantization_method", None) is not None:  # quantized model adopts float16 type
        setattr(model.config, "torch_dtype", torch.float16)
    else:
        if model_args.infer_dtype == "auto":
            output_dtype = getattr(model.config, "torch_dtype", torch.float32)

View on GitHub (pinned to f28afaf635)

Solutions

1. First export with adapters merged and no quantization: set adapter_name_or_path and leave export_quantization_bit unset.
2. Then run a second export from the merged directory with export_quantization_bit set and no adapter_name_or_path.
3. For GPTQ-style quantization, use the dedicated quantization examples (export_gptq/gptq calib data) as the second stage.

Example fix

# step 1: merge
adapter_name_or_path: output/lora
export_dir: output/merged

# step 2: quantize (new config, no adapter)
model_name_or_path: output/merged
export_dir: output/merged-gptq
export_quantization_bit: 8
export_quantization_method: gptq

assert not (adapter_name_or_path and export_quantization_bit), "merge and quantize must be two sequential exports"

When it happens

Trigger: An export YAML that sets both `adapter_name_or_path:` and `export_quantization_bit: 8` (or 4).

Common situations: Trying to produce a final deployable 8-bit/4-bit artifact directly from a LoRA adapter in one command.

Related errors

• Cannot merge adapters to a quantized model.
• Quantization dataset is necessary for exporting.
• vLLM only accepts a single adapter. Merge them first.
• DoRA is not compatible with PTQ-quantized models.
• Cannot find satisfying example, considering decrease `export

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/d8a4b71329a754e3. Report an issue: GitHub.