{"record":{"id":"d8a4b71329a754e3","repo":"hiyouga/LlamaFactory","slug":"please-merge-adapters-before-quantizing-the-model","errorCode":null,"errorMessage":"Please merge adapters before quantizing the model.","messagePattern":"Please merge adapters before quantizing the model\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/train/tuner.py","lineNumber":183,"sourceCode":"    if \"-h\" in args or \"--help\" in args:\n        get_train_args(args)\n\n    ray_args = get_ray_args(args)\n    callbacks = callbacks or []\n    if ray_args.use_ray:\n        _ray_training_function(ray_args, config={\"args\": args, \"callbacks\": callbacks})\n    else:\n        _training_function(config={\"args\": args, \"callbacks\": callbacks})\n\n\ndef export_model(args: Optional[dict[str, Any]] = None) -> None:\n    model_args, data_args, finetuning_args, _ = get_infer_args(args)\n\n    if model_args.export_dir is None:\n        raise ValueError(\"Please specify `export_dir` to save model.\")\n\n    if model_args.adapter_name_or_path is not None and model_args.export_quantization_bit is not None:\n        raise ValueError(\"Please merge adapters before quantizing the model.\")\n\n    tokenizer_module = load_tokenizer(model_args)\n    tokenizer = tokenizer_module[\"tokenizer\"]\n    processor = tokenizer_module[\"processor\"]\n    template = get_template_and_fix_tokenizer(tokenizer, data_args)\n    model = load_model(tokenizer, model_args, finetuning_args)  # must after fixing tokenizer to resize vocab\n\n    if getattr(model, \"quantization_method\", None) is not None and model_args.adapter_name_or_path is not None:\n        raise ValueError(\"Cannot merge adapters to a quantized model.\")\n\n    if not isinstance(model, PreTrainedModel):\n        raise ValueError(\"The model is not a `PreTrainedModel`, export aborted.\")\n\n    if getattr(model, \"quantization_method\", None) is not None:  # quantized model adopts float16 type\n        setattr(model.config, \"torch_dtype\", torch.float16)\n    else:\n        if model_args.infer_dtype == \"auto\":\n            output_dtype = getattr(model.config, \"torch_dtype\", torch.float32)","sourceCodeStart":165,"sourceCodeEnd":201,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/train/tuner.py#L165-L201","documentation":"export_model (src/llamafactory/train/tuner.py:183) rejects combining `adapter_name_or_path` (adapters to merge) with `export_quantization_bit` (quantize during export). Quantization tooling consumes plain merged weights, so the two operations must be sequential, not one step.","triggerScenarios":"An export YAML that sets both `adapter_name_or_path:` and `export_quantization_bit: 8` (or 4).","commonSituations":"Trying to produce a final deployable 8-bit/4-bit artifact directly from a LoRA adapter in one command.","solutions":["First export with adapters merged and no quantization: set adapter_name_or_path and leave export_quantization_bit unset.","Then run a second export from the merged directory with export_quantization_bit set and no adapter_name_or_path.","For GPTQ-style quantization, use the dedicated quantization examples (export_gptq/gptq calib data) as the second stage."],"exampleFix":"# step 1: merge\nadapter_name_or_path: output/lora\nexport_dir: output/merged\n\n# step 2: quantize (new config, no adapter)\nmodel_name_or_path: output/merged\nexport_dir: output/merged-gptq\nexport_quantization_bit: 8\nexport_quantization_method: gptq","handlingStrategy":"validation","validationCode":"assert not (adapter_name_or_path and export_quantization_bit), \"merge and quantize must be two sequential exports\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Split merge and quantization into two export configs/steps.","Lint export configs against known mutually exclusive flag pairs."],"tags":["export","quantization","lora","merge"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}