hiyouga/LlamaFactory · error · ValueError

The model is not a `PreTrainedModel`, export aborted.

Error message

The model is not a `PreTrainedModel`, export aborted.

What it means

export_model (src/llamafactory/train/tuner.py:195) requires the loaded object to be a transformers `PreTrainedModel` because it calls model.save_pretrained with sharding/dtype logic. If load_model returned another type (KTransformers model, vLLM engine object, or a custom wrapper), export aborts.

Source

Thrown at src/llamafactory/train/tuner.py:195

    model_args, data_args, finetuning_args, _ = get_infer_args(args)

    if model_args.export_dir is None:
        raise ValueError("Please specify `export_dir` to save model.")

    if model_args.adapter_name_or_path is not None and model_args.export_quantization_bit is not None:
        raise ValueError("Please merge adapters before quantizing the model.")

    tokenizer_module = load_tokenizer(model_args)
    tokenizer = tokenizer_module["tokenizer"]
    processor = tokenizer_module["processor"]
    template = get_template_and_fix_tokenizer(tokenizer, data_args)
    model = load_model(tokenizer, model_args, finetuning_args)  # must after fixing tokenizer to resize vocab

    if getattr(model, "quantization_method", None) is not None and model_args.adapter_name_or_path is not None:
        raise ValueError("Cannot merge adapters to a quantized model.")

    if not isinstance(model, PreTrainedModel):
        raise ValueError("The model is not a `PreTrainedModel`, export aborted.")

    if getattr(model, "quantization_method", None) is not None:  # quantized model adopts float16 type
        setattr(model.config, "torch_dtype", torch.float16)
    else:
        if model_args.infer_dtype == "auto":
            output_dtype = getattr(model.config, "torch_dtype", torch.float32)
            if output_dtype == torch.float32:  # if infer_dtype is auto, try using half precision first
                output_dtype = infer_optim_dtype(torch.bfloat16)
        else:
            output_dtype = getattr(torch, model_args.infer_dtype)

        setattr(model.config, "torch_dtype", output_dtype)
        model = model.to(output_dtype)
        logger.info_rank0(f"Convert model dtype to: {output_dtype}.")

    # Prepare save arguments (safe_serialization removed in transformers v5.0.0)
    save_kwargs = {
        "save_directory": model_args.export_dir,

View on GitHub (pinned to f28afaf635)

Solutions

1. Strip backend-specific flags (use_kt etc.) from the export config so load_model returns a PreTrainedModel.
2. For KTransformers, use its own weight-saving tooling instead of llamafactory-cli export.
3. If a custom loader is involved, ensure it returns a PreTrainedModel before export.

Example fix

# before (export yaml)
use_kt: true
model_name_or_path: ...

# after
model_name_or_path: ...   # plain HF load

from transformers import PreTrainedModel
assert isinstance(model, PreTrainedModel), "export requires an HF PreTrainedModel"

from transformers import PreTrainedModel
def is_exportable(model) -> bool:
    return isinstance(model, PreTrainedModel)

When it happens

Trigger: Calling export with `use_kt: true` (KTransformers backend), a megatron/hyper-parallel style model, or any config that makes load_model return a non-HF object.

Common situations: Users who trained or loaded with an alternate backend and then reuse the same model_args for `llamafactory-cli export`.

Related errors

• The current model does not support `chat`.
• The current model does not support `stream_chat`.
• Dict is not supported.
• Quantization dataset is necessary for exporting.
• `kt_cpu_activation` must be `retain` or `recompute`.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/9235ba4f926cbf45. Report an issue: GitHub.