hiyouga/LlamaFactory · error · ValueError

Unsloth is currently not supported for OFT.

Error message

Unsloth is currently not supported for OFT.

What it means

Raised in _setup_lora_tuning when use_unsloth is true and finetuning_type is oft. The Unsloth integration path (get_unsloth_peft_model) only implements accelerated LoRA; OFT adapters are not supported by it.

Source

Thrown at src/llamafactory/model/adapter.py:284

            }
        elif finetuning_args.finetuning_type == "oft":
            peft_kwargs = {
                "r": finetuning_args.oft_rank,
                "oft_block_size": finetuning_args.oft_block_size,
                "target_modules": target_modules,
                "module_dropout": finetuning_args.module_dropout,
                "modules_to_save": finetuning_args.additional_target,
            }

        if model_args.use_kt:
            if finetuning_args.finetuning_type != "lora":
                raise ValueError("KTransformers only supports LoRA finetuning.")

            peft_config = LoraConfig(task_type=TaskType.CAUSAL_LM, inference_mode=False, **peft_kwargs)
            model = get_peft_model(model, peft_config, autocast_adapter_dtype=cast_trainable_params_to_fp32)
        elif model_args.use_unsloth:
            if finetuning_args.finetuning_type == "oft":
                raise ValueError("Unsloth is currently not supported for OFT.")

            model = get_unsloth_peft_model(model, model_args, peft_kwargs)
        else:
            if finetuning_args.pissa_init:
                if finetuning_args.pissa_iter == -1:
                    logger.info_rank0("Using PiSSA initialization.")
                    peft_kwargs["init_lora_weights"] = "pissa"
                else:
                    logger.info_rank0(f"Using PiSSA initialization with FSVD steps {finetuning_args.pissa_iter}.")
                    peft_kwargs["init_lora_weights"] = f"pissa_niter_{finetuning_args.pissa_iter}"

            if finetuning_args.finetuning_type == "lora":
                peft_config = LoraConfig(
                    task_type=TaskType.CAUSAL_LM,
                    inference_mode=False,
                    **peft_kwargs,
                )
            elif finetuning_args.finetuning_type == "oft":

View on GitHub (pinned to f28afaf635)

Solutions

1. Set finetuning_type: lora when using Unsloth.
2. Remove use_unsloth to run OFT on the standard peft path.

Example fix

# before
use_unsloth: true
finetuning_type: oft

# after
use_unsloth: false
finetuning_type: oft

if model_args.get("use_unsloth"):
    assert finetuning_args.get("finetuning_type") != "oft", \
        "Unsloth does not implement OFT; use lora or disable use_unsloth"

When it happens

Trigger: Config with use_unsloth: true and finetuning_type: oft.

Common situations: Enabling Unsloth for memory/speed in an OFT experiment config.

Related errors

• `reward_model_type` cannot be oft for Freeze/Full PPO traini
• `use_llama_pro` is only valid for Freeze or LoRA training.
• KTransformers cannot be combined with Unsloth checkpoint wra
• Megatron Bridge only supports `full` and `lora` finetuning.
• Unsloth does not support lora reward model.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/b938454cffa62788. Report an issue: GitHub.