hiyouga/LlamaFactory · error · ValueError

KTransformers only supports LoRA finetuning.

Error message

KTransformers only supports LoRA finetuning.

What it means

Raised in _setup_lora_tuning when use_kt is true but finetuning_type is not lora. The KTransformers integration only wraps peft's LoraConfig; full, freeze and OFT paths have no KT implementation.

Source

Thrown at src/llamafactory/model/adapter.py:278

                "target_modules": target_modules,
                "lora_alpha": finetuning_args.lora_alpha,
                "lora_dropout": finetuning_args.lora_dropout,
                "use_rslora": finetuning_args.use_rslora,
                "use_dora": finetuning_args.use_dora,
                "modules_to_save": finetuning_args.additional_target,
            }
        elif finetuning_args.finetuning_type == "oft":
            peft_kwargs = {
                "r": finetuning_args.oft_rank,
                "oft_block_size": finetuning_args.oft_block_size,
                "target_modules": target_modules,
                "module_dropout": finetuning_args.module_dropout,
                "modules_to_save": finetuning_args.additional_target,
            }

        if model_args.use_kt:
            if finetuning_args.finetuning_type != "lora":
                raise ValueError("KTransformers only supports LoRA finetuning.")

            peft_config = LoraConfig(task_type=TaskType.CAUSAL_LM, inference_mode=False, **peft_kwargs)
            model = get_peft_model(model, peft_config, autocast_adapter_dtype=cast_trainable_params_to_fp32)
        elif model_args.use_unsloth:
            if finetuning_args.finetuning_type == "oft":
                raise ValueError("Unsloth is currently not supported for OFT.")

            model = get_unsloth_peft_model(model, model_args, peft_kwargs)
        else:
            if finetuning_args.pissa_init:
                if finetuning_args.pissa_iter == -1:
                    logger.info_rank0("Using PiSSA initialization.")
                    peft_kwargs["init_lora_weights"] = "pissa"
                else:
                    logger.info_rank0(f"Using PiSSA initialization with FSVD steps {finetuning_args.pissa_iter}.")
                    peft_kwargs["init_lora_weights"] = f"pissa_niter_{finetuning_args.pissa_iter}"

            if finetuning_args.finetuning_type == "lora":

View on GitHub (pinned to f28afaf635)

Solutions

1. Set finetuning_type: lora when enable_kt: true.
2. Disable KTransformers (remove enable_kt) if you must do full or freeze tuning.

Example fix

# before
enable_kt: true
finetuning_type: full

# after
enable_kt: true
finetuning_type: lora

if model_args.get("enable_kt"):
    assert finetuning_args.get("finetuning_type") == "lora", \
        "KTransformers requires finetuning_type: lora"

When it happens

Trigger: Config with enable_kt: true together with finetuning_type: full, freeze or oft.

Common situations: Turning on KTransformers in an existing full/freeze tuning config to speed up large-model training.

Related errors

• KTransformers thin integration currently supports LoRA finet
• KTransformers accepts a single `adapter_name_or_path`.
• KTransformers does not support lora reward model.
• `reward_model_type` cannot be lora for Freeze/Full PPO train
• `use_llama_pro` is only valid for Freeze or LoRA training.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/ff77d8db3d049ac0. Report an issue: GitHub.