hiyouga/LlamaFactory · error · ValueError

KTransformers thin integration currently supports LoRA finet

Error message

KTransformers thin integration currently supports LoRA finetuning only.

What it means

Raised by get_kt_config_dict when finetuning_args.finetuning_type is anything other than 'lora'. The KT thin integration hard-codes kt_train_mode='lora' and kt_full_weight_grad=False, so full/pissa/freeze/Galore and other LLaMA-Factory finetuning types have no KT mapping.

Source

Thrown at src/llamafactory/hparams/model_args.py:612

            raise ValueError("Disable FSDP activation checkpointing when using KTransformers.")
        if os.environ.get("FSDP_ACTIVATION_CHECKPOINTING", "false").lower() in {"1", "true", "yes"}:
            raise ValueError("Disable FSDP activation checkpointing when using KTransformers.")

        self.get_kt_activation_policy()
        if not self.disable_gradient_checkpointing:
            self.use_reentrant_gc = False
        training_args.gradient_checkpointing = False
        training_args.gradient_checkpointing_kwargs = None

    def get_kt_config_dict(
        self,
        finetuning_args: Any,
        model_max_length: int | None,
        advanced_config: dict[str, Any] | None = None,
    ) -> dict[str, Any]:
        r"""Map LLaMA-Factory-owned training values to the public KT configuration."""
        if getattr(finetuning_args, "finetuning_type", None) != "lora":
            raise ValueError("KTransformers thin integration currently supports LoRA finetuning only.")

        kt_config = dict(advanced_config or {})
        configured_capacity = kt_config.pop("kt_model_max_length", None)
        if configured_capacity is not None:
            try:
                configured_capacity = int(configured_capacity)
            except (TypeError, ValueError) as exc:
                raise ValueError("`kt_model_max_length` must be a positive integer.") from exc
            if configured_capacity <= 0:
                raise ValueError("`kt_model_max_length` must be a positive integer.")

        kt_config.update(
            {
                "kt_lora_rank": getattr(finetuning_args, "lora_rank", None),
                "kt_lora_alpha": getattr(finetuning_args, "lora_alpha", None),
                "kt_lora_dropout": getattr(finetuning_args, "lora_dropout", None),
                "kt_weight_path": self.kt_weight_path,
                "kt_non_expert_weight_path": self.kt_non_expert_weight_path,

View on GitHub (pinned to f28afaf635)

Solutions

1. Set `finetuning_type: lora` in the training YAML to use KTransformers.
2. If full finetuning is required, remove `use_kt: true` and run the standard HF Trainer path.

Example fix

# before (yaml)
use_kt: true
finetuning_type: full

# after (yaml)
use_kt: true
finetuning_type: lora
lora_rank: 16

if cfg.get('use_kt') and cfg.get('finetuning_type') != 'lora':
    raise SystemExit('KTransformers supports finetuning_type: lora only')

def is_kt_compatible(cfg: dict) -> bool:
    return not cfg.get('use_kt') or cfg.get('finetuning_type') == 'lora'

When it happens

Trigger: Calling apply_kt_config or configure_kt_loading with use_kt: true and finetuning_type: full (or freeze/pissa) in the YAML; the check runs before any KT config keys are built.

Common situations: Users assume the AMX MoE backend accelerates full finetuning of large models and set finetuning_type: full with use_kt: true.

Related errors

• KTransformers uses LLaMA-Factory's `disable_gradient_checkpo
• KTransformers supplies its checkpoint context; remove `gradi
• Disable FSDP activation checkpointing when using KTransforme
• `kt_model_max_length` must be a positive integer.
• KTransformers accepts a single `adapter_name_or_path`.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/a4b6634ac27dd358. Report an issue: GitHub.