hiyouga/LlamaFactory · error · ValueError

`kt_config` requires `use_kt: true`.

Error message

`kt_config` requires `use_kt: true`.

What it means

Raised by configure_kt_loading when _kt_inference_config is populated but use_kt is false. The inference-side kt_config is captured only during KT-enabled parsing, so finding it without use_kt means the config was partially applied — LLaMA-Factory refuses the inconsistent state instead of silently ignoring KT settings.

Source

Thrown at src/llamafactory/hparams/model_args.py:683

        kt_config = self.get_kt_config_dict(
            finetuning_args,
            model_max_length,
            self._get_advanced_kt_config(training_args),
        )
        update_kt_config = getattr(training_args, "update_kt_config", None)
        if not callable(update_kt_config):
            raise RuntimeError(
                "The installed Transformers-KT does not provide `TrainingArguments.update_kt_config()`."
            )

        adapter_dir = self._resolve_kt_adapter_artifact_dir("training")
        update_kt_config(kt_config, adapter_name_or_path=adapter_dir)

    def configure_kt_loading(self, finetuning_args: Any, model_max_length: int | None) -> None:
        r"""Configure KT model loading for inference and evaluation."""
        if not self.use_kt:
            if self._kt_inference_config is not None:
                raise ValueError("`kt_config` requires `use_kt: true`.")
            return
        if self.infer_backend != EngineName.HF:
            raise ValueError("KTransformers inference requires `infer_backend: huggingface`.")

        adapter_dir = self._resolve_kt_adapter_artifact_dir("inference")

        try:
            from transformers.integrations.kt import configure_kt
        except (ImportError, ModuleNotFoundError) as exc:
            raise RuntimeError("The installed Transformers-KT does not provide `configure_kt()`.") from exc

        kt_config = self.get_kt_config_dict(
            finetuning_args,
            model_max_length,
            self._normalize_advanced_kt_config(self._kt_inference_config),
        )
        self._kt_adapter_artifact_path = adapter_dir
        self._kt_config_handle = configure_kt(kt_config)

View on GitHub (pinned to f28afaf635)

Solutions

1. Set `use_kt: true` in the YAML that also defines `kt_config`.
2. Or fully remove the `kt_config` block if KTransformers is not intended for this inference run.

Example fix

# before (yaml)
infer_backend: huggingface
kt_config:
  kt_model_max_length: 8192

# after (yaml)
use_kt: true
infer_backend: huggingface
kt_config:
  kt_model_max_length: 8192

if cfg.get('kt_config') and not cfg.get('use_kt'):
    raise SystemExit('kt_config requires use_kt: true')

When it happens

Trigger: An inference/eval entry point (chat, api, eval) builds ModelArguments with kt_config settings applied (e.g. via a path that records the inference config) while the effective use_kt resolves to false, hitting the branch at the top of configure_kt_loading.

Common situations: Configs where use_kt was toggled off (or a template merged without it) but a leftover kt_config block still reaches inference argument handling.

Related errors

• KTransformers uses LLaMA-Factory's `disable_gradient_checkpo
• KTransformers supplies its checkpoint context; remove `gradi
• Disable FSDP activation checkpointing when using KTransforme
• KTransformers thin integration currently supports LoRA finet
• `kt_model_max_length` must be a positive integer.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/0886c8e886a80a13. Report an issue: GitHub.