hiyouga/LlamaFactory · critical · RuntimeError

The installed Transformers-KT does not provide `configure_kt

Error message

The installed Transformers-KT does not provide `configure_kt()`.

What it means

Raised by configure_kt_loading when `from transformers.integrations.kt import configure_kt` fails with ImportError/ModuleNotFoundError. Like error 170 this is a dependency-contract failure: the runtime transformers must be the KT-aware fork exposing the integration module, not stock transformers.

Source

Thrown at src/llamafactory/hparams/model_args.py:693

        adapter_dir = self._resolve_kt_adapter_artifact_dir("training")
        update_kt_config(kt_config, adapter_name_or_path=adapter_dir)

    def configure_kt_loading(self, finetuning_args: Any, model_max_length: int | None) -> None:
        r"""Configure KT model loading for inference and evaluation."""
        if not self.use_kt:
            if self._kt_inference_config is not None:
                raise ValueError("`kt_config` requires `use_kt: true`.")
            return
        if self.infer_backend != EngineName.HF:
            raise ValueError("KTransformers inference requires `infer_backend: huggingface`.")

        adapter_dir = self._resolve_kt_adapter_artifact_dir("inference")

        try:
            from transformers.integrations.kt import configure_kt
        except (ImportError, ModuleNotFoundError) as exc:
            raise RuntimeError("The installed Transformers-KT does not provide `configure_kt()`.") from exc

        kt_config = self.get_kt_config_dict(
            finetuning_args,
            model_max_length,
            self._normalize_advanced_kt_config(self._kt_inference_config),
        )
        self._kt_adapter_artifact_path = adapter_dir
        self._kt_config_handle = configure_kt(kt_config)


@dataclass
class ModelArguments(
    SGLangArguments,
    VllmArguments,
    KTransformersArguments,
    ExportArguments,
    ProcessorArguments,
    QuantizationArguments,

View on GitHub (pinned to f28afaf635)

Solutions

Install the KT stack in the inference environment: `pip install -U transformers-kt accelerate-kt kt-kernel`.
Verify: `python -c "from transformers.integrations.kt import configure_kt"`.
Ensure no stock `transformers` shadows `transformers-kt` on sys.path.

Example fix

# before (bash)
pip install transformers  # stock build
llamafactory-cli chat kt_chat.yaml  # RuntimeError

# after (bash)
pip install -U transformers-kt accelerate-kt kt-kernel
llamafactory-cli chat kt_chat.yaml

Defensive patterns

Strategy: validation

Validate before calling

try:
    from transformers.integrations.kt import configure_kt  # noqa: F401
except ImportError:
    raise SystemExit('transformers-kt not installed; pip install -U transformers-kt')

Try / catch

try:
    model_args.configure_kt_loading(ft_args, cutoff_len)
except RuntimeError as e:
    if 'configure_kt()' in str(e):
        raise SystemExit('install the KT transformers fork before KT inference') from e
    raise

Prevention

Use identical dependency lockfiles for KT training and KT inference images.
Add an import smoke test to container entrypoints.

When it happens

Trigger: use_kt: true at inference time with stock transformers installed, or transformers-kt missing/outdated so the integrations.kt submodule does not exist; the import inside configure_kt_loading fails and is re-raised as this RuntimeError.

Common situations: Inference environments (serving containers, eval pods) built from a lighter requirements file that omitted the KT packages used during training.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/7cd8400a6bfb866e. Report an issue: GitHub.