{"record":{"id":"69260b55db37d0a9","repo":"hiyouga/LlamaFactory","slug":"ktransformers-inference-requires-infer-backend-h","errorCode":null,"errorMessage":"KTransformers inference requires `infer_backend: huggingface`.","messagePattern":"KTransformers inference requires `infer_backend: huggingface`\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/model_args.py","lineNumber":686,"sourceCode":"            self._get_advanced_kt_config(training_args),\n        )\n        update_kt_config = getattr(training_args, \"update_kt_config\", None)\n        if not callable(update_kt_config):\n            raise RuntimeError(\n                \"The installed Transformers-KT does not provide `TrainingArguments.update_kt_config()`.\"\n            )\n\n        adapter_dir = self._resolve_kt_adapter_artifact_dir(\"training\")\n        update_kt_config(kt_config, adapter_name_or_path=adapter_dir)\n\n    def configure_kt_loading(self, finetuning_args: Any, model_max_length: int | None) -> None:\n        r\"\"\"Configure KT model loading for inference and evaluation.\"\"\"\n        if not self.use_kt:\n            if self._kt_inference_config is not None:\n                raise ValueError(\"`kt_config` requires `use_kt: true`.\")\n            return\n        if self.infer_backend != EngineName.HF:\n            raise ValueError(\"KTransformers inference requires `infer_backend: huggingface`.\")\n\n        adapter_dir = self._resolve_kt_adapter_artifact_dir(\"inference\")\n\n        try:\n            from transformers.integrations.kt import configure_kt\n        except (ImportError, ModuleNotFoundError) as exc:\n            raise RuntimeError(\"The installed Transformers-KT does not provide `configure_kt()`.\") from exc\n\n        kt_config = self.get_kt_config_dict(\n            finetuning_args,\n            model_max_length,\n            self._normalize_advanced_kt_config(self._kt_inference_config),\n        )\n        self._kt_adapter_artifact_path = adapter_dir\n        self._kt_config_handle = configure_kt(kt_config)\n\n\n@dataclass","sourceCodeStart":668,"sourceCodeEnd":704,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/model_args.py#L668-L704","documentation":"Raised by configure_kt_loading when use_kt is true and infer_backend is anything other than EngineName.HF. KT's inference path hooks into transformers' from_pretrained via transformers.integrations.kt.configure_kt; vLLM/SGLang engines bypass that hook entirely, so the combination is rejected up front.","triggerScenarios":"A chat/eval/api YAML with use_kt: true plus infer_backend: vllm (or sglang); configure_kt_loading compares infer_backend to EngineName.HF and raises.","commonSituations":"Users train with KT LoRA, then reuse the config for vLLM-served inference only changing the stage, expecting the adapter to load transparently.","solutions":["Set `infer_backend: huggingface` in the inference/eval YAML when using KTransformers.","Or export/merge the KT LoRA adapter into a standard checkpoint (`llamafactory-cli export`) and serve that with vLLM/SGLang without use_kt."],"exampleFix":"# before (yaml)\nuse_kt: true\ninfer_backend: vllm\n\n# after (yaml)\nuse_kt: true\ninfer_backend: huggingface","handlingStrategy":"validation","validationCode":"if cfg.get('use_kt') and cfg.get('infer_backend', 'huggingface') != 'huggingface':\n    raise SystemExit('KT inference requires infer_backend: huggingface')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Split training and serving configs; merge adapters before switching inference backends."],"tags":["ktransformers","vllm","sglang","inference","llamafactory"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}