{"record":{"id":"ff77d8db3d049ac0","repo":"hiyouga/LlamaFactory","slug":"ktransformers-only-supports-lora-finetuning","errorCode":null,"errorMessage":"KTransformers only supports LoRA finetuning.","messagePattern":"KTransformers only supports LoRA finetuning\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/model/adapter.py","lineNumber":278,"sourceCode":"                \"target_modules\": target_modules,\n                \"lora_alpha\": finetuning_args.lora_alpha,\n                \"lora_dropout\": finetuning_args.lora_dropout,\n                \"use_rslora\": finetuning_args.use_rslora,\n                \"use_dora\": finetuning_args.use_dora,\n                \"modules_to_save\": finetuning_args.additional_target,\n            }\n        elif finetuning_args.finetuning_type == \"oft\":\n            peft_kwargs = {\n                \"r\": finetuning_args.oft_rank,\n                \"oft_block_size\": finetuning_args.oft_block_size,\n                \"target_modules\": target_modules,\n                \"module_dropout\": finetuning_args.module_dropout,\n                \"modules_to_save\": finetuning_args.additional_target,\n            }\n\n        if model_args.use_kt:\n            if finetuning_args.finetuning_type != \"lora\":\n                raise ValueError(\"KTransformers only supports LoRA finetuning.\")\n\n            peft_config = LoraConfig(task_type=TaskType.CAUSAL_LM, inference_mode=False, **peft_kwargs)\n            model = get_peft_model(model, peft_config, autocast_adapter_dtype=cast_trainable_params_to_fp32)\n        elif model_args.use_unsloth:\n            if finetuning_args.finetuning_type == \"oft\":\n                raise ValueError(\"Unsloth is currently not supported for OFT.\")\n\n            model = get_unsloth_peft_model(model, model_args, peft_kwargs)\n        else:\n            if finetuning_args.pissa_init:\n                if finetuning_args.pissa_iter == -1:\n                    logger.info_rank0(\"Using PiSSA initialization.\")\n                    peft_kwargs[\"init_lora_weights\"] = \"pissa\"\n                else:\n                    logger.info_rank0(f\"Using PiSSA initialization with FSVD steps {finetuning_args.pissa_iter}.\")\n                    peft_kwargs[\"init_lora_weights\"] = f\"pissa_niter_{finetuning_args.pissa_iter}\"\n\n            if finetuning_args.finetuning_type == \"lora\":","sourceCodeStart":260,"sourceCodeEnd":296,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/model/adapter.py#L260-L296","documentation":"Raised in _setup_lora_tuning when use_kt is true but finetuning_type is not lora. The KTransformers integration only wraps peft's LoraConfig; full, freeze and OFT paths have no KT implementation.","triggerScenarios":"Config with enable_kt: true together with finetuning_type: full, freeze or oft.","commonSituations":"Turning on KTransformers in an existing full/freeze tuning config to speed up large-model training.","solutions":["Set finetuning_type: lora when enable_kt: true.","Disable KTransformers (remove enable_kt) if you must do full or freeze tuning."],"exampleFix":"# before\nenable_kt: true\nfinetuning_type: full\n\n# after\nenable_kt: true\nfinetuning_type: lora","handlingStrategy":"validation","validationCode":"if model_args.get(\"enable_kt\"):\n    assert finetuning_args.get(\"finetuning_type\") == \"lora\", \\\n        \"KTransformers requires finetuning_type: lora\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Treat enable_kt as implying lora in config templates and linters.","Do not port full/freeze configs to KT without switching the method."],"tags":["ktransformers","lora","finetuning-type"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}