{"record":{"id":"e7cc06dfaad2eb6f","repo":"hiyouga/LlamaFactory","slug":"ktransformers-cannot-be-combined-with-unsloth-chec","errorCode":null,"errorMessage":"KTransformers cannot be combined with Unsloth checkpoint wrapping.","messagePattern":"KTransformers cannot be combined with Unsloth checkpoint wrapping\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/model_args.py","lineNumber":583,"sourceCode":"\n    def _get_advanced_kt_config(self, training_args: Any) -> dict[str, Any]:\n        raw_config = getattr(training_args, \"kt_config\", None)\n        accelerator_config = self._get_accelerator_kt_config(training_args)\n        if raw_config is None:\n            if accelerator_config is not None:\n                raise ValueError(\n                    \"Put KTransformers settings in the LLaMA-Factory training YAML `kt_config`; \"\n                    \"remove `kt_config` from the Accelerate config.\"\n                )\n            return {}\n        if accelerator_config is not None and accelerator_config != raw_config:\n            raise ValueError(\"LLaMA-Factory YAML and Accelerate config cannot define different KT settings.\")\n        return self._normalize_advanced_kt_config(raw_config)\n\n    def configure_kt_checkpointing(self, training_args: Any) -> None:\n        r\"\"\"Keep LLaMA-Factory as the single gradient-checkpointing entry point.\"\"\"\n        if self.use_unsloth or self.use_unsloth_gc:\n            raise ValueError(\"KTransformers cannot be combined with Unsloth checkpoint wrapping.\")\n        if getattr(training_args, \"gradient_checkpointing\", False):\n            raise ValueError(\n                \"KTransformers uses LLaMA-Factory's `disable_gradient_checkpointing`; \"\n                \"remove `gradient_checkpointing: true`.\"\n            )\n        if getattr(training_args, \"gradient_checkpointing_kwargs\", None) is not None:\n            raise ValueError(\"KTransformers supplies its checkpoint context; remove `gradient_checkpointing_kwargs`.\")\n\n        fsdp_config = getattr(training_args, \"fsdp_config\", None)\n        if isinstance(fsdp_config, dict) and fsdp_config.get(\"activation_checkpointing\"):\n            raise ValueError(\"Disable FSDP activation checkpointing when using KTransformers.\")\n        if os.environ.get(\"FSDP_ACTIVATION_CHECKPOINTING\", \"false\").lower() in {\"1\", \"true\", \"yes\"}:\n            raise ValueError(\"Disable FSDP activation checkpointing when using KTransformers.\")\n\n        self.get_kt_activation_policy()\n        if not self.disable_gradient_checkpointing:\n            self.use_reentrant_gc = False\n        training_args.gradient_checkpointing = False","sourceCodeStart":565,"sourceCodeEnd":601,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/model_args.py#L565-L601","documentation":"Raised by configure_kt_checkpointing (model_args.py:583) when use_kt is combined with use_unsloth or use_unsloth_gc. Unsloth wraps gradient checkpointing with its own context manager, and KTransformers requires LlamaFactory's own checkpoint path be the single entry point; the two wrappers would nest incorrectly, so the combination is rejected before training starts.","triggerScenarios":"A config with use_kt: true and use_unsloth: true (a common single-GPU memory-saving combo attempt); or use_kt with use_unsloth_gc: true to try Unsloth's faster checkpointing alongside CPU-offloaded experts.","commonSituations":"Users stacking every memory optimization flag they know; migrating an Unsloth recipe to KTransformers without removing the old flags; copy-pasting example configs from different backends into one file.","solutions":["Remove use_unsloth and use_unsloth_gc from the config when using KTransformers","If you prefer Unsloth, drop use_kt and the kt_* options instead","Remember KT handles checkpointing via disable_gradient_checkpointing / kt_cpu_activation only"],"exampleFix":"# before\nuse_kt: true\nuse_unsloth: true\n\n# after\nuse_kt: true\n# use_unsloth removed","handlingStrategy":"validation","validationCode":"if cfg.get('use_kt'):\n    assert not cfg.get('use_unsloth') and not cfg.get('use_unsloth_gc'), 'KT and Unsloth are mutually exclusive'","typeGuard":"def kt_unsloth_compatible(cfg: dict) -> bool:\n    return not (cfg.get('use_kt') and (cfg.get('use_unsloth') or cfg.get('use_unsloth_gc')))","tryCatchPattern":null,"preventionTips":["Pick one memory-optimization backend per experiment; do not stack flags","When migrating recipes between backends, delete the old backend's flags entirely"],"tags":["ktransformers","unsloth","incompatible-backends"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}