hiyouga/LlamaFactory · error · ValueError

KTransformers cannot be combined with Unsloth checkpoint wra

Error message

KTransformers cannot be combined with Unsloth checkpoint wrapping.

What it means

Raised by configure_kt_checkpointing (model_args.py:583) when use_kt is combined with use_unsloth or use_unsloth_gc. Unsloth wraps gradient checkpointing with its own context manager, and KTransformers requires LlamaFactory's own checkpoint path be the single entry point; the two wrappers would nest incorrectly, so the combination is rejected before training starts.

Source

Thrown at src/llamafactory/hparams/model_args.py:583

    def _get_advanced_kt_config(self, training_args: Any) -> dict[str, Any]:
        raw_config = getattr(training_args, "kt_config", None)
        accelerator_config = self._get_accelerator_kt_config(training_args)
        if raw_config is None:
            if accelerator_config is not None:
                raise ValueError(
                    "Put KTransformers settings in the LLaMA-Factory training YAML `kt_config`; "
                    "remove `kt_config` from the Accelerate config."
                )
            return {}
        if accelerator_config is not None and accelerator_config != raw_config:
            raise ValueError("LLaMA-Factory YAML and Accelerate config cannot define different KT settings.")
        return self._normalize_advanced_kt_config(raw_config)

    def configure_kt_checkpointing(self, training_args: Any) -> None:
        r"""Keep LLaMA-Factory as the single gradient-checkpointing entry point."""
        if self.use_unsloth or self.use_unsloth_gc:
            raise ValueError("KTransformers cannot be combined with Unsloth checkpoint wrapping.")
        if getattr(training_args, "gradient_checkpointing", False):
            raise ValueError(
                "KTransformers uses LLaMA-Factory's `disable_gradient_checkpointing`; "
                "remove `gradient_checkpointing: true`."
            )
        if getattr(training_args, "gradient_checkpointing_kwargs", None) is not None:
            raise ValueError("KTransformers supplies its checkpoint context; remove `gradient_checkpointing_kwargs`.")

        fsdp_config = getattr(training_args, "fsdp_config", None)
        if isinstance(fsdp_config, dict) and fsdp_config.get("activation_checkpointing"):
            raise ValueError("Disable FSDP activation checkpointing when using KTransformers.")
        if os.environ.get("FSDP_ACTIVATION_CHECKPOINTING", "false").lower() in {"1", "true", "yes"}:
            raise ValueError("Disable FSDP activation checkpointing when using KTransformers.")

        self.get_kt_activation_policy()
        if not self.disable_gradient_checkpointing:
            self.use_reentrant_gc = False
        training_args.gradient_checkpointing = False

View on GitHub (pinned to f28afaf635)

Solutions

1. Remove use_unsloth and use_unsloth_gc from the config when using KTransformers
2. If you prefer Unsloth, drop use_kt and the kt_* options instead
3. Remember KT handles checkpointing via disable_gradient_checkpointing / kt_cpu_activation only

Example fix

# before
use_kt: true
use_unsloth: true

# after
use_kt: true
# use_unsloth removed

if cfg.get('use_kt'):
    assert not cfg.get('use_unsloth') and not cfg.get('use_unsloth_gc'), 'KT and Unsloth are mutually exclusive'

def kt_unsloth_compatible(cfg: dict) -> bool:
    return not (cfg.get('use_kt') and (cfg.get('use_unsloth') or cfg.get('use_unsloth_gc')))

When it happens

Trigger: A config with use_kt: true and use_unsloth: true (a common single-GPU memory-saving combo attempt); or use_kt with use_unsloth_gc: true to try Unsloth's faster checkpointing alongside CPU-offloaded experts.

Common situations: Users stacking every memory optimization flag they know; migrating an Unsloth recipe to KTransformers without removing the old flags; copy-pasting example configs from different backends into one file.

Related errors

• `kt_cpu_activation` must be `retain` or `recompute`.
• `kt_cpu_activation` is only valid when `use_kt: true`.
• `kt_cpu_activation: recompute` requires GPU gradient checkpo
• LLaMA-Factory `kt_config` must be a flat mapping.
• These `kt_config` values are derived from LLaMA-Factory argu

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/e7cc06dfaad2eb6f. Report an issue: GitHub.