hiyouga/LlamaFactory · error · ValueError

KTransformers uses LLaMA-Factory's `disable_gradient_checkpo

Error message

KTransformers uses LLaMA-Factory's `disable_gradient_checkpointing`; remove `gradient_checkpointing: true`.

What it means

Raised by KTransformersArguments.configure_kt_checkpointing when use_kt is enabled and training_args.gradient_checkpointing is truthy. KTransformers manages its own checkpointing context and treats LLaMA-Factory's disable_gradient_checkpointing flag as the single control knob, so any explicit gradient_checkpointing: true in the config is rejected to avoid two checkpoint owners.

Source

Thrown at src/llamafactory/hparams/model_args.py:585

        raw_config = getattr(training_args, "kt_config", None)
        accelerator_config = self._get_accelerator_kt_config(training_args)
        if raw_config is None:
            if accelerator_config is not None:
                raise ValueError(
                    "Put KTransformers settings in the LLaMA-Factory training YAML `kt_config`; "
                    "remove `kt_config` from the Accelerate config."
                )
            return {}
        if accelerator_config is not None and accelerator_config != raw_config:
            raise ValueError("LLaMA-Factory YAML and Accelerate config cannot define different KT settings.")
        return self._normalize_advanced_kt_config(raw_config)

    def configure_kt_checkpointing(self, training_args: Any) -> None:
        r"""Keep LLaMA-Factory as the single gradient-checkpointing entry point."""
        if self.use_unsloth or self.use_unsloth_gc:
            raise ValueError("KTransformers cannot be combined with Unsloth checkpoint wrapping.")
        if getattr(training_args, "gradient_checkpointing", False):
            raise ValueError(
                "KTransformers uses LLaMA-Factory's `disable_gradient_checkpointing`; "
                "remove `gradient_checkpointing: true`."
            )
        if getattr(training_args, "gradient_checkpointing_kwargs", None) is not None:
            raise ValueError("KTransformers supplies its checkpoint context; remove `gradient_checkpointing_kwargs`.")

        fsdp_config = getattr(training_args, "fsdp_config", None)
        if isinstance(fsdp_config, dict) and fsdp_config.get("activation_checkpointing"):
            raise ValueError("Disable FSDP activation checkpointing when using KTransformers.")
        if os.environ.get("FSDP_ACTIVATION_CHECKPOINTING", "false").lower() in {"1", "true", "yes"}:
            raise ValueError("Disable FSDP activation checkpointing when using KTransformers.")

        self.get_kt_activation_policy()
        if not self.disable_gradient_checkpointing:
            self.use_reentrant_gc = False
        training_args.gradient_checkpointing = False
        training_args.gradient_checkpointing_kwargs = None

View on GitHub (pinned to f28afaf635)

Solutions

1. Remove `gradient_checkpointing: true` from the training YAML / CLI args when `use_kt: true` is set.
2. Control checkpointing only through `disable_gradient_checkpointing` (false = KT recompute on, true = retain).
3. Re-run training after confirming no other config layer (Accelerate YAML, CLI overrides) re-adds the flag.

Example fix

# before (yaml)
cutoff_len: 4096
use_kt: true
gradient_checkpointing: true

# after (yaml)
cutoff_len: 4096
use_kt: true
disable_gradient_checkpointing: false

# before launching
if cfg.get('use_kt') and cfg.get('gradient_checkpointing'):
    raise SystemExit('remove gradient_checkpointing for KT runs')

When it happens

Trigger: Running llamafactory-cli train with use_kt: true plus gradient_checkpointing: true in the same YAML, or passing --gradient_checkpointing on the CLI; apply_kt_config -> configure_kt_checkpointing checks the flag before KT setup.

Common situations: Users copy a standard LLaMA-Factory LoRA SFT config (which commonly sets gradient_checkpointing: true to save VRAM) and add use_kt: true on top; the copied flag then aborts startup.

Related errors

• KTransformers supplies its checkpoint context; remove `gradi
• Disable FSDP activation checkpointing when using KTransforme
• KTransformers thin integration currently supports LoRA finet
• `kt_model_max_length` must be a positive integer.
• `kt_config` requires `use_kt: true`.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/ba3a71e0a808bee9. Report an issue: GitHub.