hiyouga/LlamaFactory · error · ValueError

Disable FSDP activation checkpointing when using KTransforme

Error message

Disable FSDP activation checkpointing when using KTransformers.

What it means

Raised by configure_kt_checkpointing when training_args.fsdp_config is a dict whose activation_checkpointing entry is truthy. KTransformers provides its own activation policy (cpu/gpu retain vs recompute) and cannot share activation checkpointing duty with FSDP's wrapper-level scheme.

Source

Thrown at src/llamafactory/hparams/model_args.py:594

        if accelerator_config is not None and accelerator_config != raw_config:
            raise ValueError("LLaMA-Factory YAML and Accelerate config cannot define different KT settings.")
        return self._normalize_advanced_kt_config(raw_config)

    def configure_kt_checkpointing(self, training_args: Any) -> None:
        r"""Keep LLaMA-Factory as the single gradient-checkpointing entry point."""
        if self.use_unsloth or self.use_unsloth_gc:
            raise ValueError("KTransformers cannot be combined with Unsloth checkpoint wrapping.")
        if getattr(training_args, "gradient_checkpointing", False):
            raise ValueError(
                "KTransformers uses LLaMA-Factory's `disable_gradient_checkpointing`; "
                "remove `gradient_checkpointing: true`."
            )
        if getattr(training_args, "gradient_checkpointing_kwargs", None) is not None:
            raise ValueError("KTransformers supplies its checkpoint context; remove `gradient_checkpointing_kwargs`.")

        fsdp_config = getattr(training_args, "fsdp_config", None)
        if isinstance(fsdp_config, dict) and fsdp_config.get("activation_checkpointing"):
            raise ValueError("Disable FSDP activation checkpointing when using KTransformers.")
        if os.environ.get("FSDP_ACTIVATION_CHECKPOINTING", "false").lower() in {"1", "true", "yes"}:
            raise ValueError("Disable FSDP activation checkpointing when using KTransformers.")

        self.get_kt_activation_policy()
        if not self.disable_gradient_checkpointing:
            self.use_reentrant_gc = False
        training_args.gradient_checkpointing = False
        training_args.gradient_checkpointing_kwargs = None

    def get_kt_config_dict(
        self,
        finetuning_args: Any,
        model_max_length: int | None,
        advanced_config: dict[str, Any] | None = None,
    ) -> dict[str, Any]:
        r"""Map LLaMA-Factory-owned training values to the public KT configuration."""
        if getattr(finetuning_args, "finetuning_type", None) != "lora":
            raise ValueError("KTransformers thin integration currently supports LoRA finetuning only.")

View on GitHub (pinned to f28afaf635)

Solutions

1. Set `activation_checkpointing: false` or remove the key from `fsdp_config` in the training YAML.
2. Use LLaMA-Factory's `disable_gradient_checkpointing` / `kt_cpu_activation` to control KT activation memory instead.

Example fix

# before (yaml)
fsdp_config:
  fsdp_offload_params: true
  activation_checkpointing: true
use_kt: true

# after (yaml)
fsdp_config:
  fsdp_offload_params: true
use_kt: true

fsdp = cfg.get('fsdp_config') or {}
if cfg.get('use_kt') and isinstance(fsdp, dict) and fsdp.get('activation_checkpointing'):
    raise SystemExit('disable fsdp activation_checkpointing for KT')

When it happens

Trigger: Launching KT training with an fsdp_config YAML section containing activation_checkpointing: true (typical of FSDP full-sharding configs) while use_kt: true.

Common situations: Users on multi-node FSDP templates enable activation checkpointing to fit large MoE models, then switch to the KT AMX backend without pruning the fsdp_config block.

Related errors

• KTransformers uses LLaMA-Factory's `disable_gradient_checkpo
• KTransformers supplies its checkpoint context; remove `gradi
• KTransformers thin integration currently supports LoRA finet
• `kt_model_max_length` must be a positive integer.
• `kt_config` requires `use_kt: true`.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/8c492bf850e91445. Report an issue: GitHub.