hiyouga/LlamaFactory · error · ValueError

`kt_cpu_activation: recompute` requires GPU gradient checkpo

Error message

`kt_cpu_activation: recompute` requires GPU gradient checkpointing. Set `disable_gradient_checkpointing: false` or use `kt_cpu_activation: retain`.

What it means

Raised by get_kt_activation_policy (model_args.py:540) when the resolved CPU policy is 'recompute' but the GPU policy is 'retain' — i.e. kt_cpu_activation: recompute while gradient checkpointing is disabled (disable_gradient_checkpointing: true). CPU recompute on backward only works if the GPU side also checkpoints, otherwise activations the CPU needs are gone; the error text tells you the two consistent resolutions.

Source

Thrown at src/llamafactory/hparams/model_args.py:540

            "kt_skip_expert_loading",
            "kt_train_mode",
            "kt_use_lora_experts",
            "kt_weight_path",
        }
    )

    def __post_init__(self) -> None:
        if self.kt_cpu_activation not in {None, "retain", "recompute"}:
            raise ValueError("`kt_cpu_activation` must be `retain` or `recompute`.")
        if not self.use_kt and self.kt_cpu_activation is not None:
            raise ValueError("`kt_cpu_activation` is only valid when `use_kt: true`.")

    def get_kt_activation_policy(self) -> dict[str, str]:
        r"""Resolve LF's GPU checkpoint switch and KT's CPU activation setting."""
        gpu_activation = "retain" if self.disable_gradient_checkpointing else "recompute"
        cpu_activation = self.kt_cpu_activation or gpu_activation
        if cpu_activation == "recompute" and gpu_activation == "retain":
            raise ValueError(
                "`kt_cpu_activation: recompute` requires GPU gradient checkpointing. "
                "Set `disable_gradient_checkpointing: false` or use `kt_cpu_activation: retain`."
            )

        return {"cpu": cpu_activation, "gpu": gpu_activation}

    @staticmethod
    def _get_accelerator_kt_config(training_args: Any) -> Any:
        accelerator_config = getattr(training_args, "accelerator_config", None)
        if isinstance(accelerator_config, dict):
            return accelerator_config.get("kt_config")
        return getattr(accelerator_config, "kt_config", None)

    def _normalize_advanced_kt_config(self, raw_config: Any) -> dict[str, Any]:
        if raw_config is None:
            return {}
        if not isinstance(raw_config, dict):
            raise TypeError("LLaMA-Factory `kt_config` must be a flat mapping.")

View on GitHub (pinned to f28afaf635)

Solutions

1. Set disable_gradient_checkpointing: false so GPU gradient checkpointing is on
2. Or switch to kt_cpu_activation: retain (or remove it) if you want to keep checkpointing disabled
3. Keep the pair in one YAML block so they are edited together

Example fix

# before
disable_gradient_checkpointing: true
kt_cpu_activation: recompute

# after
disable_gradient_checkpointing: false
kt_cpu_activation: recompute

policy = cfg.get('kt_cpu_activation')
gc_disabled = cfg.get('disable_gradient_checkpointing', False)
if policy == 'recompute' and gc_disabled:
    cfg['disable_gradient_checkpointing'] = False  # or drop the policy

def kt_policy_consistent(cfg: dict) -> bool:
    return not (cfg.get('kt_cpu_activation') == 'recompute' and cfg.get('disable_gradient_checkpointing', False))

When it happens

Trigger: kt_cpu_activation: recompute together with disable_gradient_checkpointing: true; leaving kt_cpu_activation unset while disabling gradient checkpointing is fine (policy follows GPU), but explicitly requesting recompute with checkpointing off is not.

Common situations: Users disabling gradient checkpointing to speed up step time, then asking CPU to recompute anyway; memory-tuning CPU RAM without considering the GPU-side coupling; merging configs where disable_gradient_checkpointing comes from a base file.

Related errors

• KTransformers uses LLaMA-Factory's `disable_gradient_checkpo
• KTransformers supplies its checkpoint context; remove `gradi
• `kt_cpu_activation` must be `retain` or `recompute`.
• `kt_cpu_activation` is only valid when `use_kt: true`.
• LLaMA-Factory `kt_config` must be a flat mapping.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/e7efcf6c50ca0c4f. Report an issue: GitHub.