{"record":{"id":"325adff8ecbc573d","repo":"hiyouga/LlamaFactory","slug":"self-class-name-does-not-support-gradien","errorCode":null,"errorMessage":"{self.__class__.__name__} does not support gradient checkpointing.","messagePattern":"(.+?) does not support gradient checkpointing\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/model/model_utils/checkpointing.py","lineNumber":135,"sourceCode":"        else:\n            return func(*args, **kwargs)\n\n    return custom_gradient_checkpointing_func\n\n\ndef _gradient_checkpointing_enable(\n    self: \"PreTrainedModel\",\n    gradient_checkpointing_kwargs: Optional[dict[str, Any]] = None,\n    use_unsloth_gc: bool = False,\n) -> None:\n    r\"\"\"Activates gradient checkpointing for the current model.\n\n    Modification of the original method to enable gradient checkpointing for block-wise optimizer.\n    \"\"\"\n    from torch.utils.checkpoint import checkpoint\n\n    if not self.supports_gradient_checkpointing:\n        raise ValueError(f\"{self.__class__.__name__} does not support gradient checkpointing.\")\n\n    if gradient_checkpointing_kwargs is None:\n        gradient_checkpointing_kwargs = {\"use_reentrant\": True}\n\n    if use_unsloth_gc:\n        gradient_checkpointing_func = get_unsloth_gradient_checkpointing_func()\n    else:\n        gradient_checkpointing_func = partial(checkpoint, **gradient_checkpointing_kwargs)\n\n    gradient_checkpointing_func = get_custom_gradient_checkpointing_func(gradient_checkpointing_func)\n    if \"value\" in inspect.signature(self._set_gradient_checkpointing).parameters:  # old GC format\n        self.apply(partial(self._set_gradient_checkpointing, value=True))\n        self.enable_input_require_grads()\n        logger.warning_rank0_once(\"You are using the old GC format, some features (e.g. BAdam) will be invalid.\")\n    else:  # have already enabled input require gradients\n        self._set_gradient_checkpointing(enable=True, gradient_checkpointing_func=gradient_checkpointing_func)\n\n","sourceCodeStart":117,"sourceCodeEnd":153,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/model/model_utils/checkpointing.py#L117-L153","documentation":"LlamaFactory monkey-patches transformers' _gradient_checkpointing_enable (checkpointing.py:135) to support custom checkpointing funcs and block-wise optimizers. Before enabling, it checks self.supports_gradient_checkpointing on the PreTrainedModel; if the model class declares no gradient-checkpointing support, it raises ValueError. This mirrors transformers' own behavior but surfaces during patching/enable when the training args request gradient_checkpointing=true.","triggerScenarios":"Running training with gradient_checkpointing: true on a model class whose supports_gradient_checkpointing attribute is False (custom models, some multimodal wrappers, models converted without the flag). Called from _enable_gradient_checkpointing during trainer/model setup.","commonSituations":"Fine-tuning a custom or newly added model architecture that forgot to set supports_gradient_checkpointing = True; loading a community model whose modeling file does not declare GC support; using a wrapper model class not registered as GC-capable.","solutions":["Disable gradient checkpointing in the training config (gradient_checkpointing: false) — usually only viable if VRAM is sufficient.","If it is your own model class, set supports_gradient_checkpointing = True on the PreTrainedModel subclass and ensure layers use checkpoint-able modules.","Check whether a newer transformers/LlamaFactory version adds GC support for that model_type.","Switch to a supported model variant or reduce batch/sequence length to fit without checkpointing."],"exampleFix":"# before (yaml)\ngradient_checkpointing: true   # custom model without GC support -> ValueError\n\n# after (yaml)\ngradient_checkpointing: false\nper_device_train_batch_size: 1","handlingStrategy":"validation","validationCode":"model = AutoModelForCausalLM.from_pretrained(name, trust_remote_code=True)\nif training_args.gradient_checkpointing:\n    assert getattr(model, \"supports_gradient_checkpointing\", False), (\n        f\"{model.__class__.__name__} lacks GC support; disable gradient_checkpointing\"\n    )","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check supports_gradient_checkpointing right after loading, before trainer setup.","For custom model classes, declare supports_gradient_checkpointing = True and test checkpointing explicitly."],"tags":["gradient-checkpointing","vram","custom-model","training-config"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}