huggingface/transformers · error · OSError

save_pretrained(..., distributed_checkpoint=True) requires t

Error message

save_pretrained(..., distributed_checkpoint=True) requires torch>=2.7.

What it means

The distributed-checkpoint save path (DCP + HuggingFaceStorageWriter) used by save_pretrained(..., distributed_checkpoint=True) relies on APIs that only exist in torch 2.7 and newer. On older torch the save raises OSError before touching the disk, so no partial checkpoint is written.

Source

Thrown at src/transformers/distributed/mixin.py:233

            save_on_this_rank = save_on_this_rank and _get_torch_distributed_rank() == 0
        return save_on_this_rank

    def save_distributed_checkpoint(
        self,
        model_to_save,
        save_directory: str | os.PathLike,
        *,
        push_to_hub: bool = False,
        save_on_this_rank: bool = True,
        repo_id: str | None = None,
        files_timestamps: dict | None = None,
        commit_message: str | None = None,
        token: str | bool | None = None,
        create_pr: bool = False,
    ) -> None:
        """Save an FSDP-wrapped model via DCP and optionally push to the Hub."""
        if not is_torch_greater_or_equal("2.7"):
            raise OSError("save_pretrained(..., distributed_checkpoint=True) requires torch>=2.7.")
        if not is_fsdp_managed_module(model_to_save):
            raise ValueError(
                "save_pretrained(..., distributed_checkpoint=True) is only supported for FSDP-wrapped models."
            )
        if getattr(model_to_save, "_device_mesh", None) is None:
            raise ValueError(
                "save_pretrained(..., distributed_checkpoint=True) requires the model to have been "
                "initialized with a distributed_config (_device_mesh is None)."
            )
        save_model_checkpoint_distributed(model_to_save, save_directory)

        if push_to_hub and save_on_this_rank:
            model_card = create_and_tag_model_card(repo_id, self.model_tags, token=token)
            model_card.save(os.path.join(save_directory, "README.md"))
            self._upload_modified_files(
                save_directory,
                repo_id,
                files_timestamps,

View on GitHub (pinned to a597f97485)

Solutions

1. Upgrade torch: pip install -U 'torch>=2.7'.
2. If torch cannot be upgraded, save with the regular gathered path (distributed_checkpoint=False) which full-state-dict gathers to rank 0.
3. Pin transformers/torch together in requirements to avoid silent version drift.

Example fix

# before
torch==2.6.0  # requirements.txt, save_pretrained(..., distributed_checkpoint=True) raises OSError

# after
torch>=2.7.0  # requirements.txt

from transformers.utils import is_torch_greater_or_equal

def assert_torch_for_distributed_save() -> None:
    if not is_torch_greater_or_equal("2.7"):
        raise RuntimeError("distributed_checkpoint=True needs torch>=2.7; upgrade or save gathered instead")

try:
    model.save_pretrained(out, distributed_checkpoint=True)
except OSError as e:
    if "requires torch>=2.7" in str(e):
        model.save_pretrained(out)  # fallback: gathered, rank-0 save
    else:
        raise

When it happens

Trigger: Calling Model.save_pretrained(..., distributed_checkpoint=True) (or the mixin's save_pretrained_distributed) in an environment where importlib metadata reports torch < 2.7.

Common situations: Training environments pinned to torch 2.4-2.6 (common for older CUDA images); CI images lagging behind; upgrading transformers without upgrading torch.

Related errors

• FSDP2 requires `torch>=2.7` (distributed checkpoint save/loa
• Distributed checkpointing requires `torch>=2.7`.
• save_pretrained(..., distributed_checkpoint=True) is only su
• Saving an FSDP-wrapped model requires torch.distributed to b
• We tried to initialize torch.distributed for you, but it fai

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/f67c612a8afca0aa. Report an issue: GitHub.