huggingface/transformers · error · ValueError

save_pretrained(..., distributed_checkpoint=True) requires t

Error message

save_pretrained(..., distributed_checkpoint=True) requires the model to have been initialized with a distributed_config (_device_mesh is None).

What it means

The distributed save path needs the device mesh that transformers builds when the model is prepared with a distributed_config. If model._device_mesh is None the model bypassed distributed initialization (freshly constructed, loaded without distributed_config, or a re-wrapped copy), and the save cannot proceed — the DCP writer has no mesh to coordinate ranks over.

Source

Thrown at src/transformers/distributed/mixin.py:239

        save_directory: str | os.PathLike,
        *,
        push_to_hub: bool = False,
        save_on_this_rank: bool = True,
        repo_id: str | None = None,
        files_timestamps: dict | None = None,
        commit_message: str | None = None,
        token: str | bool | None = None,
        create_pr: bool = False,
    ) -> None:
        """Save an FSDP-wrapped model via DCP and optionally push to the Hub."""
        if not is_torch_greater_or_equal("2.7"):
            raise OSError("save_pretrained(..., distributed_checkpoint=True) requires torch>=2.7.")
        if not is_fsdp_managed_module(model_to_save):
            raise ValueError(
                "save_pretrained(..., distributed_checkpoint=True) is only supported for FSDP-wrapped models."
            )
        if getattr(model_to_save, "_device_mesh", None) is None:
            raise ValueError(
                "save_pretrained(..., distributed_checkpoint=True) requires the model to have been "
                "initialized with a distributed_config (_device_mesh is None)."
            )
        save_model_checkpoint_distributed(model_to_save, save_directory)

        if push_to_hub and save_on_this_rank:
            model_card = create_and_tag_model_card(repo_id, self.model_tags, token=token)
            model_card.save(os.path.join(save_directory, "README.md"))
            self._upload_modified_files(
                save_directory,
                repo_id,
                files_timestamps,
                commit_message=commit_message,
                token=token,
                create_pr=create_pr,
            )

    def gather_sharded_state_dict_for_save(

View on GitHub (pinned to a597f97485)

Solutions

Load the model through from_pretrained(..., distributed_config={'fsdp_size': N, 'tp_size': M}) so _device_mesh is set.
If you wrapped FSDP yourself, unwrap and re-load via the transformers distributed path before distributed save.
For ad-hoc copies, fall back to the non-distributed save on rank 0 after gathering the full state dict.

Example fix

# before
model = AutoModelForCausalLM.from_pretrained(model_id)  # _device_mesh is None
model.save_pretrained(out_dir, distributed_checkpoint=True)  # raises

# after
model = AutoModelForCausalLM.from_pretrained(model_id, distributed_config={"fsdp_size": 4})
model.save_pretrained(out_dir, distributed_checkpoint=True)

Defensive patterns

Strategy: validation

Validate before calling

def has_device_mesh(model) -> bool:
    return getattr(model, "_device_mesh", None) is not None

Type guard

def is_distributed_initialized_model(model) -> bool:
    return getattr(model, "_device_mesh", None) is not None and getattr(model, "_tp_plan", None) is not None

Try / catch

try:
    model.save_pretrained(out, distributed_checkpoint=True)
except ValueError as e:
    if "_device_mesh is None" in str(e):
        model = Model.from_pretrained(model_id, distributed_config=cfg)  # reload via distributed path
        model.save_pretrained(out, distributed_checkpoint=True)
    else:
        raise

Prevention

Never hand-wrap with torch's fully_shard if you want transformers' distributed save; use distributed_config.
Check model._device_mesh is not None before saving.
Build EMA/eval copies only after establishing the distributed save path.

When it happens

Trigger: Constructing the model via __init__ instead of from_pretrained(distributed_config=...); loading with distributed_config=None; deep-copying or re-instantiating the model after training and calling save_pretrained(..., distributed_checkpoint=True) on the copy.

Common situations: Fine-tuning scripts that build models manually; wrapping the raw model with torch's own fully_shard afterwards (which does not set _device_mesh); saving an EMA/eval copy of the model.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/d3a43da819b1656e. Report an issue: GitHub.