Lightning-AI/pytorch-lightning · error · TypeError

`Trainer.save_checkpoint(..., storage_options=...)` with `st

Error message

`Trainer.save_checkpoint(..., storage_options=...)` with `storage_options` arg is not supported for `{self.__class__.__name__}` as `CheckpointIO` is not used.

What it means

DeepSpeedStrategy saves checkpoints through DeepSpeed's own pipeline (a `CheckpointIO` plugin is not used), so there is nowhere to hand per-storage backend options. Passing `storage_options` (designed for fsspec-based checkpoint plugins like Async or filesystem-specific IO) raises TypeError.

Source

Thrown at src/lightning/pytorch/strategies/deepspeed.py:652

    @override
    def save_checkpoint(self, checkpoint: dict, filepath: _PATH, storage_options: Optional[Any] = None) -> None:
        """Save model/training states as a checkpoint file through state-dump and file-write.

        Args:
            checkpoint: The checkpoint state dictionary
            filepath: write-target file's path
            storage_options: not used for ``DeepSpeedStrategy`` as ``CheckpointIO`` is not used

        Raises:
            TypeError:
                If ``storage_options`` arg is passed in

        """
        # broadcast the filepath from rank 0 to ensure all the states are saved in a common filepath
        filepath = self.broadcast(filepath)

        if storage_options is not None:
            raise TypeError(
                "`Trainer.save_checkpoint(..., storage_options=...)` with `storage_options` arg"
                f" is not supported for `{self.__class__.__name__}` as `CheckpointIO` is not used."
            )

        if self.zero_stage_3 and self._multi_device and self.is_global_zero:
            warning_cache.warn(
                "When saving the DeepSpeed Stage 3 checkpoint, "
                "each worker will save a shard of the checkpoint within a directory. "
                "If a single file is required after training, "
                "see https://lightning.ai/docs/pytorch/stable/advanced/model_parallel.html#"
                "deepspeed-zero-stage-3-single-file for instructions."
            )
        # Use deepspeed's internal checkpointing function to handle partitioned weights across processes
        # dump states as a checkpoint dictionary object
        _exclude_keys = ["state_dict", "optimizer_states"]
        checkpoint = {k: v for k, v in checkpoint.items() if k not in _exclude_keys}
        self.deepspeed_engine.save_checkpoint(
            filepath,

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Drop `storage_options` when using DeepSpeedStrategy
2. If you need remote/custom storage, copy the checkpoint after saving, or use a strategy that supports CheckpointIO
3. Branch on strategy type before calling save_checkpoint

Example fix

# before
trainer.save_checkpoint(ckpt, storage_options={"fs": s3fs})  # DeepSpeedStrategy

# after
trainer.save_checkpoint(ckpt)
# upload ckpt dir to remote storage afterwards if needed

from lightning.pytorch.strategies import DeepSpeedStrategy
if isinstance(trainer.strategy, DeepSpeedStrategy):
    trainer.save_checkpoint(path)  # no storage_options
else:
    trainer.save_checkpoint(path, storage_options=opts)

def supports_storage_options(trainer) -> bool:
    return getattr(trainer.strategy, "checkpoint_io", None) is not None

When it happens

Trigger: `trainer.save_checkpoint(path, storage_options={...})` (or `strategy.save_checkpoint(..., storage_options=...)`) while the strategy is DeepSpeedStrategy.

Common situations: Code shared across strategies that always passes storage_options (e.g. for saving to S3 via the TorchX/Async plugin); switching a working FSDP run to DeepSpeed without removing the kwarg.

Related errors

• `FSDPStrategy.save_checkpoint(..., storage_options=...)` is
• checkpoint_io set through both strategy class and plugins, c
• No models were set up for backward. Did you forget to call `
• When using multiple models + deepspeed, please provide the m
• The `{type(self._strategy).__name__}` requires the model and

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/ec4cecf5ac5b218c. Report an issue: GitHub.