Lightning-AI/pytorch-lightning · error · TypeError

`DeepSpeedStrategy.save_checkpoint(..., storage_options=...)

Error message

`DeepSpeedStrategy.save_checkpoint(..., storage_options=...)` is not supported because `DeepSpeedStrategy` does not use the `CheckpointIO`.

What it means

DeepSpeedStrategy serializes checkpoints itself through the DeepSpeedEngine, bypassing the CheckpointIO plugin that would interpret storage_options. Passing a non-None storage_options to save_checkpoint therefore raises TypeError.

Source

Thrown at src/lightning/fabric/strategies/deepspeed.py:431

        """Save model, optimizer, and other state in a checkpoint directory.

        Args:
            path: A path to where the files should be saved
            state: A dictionary with contents to be saved. If the dict contains modules or optimizers, their
                state-dict will be retrieved and converted automatically.
            storage_options: Unused by this strategy, since it doesn't use a ``CheckpointIO`` plugin.
            filter: Unsupported.

        Raises:
            TypeError:
                If the unused ``storage_options`` gets passed.
            ValueError:
                When no :class:`deepspeed.DeepSpeedEngine` objects were found in the state, or when multiple
                :class:`deepspeed.DeepSpeedEngine` objects were found.

        """
        if storage_options is not None:
            raise TypeError(
                "`DeepSpeedStrategy.save_checkpoint(..., storage_options=...)` is not supported because"
                " `DeepSpeedStrategy` does not use the `CheckpointIO`."
            )
        if filter is not None:
            raise TypeError(
                "`DeepSpeedStrategy.save_checkpoint(..., filter=...)` is not supported because"
                " `DeepSpeedStrategy` manages the state serialization internally."
            )

        engines = _get_deepspeed_engines_from_state(state)
        if len(engines) == 0:
            raise ValueError(
                "Could not find a DeepSpeed model in the provided checkpoint state. Please provide the model as"
                " part of the state like so: `save_checkpoint(..., state={'model': model, ...})`. Make sure"
                " you set up the model (and optimizers if any) through the strategy before saving the checkpoint."
            )
        if len(engines) > 1:
            raise ValueError(

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Drop the storage_options argument when saving under DeepSpeed
2. Branch your checkpoint helper: pass storage_options only for strategies that use CheckpointIO
3. For deepspeed-specific needs, configure checkpointing via DeepSpeed config (e.g. checkpoint options in the deepspeed json)

Example fix

# before
fabric.save_checkpoint(state, path, storage_options={"write_in_parallel": True})
# after (deepspeed)
fabric.save_checkpoint(state, path)

from lightning.fabric.strategies.deepspeed import DeepSpeedStrategy
save_kwargs = {} if isinstance(fabric.strategy, DeepSpeedStrategy) else {"storage_options": opts}
fabric.save_checkpoint(path, state, **save_kwargs)

from lightning.fabric.strategies.deepspeed import DeepSpeedStrategy

def supports_storage_options(strategy) -> bool:
    return not isinstance(strategy, DeepSpeedStrategy)

try:
    fabric.save_checkpoint(path, state, storage_options=opts)
except TypeError:
    fabric.save_checkpoint(path, state)

When it happens

Trigger: fabric.save_checkpoint(path, state, storage_options={...}) while strategy='deepspeed'; e.g. options meant for fsspec/AsyncCheckpointIO like 'write_in_parallel'.

Common situations: Shared checkpointing helper code that always forwards storage_options regardless of strategy; migrating from FSDP (where storage_options work) to DeepSpeed.

Related errors

• `DeepSpeedStrategy.save_checkpoint(..., filter=...)` is not
• Could not find a DeepSpeed model in the provided checkpoint
• Found multiple DeepSpeed engine modules in the given state.
• Got DeepSpeedStrategy.load_checkpoint(..., state={state!r})
• Could not find a DeepSpeed model in the provided checkpoint

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/42c6f0ca34d44598. Report an issue: GitHub.