Lightning-AI/pytorch-lightning · error · TypeError

`DeepSpeedStrategy.save_checkpoint(..., filter=...)` is not

Error message

`DeepSpeedStrategy.save_checkpoint(..., filter=...)` is not supported because `DeepSpeedStrategy` manages the state serialization internally.

What it means

DeepSpeed's engine serializes the entire training state itself, so a custom filter callback for selecting/filtering what gets saved cannot be applied. Passing filter=... to DeepSpeedStrategy.save_checkpoint raises TypeError.

Source

Thrown at src/lightning/fabric/strategies/deepspeed.py:436

                state-dict will be retrieved and converted automatically.
            storage_options: Unused by this strategy, since it doesn't use a ``CheckpointIO`` plugin.
            filter: Unsupported.

        Raises:
            TypeError:
                If the unused ``storage_options`` gets passed.
            ValueError:
                When no :class:`deepspeed.DeepSpeedEngine` objects were found in the state, or when multiple
                :class:`deepspeed.DeepSpeedEngine` objects were found.

        """
        if storage_options is not None:
            raise TypeError(
                "`DeepSpeedStrategy.save_checkpoint(..., storage_options=...)` is not supported because"
                " `DeepSpeedStrategy` does not use the `CheckpointIO`."
            )
        if filter is not None:
            raise TypeError(
                "`DeepSpeedStrategy.save_checkpoint(..., filter=...)` is not supported because"
                " `DeepSpeedStrategy` manages the state serialization internally."
            )

        engines = _get_deepspeed_engines_from_state(state)
        if len(engines) == 0:
            raise ValueError(
                "Could not find a DeepSpeed model in the provided checkpoint state. Please provide the model as"
                " part of the state like so: `save_checkpoint(..., state={'model': model, ...})`. Make sure"
                " you set up the model (and optimizers if any) through the strategy before saving the checkpoint."
            )
        if len(engines) > 1:
            raise ValueError(
                "Found multiple DeepSpeed engine modules in the given state. Saving checkpoints with DeepSpeed is"
                " currently limited to a single model per checkpoint. To save multiple models, call the"
                " save method for each model separately with a different path."
            )
        engine = engines[0]

View on GitHub (pinned to 9fed5c27d2)

Solutions

Remove the filter argument under DeepSpeed and rely on what the engine saves
If you need a reduced checkpoint, save the raw deepspeed checkpoint and post-process it, or select state contents before calling save_checkpoint
Gate filter usage on strategy type in your save helper

Example fix

# before
fabric.save_checkpoint(path, state, filter=lambda k, v: k == "model")
# after (deepspeed)
fabric.save_checkpoint(path, state)

Defensive patterns

Strategy: validation

Validate before calling

from lightning.fabric.strategies.deepspeed import DeepSpeedStrategy
if isinstance(fabric.strategy, DeepSpeedStrategy):
    fabric.save_checkpoint(path, state)  # no filter
else:
    fabric.save_checkpoint(path, state, filter=filter_fn)

Type guard

from lightning.fabric.strategies.deepspeed import DeepSpeedStrategy

def supports_checkpoint_filter(strategy) -> bool:
    return not isinstance(strategy, DeepSpeedStrategy)

Try / catch

try:
    fabric.save_checkpoint(path, state, filter=fn)
except TypeError:
    fabric.save_checkpoint(path, state)

Prevention

Apply filtering before building state instead of via the filter kwarg under deepspeed

When it happens

Trigger: fabric.save_checkpoint(path, state, filter=fn) with strategy='deepspeed', where filter is a callable transforming the state dict in other strategies.

Common situations: Checkpoint-filtering helpers (e.g. saving only the model, stripping EMA/optimizer state) reused across strategies; FSDP/DDP code migrated to DeepSpeed.

Related errors

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/085aeb638ca36345. Report an issue: GitHub.