Lightning-AI/pytorch-lightning · error · RuntimeError

Found more than one stateful callback of type `{type(callbac

Error message

Found more than one stateful callback of type `{type(callback).__name__}`. In the current configuration, this callback does not support being saved alongside other instances of the same type. Please consult the documentation of `{type(callback).__name__}` regarding valid settings for the callback state to be checkpointable. HINT: The `callback.state_key` must be unique among all callbacks in the Trainer.

What it means

Raised by _validate_callbacks_list at Trainer init when two or more callbacks override state_dict (are stateful) and share the same state_key (class name plus distinguishing init arguments). Because checkpointing stores callback state keyed by state_key, duplicates would overwrite each other and cannot be restored unambiguously.

Source

Thrown at src/lightning/pytorch/trainer/connectors/callback_connector.py:273

        checkpoint_callbacks: list[Callback] = []

        for cb in callbacks:
            if isinstance(cb, (BatchSizeFinder, LearningRateFinder)):
                tuner_callbacks.append(cb)
            elif isinstance(cb, Checkpoint):
                checkpoint_callbacks.append(cb)
            else:
                other_callbacks.append(cb)

        return tuner_callbacks + other_callbacks + checkpoint_callbacks


def _validate_callbacks_list(callbacks: list[Callback]) -> None:
    stateful_callbacks = [cb for cb in callbacks if is_overridden("state_dict", instance=cb, parent=Callback)]
    seen_callbacks = set()
    for callback in stateful_callbacks:
        if callback.state_key in seen_callbacks:
            raise RuntimeError(
                f"Found more than one stateful callback of type `{type(callback).__name__}`. In the current"
                " configuration, this callback does not support being saved alongside other instances of the same type."
                f" Please consult the documentation of `{type(callback).__name__}` regarding valid settings for"
                " the callback state to be checkpointable."
                " HINT: The `callback.state_key` must be unique among all callbacks in the Trainer."
            )
        seen_callbacks.add(callback.state_key)

View on GitHub (pinned to 9fed5c27d2)

Solutions

  1. Give each instance distinct constructor arguments so state_key differs, e.g., MyCallback(name="a") vs MyCallback(name="b")
  2. Keep only one stateful callback of that type
  3. Implement state_dict on only one of the callbacks, or design the callback to aggregate multiple roles internally

Example fix

# before
trainer = Trainer(callbacks=[MyCallback(lr=1.0), MyCallback(lr=1.0)])
# after
trainer = Trainer(callbacks=[MyCallback(lr=1.0), MyCallback(lr=0.5)])  # distinct init args -> unique state_key
Defensive patterns

Strategy: validation

Validate before calling

from lightning.pytorch.callbacks import Callback
from lightning.pytorch.utilities import is_overridden

def validate_state_keys(callbacks):
    seen = set()
    for cb in callbacks:
        if is_overridden("state_dict", instance=cb, parent=Callback):
            key = cb.state_key
            assert key not in seen, f"duplicate state_key: {key}"
            seen.add(key)

validate_state_keys(my_callbacks)
trainer = Trainer(callbacks=my_callbacks)

Prevention

When it happens

Trigger: Adding two instances of a custom stateful callback (or a stateful ModelCheckpoint-like callback) with identical constructor arguments, e.g., callbacks=[MyCallback(lr=1), MyCallback(lr=1)]; the state_key only differs when init args differ.

Common situations: Ensembling or multi-experiment setups that add several same-config callbacks; copying a callback instance list; adding two built-in ModelCheckpoint callbacks with identical settings.

Related errors


AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/1d63f1070e644b43. Report an issue: GitHub.