Lightning-AI/pytorch-lightning · error · ValueError

The path {str(path)!r} does not point to a valid checkpoint.

Error message

The path {str(path)!r} does not point to a valid checkpoint. Make sure the path points to either a directory with FSDP checkpoint shards, or a single file with a full checkpoint.

What it means

After attempting both the distributed-checkpoint path (a directory of shards) and the single-file full checkpoint path, the loader found neither a valid __1_0.distcp-style shard directory nor a loadable single file. The path exists but its contents match neither layout, or the file is corrupt/not a checkpoint.

Source

Thrown at src/lightning/fabric/strategies/fsdp.py:654

                with _get_full_state_dict_context(module, world_size=self.world_size, rank0_only=False):
                    temp_state_dict = _rekey_optimizer_state_if_needed(checkpoint.pop(optim_key), module)
                    optim_state_dict = FSDP.optim_state_dict_to_load(
                        optim_state_dict=temp_state_dict,
                        model=module,
                        optim=optim,
                    )
                    optim.load_state_dict(optim_state_dict)

            requested_metadata_keys = state.keys() - modules.keys() - optimizers.keys()
            _validate_keys_for_strict_loading(requested_metadata_keys, checkpoint.keys(), strict=strict)

            # Load metadata (anything not a module or optimizer)
            _move_state_into(source=checkpoint, destination=state, keys=requested_metadata_keys)

            # return the remaining metadata that wasn't requested as part of `state`
            return checkpoint

        raise ValueError(
            f"The path {str(path)!r} does not point to a valid checkpoint. Make sure the path points to either a"
            " directory with FSDP checkpoint shards, or a single file with a full checkpoint."
        )

    @classmethod
    @override
    def register_strategies(cls, strategy_registry: _StrategyRegistry) -> None:
        if not torch.distributed.is_available():
            return

        strategy_registry.register(
            "fsdp",
            cls,
            description="Fully Sharded Data Parallel (FSDP) training",
        )
        strategy_registry.register(
            "fsdp_cpu_offload",
            cls,

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Verify the target: a valid sharded checkpoint is a directory containing .distcp shard files plus metadata; a full checkpoint is one file loadable with torch.load
2. If the file is a plain state_dict saved with torch.save, load it manually with torch.load and module.load_state_dict
3. Re-download or re-save the checkpoint and confirm file sizes/shard count match world size
4. Check for path typos and that all ranks resolve the same path

Example fix

# before
strategy.load_checkpoint('ckpt.dir', state={'model': model})  # empty dir
# after
import os
assert os.path.isdir('ckpt.dir') and any(f.endswith('.distcp') for f in os.listdir('ckpt.dir'))
strategy.load_checkpoint('ckpt.dir', state={'model': model})

import os, torch

def valid_ckpt(p):
    if os.path.isdir(p):
        return any(f.endswith('.distcp') or f == '.metadata' for f in os.listdir(p))
    if os.path.isfile(p):
        try:
            torch.load(p, map_location='meta', weights_only=False)
            return True
        except Exception:
            return False
    return False

assert valid_ckpt(path), f'{path} is not a valid FSDP checkpoint'

def is_fsdp_sharded_dir(p: str) -> bool:
    import os
    return os.path.isdir(p) and any(f.endswith('.distcp') for f in os.listdir(p))

try:
    strategy.load_checkpoint(path, state=state)
except ValueError as e:
    if 'does not point to a valid checkpoint' in str(e):
        state_dict = torch.load(path, map_location='cpu')
        model.load_state_dict(state_dict['model'])

When it happens

Trigger: Passing a path that is an empty or wrong-content directory, a .ckpt saved by a different mechanism (e.g. torch.save of a raw state_dict), a truncated download, or a file that torch.load cannot read.

Common situations: Downloading checkpoints with interrupted transfers; pointing at a directory that only contains .metadata or partial shards; using a full-state-dict file with the sharded loader path; wrong path string (typos, missing rank subfolder).

Related errors

• Found multiple FSDP models in the given state. Loading check
• Found multiple FSDP models in the given state. Saving checkp
• Got FSDPStrategy.load_checkpoint(..., state={state!r}) but a
• Could not find a FSDP model in the provided checkpoint state
• Got `XLAFSDPStrategy.load_checkpoint(..., state={state!r})`

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/9cea74478706e84e. Report an issue: GitHub.