Lightning-AI/pytorch-lightning · error · ValueError

The path {str(path)!r} does not point to a valid full checkp

Error message

The path {str(path)!r} does not point to a valid full checkpoint. Make sure the path points to a directory with a full XLAFSDP checkpoint.

What it means

When state_dict_type='full', load_checkpoint expects path to be the consolidated single checkpoint file (not the shard directory). The source checks path.is_file(); if the path is a directory of shards (or nonexistent), the full-checkpoint load cannot proceed. The message text about 'a directory with a full checkpoint' is slightly misleading — the path must be a file.

Source

Thrown at src/lightning/fabric/strategies/xla_fsdp.py:596

            for key in requested_metadata_keys:
                if key in loaded_metadata_keys:
                    state[key] = sharded_ckpt[key]
                    loaded_metadata_keys.remove(key)

            metadata = {}
            if len(loaded_metadata_keys):
                for key in loaded_metadata_keys:
                    metadata[key] = sharded_ckpt[key]

            # remove "shard_metadata" that is loaded in
            if "shard_metadata" in metadata:
                metadata.pop("shard_metadata")

            return metadata

        if self._state_dict_type == "full":
            if not path.is_file():
                raise ValueError(
                    f"The path {str(path)!r} does not point to a valid full checkpoint. Make sure the path points to a"
                    " directory with a full XLAFSDP checkpoint."
                )
            if len(optimizers) > 0 or len(state.keys() - modules.keys() - optimizers.keys()) > 0:
                rank_zero_warn(
                    "Loading a full checkpoint will only load the full model."
                    " The optimizer and any additional metadata are not included."
                )
            if len(modules) > 0:
                raise ValueError(
                    "Found a XLAFSDP model in the provided checkpoint state."
                    " Please provide the model without any XLAFSDP wrapper."
                )
            if "model" not in state or not isinstance(model := state["model"], torch.nn.Module):
                raise NotImplementedError("XLAFSDP only supports a single model instance with 'model' as the key.")
            full_ckpt = torch.load(path, weights_only=weights_only)
            model.load_state_dict(full_ckpt.pop("model"), strict=strict)
            return full_ckpt

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Point path at the consolidated full checkpoint file (e.g. produced by consolidate_sharded_ckpts)
2. If you only have shards, either load with state_dict_type='sharded' or run torch_xla's consolidate_sharded_ckpts to build the full file first
3. Verify the file exists before calling load_checkpoint

Example fix

# before
strategy = XLAFSDPStrategy(state_dict_type='full')
fabric.load_checkpoint('ckpt/shards/', state={'model': model})

# after
strategy = XLAFSDPStrategy(state_dict_type='full')
fabric.load_checkpoint('ckpt/consolidated.ckpt', state={'model': model})

from pathlib import Path
if strategy_state_dict_type == 'full':
    assert Path(ckpt).is_file(), 'full load requires the consolidated .ckpt file path, not the shard dir'

When it happens

Trigger: Passing the sharded checkpoint directory (produced by state_dict_type='sharded') while the strategy is configured with state_dict_type='full'; passing a typo'd or nonexistent file path.

Common situations: Switching strategy config from sharded to full without re-consolidating the checkpoint; resume scripts that always pass the save directory; mixing conventions from other Lightning strategies where checkpoints are files.

Related errors

• The path {str(file)!r} does not point to valid sharded check
• Found a XLAFSDP model in the provided checkpoint state. Plea
• XLAFSDP only supports a single model instance with 'model' a
• Got `XLAFSDPStrategy.load_checkpoint(..., state={state!r})`
• Loading a single module or optimizer object from a checkpoin

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/a138bbd95930b426. Report an issue: GitHub.