{"record":{"id":"73a3537bbc8fc450","repo":"Lightning-AI/pytorch-lightning","slug":"remote-fsspec-distributed-checkpoints-require-t","errorCode":null,"errorMessage":"Remote (fsspec) distributed checkpoints require `torch.distributed.checkpoint._fsspec_filesystem`, which is not available in this PyTorch build. Use a local checkpoint path or upgrade PyTorch.","messagePattern":"Remote \\(fsspec\\) distributed checkpoints require `torch\\.distributed\\.checkpoint\\._fsspec_filesystem`, which is not available in this PyTorch build\\. Use a local checkpoint path or upgrade PyTorch\\.","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"src/lightning/fabric/utilities/cloud_io.py","lineNumber":282,"sourceCode":"    if _is_local_file_protocol(str(path)):\n        from torch.distributed.checkpoint import FileSystemReader\n\n        return FileSystemReader(path=path)\n    FsspecReader = _import_fsspec_dcp_filesystem(\"FsspecReader\")\n    return FsspecReader(path=str(path))\n\n\ndef _import_fsspec_dcp_filesystem(name: str) -> Any:\n    \"\"\"Import ``FsspecReader``/``FsspecWriter`` from torch's private DCP fsspec module.\n\n    These live in a private module that not every PyTorch build ships, so raise an actionable error\n    instead of letting a bare ``ImportError`` surface from deep in the call stack.\n\n    \"\"\"\n    try:\n        module = importlib.import_module(\"torch.distributed.checkpoint._fsspec_filesystem\")\n    except ImportError as e:\n        raise ImportError(\n            \"Remote (fsspec) distributed checkpoints require\"\n            \" `torch.distributed.checkpoint._fsspec_filesystem`, which is not available in this\"\n            \" PyTorch build. Use a local checkpoint path or upgrade PyTorch.\"\n        ) from e\n    return getattr(module, name)\n","sourceCodeStart":264,"sourceCodeEnd":288,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/fabric/utilities/cloud_io.py#L264-L288","documentation":"Raised by _import_fsspec_dcp_filesystem when importing torch.distributed.checkpoint._fsspec_filesystem fails; this private module is needed to write/read torch.distributed.checkpoint (DCP) shards directly to remote (fsspec) filesystems like s3:// or gs://. Builds/versions of PyTorch lacking it cannot do remote DCP I/O, so Lightning converts the deep ImportError into an actionable message: use a local path or upgrade PyTorch.","triggerScenarios":"Calling save_checkpoint/load_checkpoint with a remote URL (s3://bucket/..., gs://...) on a distributed checkpoint path while the installed PyTorch does not expose torch.distributed.checkpoint._fsspec_filesystem (older PyTorch versions, some nightly/modified builds).","commonSituations":"Migrating distributed checkpoints from local disk to S3/GCS with an older PyTorch; environment pinning PyTorch < 2.x/older 2.x; slim wheels or forks where the private module was removed/renamed.","solutions":["Save distributed checkpoints to a local path first, then upload (aws s3 cp / gsutil cp) to remote storage","Upgrade PyTorch to a version that includes torch.distributed.checkpoint._fsspec_filesystem support for remote filesystems","Verify with: python -c \"import torch.distributed.checkpoint._fsspec_filesystem\" before attempting remote DCP saves"],"exampleFix":"# before\nfabric.save_checkpoint('s3://bucket/run/ckpt', state=state)  # ImportError on old torch\n\n# after\nfabric.save_checkpoint('/tmp/run/ckpt', state=state)\nsubprocess.run(['aws', 's3', 'cp', '--recursive', '/tmp/run/ckpt', 's3://bucket/run/ckpt'])","handlingStrategy":"fallback","validationCode":"import importlib.util\nremote_ok = importlib.util.find_spec('torch.distributed.checkpoint._fsspec_filesystem') is not None\nif not remote_ok:\n    ckpt_path = local_staging_dir  # save locally, then upload","typeGuard":null,"tryCatchPattern":"try:\n    fabric.save_checkpoint('s3://bucket/ckpt', state=state)\nexcept ImportError:\n    fabric.save_checkpoint('/tmp/ckpt', state=state)\n    upload_to_s3('/tmp/ckpt')","preventionTips":["Check for the private DCP fsspec module before adopting remote DCP paths","Keep a local-staging + upload fallback in checkpoint utilities"],"tags":["distributed-checkpoint","fsspec","remote-storage","import-error","pytorch-version","lightning-fabric"],"backgroundTag":"missing-optional-dependency","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}