{"record":{"id":"042410875df96a3e","repo":"Lightning-AI/pytorch-lightning","slug":"fn-ckpt-path-hpc-is-set-but-no-hpc-checkpo","errorCode":null,"errorMessage":"`.{fn}(ckpt_path=\"hpc\")` is set but no HPC checkpoint was found. Please pass an exact checkpoint path to `.{fn}(ckpt_path=...)`","messagePattern":"`\\.(.+?)\\(ckpt_path=\"hpc\"\\)` is set but no HPC checkpoint was found\\. Please pass an exact checkpoint path to `\\.(.+?)\\(ckpt_path=\\.\\.\\.\\)`","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/lightning/pytorch/trainer/connectors/checkpoint_connector.py","lineNumber":199,"sourceCode":"        elif ckpt_path == \"last\":\n            candidates = {getattr(ft, \"ckpt_path\", None) for ft in ft_checkpoints}\n            for callback in self.trainer.checkpoint_callbacks:\n                if isinstance(callback, ModelCheckpoint):\n                    candidates |= callback._find_last_checkpoints(self.trainer)\n            candidates_fs = {path: get_filesystem(path) for path in candidates if path}\n            candidates_ts = {path: fs.modified(path) for path, fs in candidates_fs.items() if fs.exists(path)}\n            if not candidates_ts:\n                # not an error so it can be set and forget before the first `fit` run\n                rank_zero_warn(\n                    f'.{fn}(ckpt_path=\"last\") is set, but there is no last checkpoint available.'\n                    \" No checkpoint will be loaded. HINT: Set `ModelCheckpoint(..., save_last=True)`.\"\n                )\n                return None\n            ckpt_path = max(candidates_ts, key=candidates_ts.get)  # type: ignore[arg-type]\n\n        elif ckpt_path == \"hpc\":\n            if not self._hpc_resume_path:\n                raise ValueError(\n                    f'`.{fn}(ckpt_path=\"hpc\")` is set but no HPC checkpoint was found.'\n                    f\" Please pass an exact checkpoint path to `.{fn}(ckpt_path=...)`\"\n                )\n            ckpt_path = self._hpc_resume_path\n\n        elif _is_registry(ckpt_path) and module_available(\"litmodels\"):\n            ckpt_path = find_model_local_ckpt_path(\n                ckpt_path,\n                default_model_registry=self.trainer._model_registry,\n                default_root_dir=self.trainer.default_root_dir,\n            )\n\n        if not ckpt_path:\n            raise ValueError(\n                f\"`.{fn}()` found no path for the best weights: {ckpt_path!r}. Please\"\n                f\" specify a path for a checkpoint `.{fn}(ckpt_path=PATH)`\"\n            )\n        return ckpt_path","sourceCodeStart":181,"sourceCodeEnd":217,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/pytorch/trainer/connectors/checkpoint_connector.py#L181-L217","documentation":"Raised by _parse_ckpt_path when ckpt_path=\"hpc\" is requested but the connector found no HPC checkpoint. HPC checkpoints are only discovered when the connector's hpc_resume_path was set (e.g., saved during preemption/interruption handling in a supported environment), so in normal runs there is nothing to resume.","triggerScenarios":"trainer.fit(ckpt_path=\"hpc\") (or validate/test/predict) when no HPC interruption checkpoint exists — i.e., the run was never preempted or the HPC save files (e.g., saved at crash time) are absent from the resume location.","commonSituations":"Reusing resume boilerplate copied from SLURM/preemption workflows on a normal run; after manually cleaning the output directory; expecting \"hpc\" to behave like \"last\".","solutions":["Pass the exact checkpoint path instead: trainer.fit(ckpt_path=\"path/to/last.ckpt\")","Use ckpt_path=\"last\" to pick the most recent non-HPC checkpoint, or \"best\" for the best one","If resuming after preemption is intended, verify the HPC checkpoint files exist and the environment (SLURM signal handling) actually triggered the save"],"exampleFix":"# before\ntrainer.fit(model, ckpt_path=\"hpc\")\n# after\ntrainer.fit(model, ckpt_path=\"lightning_logs/version_0/checkpoints/last.ckpt\")","handlingStrategy":"validation","validationCode":"if ckpt_path == \"hpc\":\n    # only valid in preemption-enabled environments; prefer explicit path\n    ckpt_path = trainer.checkpoint_connector._hpc_resume_path or \"last\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Prefer explicit paths or 'last' for normal resumes","Reserve 'hpc' for SLURM/preemption workflows where the save was triggered"],"tags":["lightning","hpc","resume","checkpoint","slurm"],"backgroundTag":"checkpoint-not-found","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}