{"record":{"id":"d64a1df0c0294de1","repo":"Lightning-AI/pytorch-lightning","slug":"calling-trainer-fit-twice-on-the-same-trainer-d64a1d","errorCode":null,"errorMessage":"Calling `trainer.fit()` twice on the same Trainer instance using a spawn-based strategy is not supported. You can work around this limitation by creating a new Trainer instance and passing the `fit(ckpt_path=...)` argument.","messagePattern":"Calling `trainer\\.fit\\(\\)` twice on the same Trainer instance using a spawn-based strategy is not supported\\. You can work around this limitation by creating a new Trainer instance and passing the `fit\\(ckpt_path=\\.\\.\\.\\)` argument\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"src/lightning/pytorch/strategies/launchers/xla.py","lineNumber":80,"sourceCode":"\n    @override\n    def launch(self, function: Callable, *args: Any, trainer: Optional[\"pl.Trainer\"] = None, **kwargs: Any) -> Any:\n        \"\"\"Launches processes that run the given function in parallel.\n\n        The function is allowed to have a return value. However, when all processes join, only the return value\n        of worker process 0 gets returned from this `launch` method in the main process.\n\n        Arguments:\n            function: The entry point for all launched processes.\n            *args: Optional positional arguments to be passed to the given function.\n            trainer: Optional reference to the :class:`~lightning.pytorch.trainer.trainer.Trainer` for which\n                a selected set of attributes get restored in the main process after processes join.\n            **kwargs: Optional keyword arguments to be passed to the given function.\n\n        \"\"\"\n        if self._already_fit and trainer is not None and trainer.state.fn == TrainerFn.FITTING:\n            # resolving https://github.com/Lightning-AI/pytorch-lightning/issues/18775 will lift this restriction\n            raise NotImplementedError(\n                \"Calling `trainer.fit()` twice on the same Trainer instance using a spawn-based strategy is not\"\n                \" supported. You can work around this by creating a new Trainer instance and passing the\"\n                \" `fit(ckpt_path=...)` argument.\"\n            )\n\n        # pjrt requires that the queue is serializable\n        return_queue = mp.Manager().Queue()\n\n        import torch_xla.distributed.xla_multiprocessing as xmp\n\n        spawn_kwargs = {}\n        nprocs = self._strategy.num_processes\n        if nprocs == 1:\n            # avoid warning: \"Unsupported nprocs\". If it's 1, it will call the launched function directly.\n            # otherwise it will use all devices\n            spawn_kwargs[\"nprocs\"] = nprocs\n\n        process_context = xmp.spawn(","sourceCodeStart":62,"sourceCodeEnd":98,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/pytorch/strategies/launchers/xla.py#L62-L98","documentation":"Same restriction as the multiprocessing spawn launcher: the XLA launcher (fork/spawn based) cannot relaunch a second fit() on the same Trainer instance, because trainer state cannot be safely restored across repeated XLA multiprocess launches (issue #18775).","triggerScenarios":"Calling trainer.fit() a second time with trainer.state.fn == FITTING while the strategy uses the XLA launcher and _already_fit is True (e.g. XLAStrategy on TPU).","commonSituations":"Fine-tuning or continuing training on TPU with the same Trainer; two-stage training scripts calling fit twice.","solutions":["Create a new Trainer instance and pass ckpt_path to resume","Combine both stages into one fit() call","Track Lightning-AI/pytorch-lightning#18775"],"exampleFix":"# before\ntrainer.fit(model)\ntrainer.fit(model, ckpt_path=\"last.ckpt\")  # NotImplementedError\n\n# after\ntrainer.fit(model)\nnew_trainer = L.Trainer(max_epochs=10)\nnew_trainer.fit(model, ckpt_path=\"last.ckpt\")","handlingStrategy":"fallback","validationCode":"if getattr(trainer.strategy.launcher, \"_already_fit\", False):\n    trainer = L.Trainer(max_epochs=..., plugins=...)  # new trainer, resume via ckpt_path","typeGuard":null,"tryCatchPattern":null,"preventionTips":["One Trainer per fit() for XLA/TPU runs","Resume with fit(ckpt_path=...) on the fresh Trainer"],"tags":["xla","tpu","fit-twice","trainer"],"backgroundTag":"trainer-reuse-not-supported","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}