{"record":{"id":"698bd812a432635f","repo":"Lightning-AI/pytorch-lightning","slug":"the-launcher-can-only-create-subprocesses-once-698bd8","errorCode":null,"errorMessage":"The launcher can only create subprocesses once.","messagePattern":"The launcher can only create subprocesses once\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"src/lightning/pytorch/strategies/launchers/subprocess_script.py","lineNumber":153,"sourceCode":"\n            hydra_in_use = False\n            cwd: Optional[str] = None\n            if _HYDRA_AVAILABLE:\n                from hydra.core.hydra_config import HydraConfig\n\n                hydra_in_use = HydraConfig.initialized()\n\n            if hydra_in_use:\n                command, cwd = _hydra_subprocess_cmd(local_rank)\n            else:\n                command = _basic_subprocess_cmd()\n\n            new_process = subprocess.Popen(command, env=env_copy, cwd=cwd)\n            self.procs.append(new_process)\n\n    def _check_can_spawn_children(self) -> None:\n        if len(self.procs) > 0:\n            raise RuntimeError(\"The launcher can only create subprocesses once.\")\n        if self.cluster_environment.local_rank() != 0:\n            raise RuntimeError(\n                \"Lightning attempted to launch new distributed processes with `local_rank > 0`. This should not happen.\"\n                \" Possible reasons: 1) LOCAL_RANK environment variable was incorrectly modified by the user,\"\n                \" 2) `ClusterEnvironment.creates_processes_externally` incorrectly implemented.\"\n            )\n","sourceCodeStart":135,"sourceCodeEnd":160,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/pytorch/strategies/launchers/subprocess_script.py#L135-L160","documentation":"_SubprocessScriptLauncher spawns all distributed worker processes once, from rank 0, by re-executing the user's script. It records spawned processes in self.procs and refuses to spawn again if that list is non-empty, because the launcher is designed for a single launch per run.","triggerScenarios":"Calling launcher.launch() twice on the same _SubprocessScriptLauncher instance, e.g. invoking fit() and then test()/validate() in a way that re-enters the launcher within one process, with self.procs already populated.","commonSituations":"Running fit() followed by a second distributed entry point in the same script with a strategy whose launcher state persisted; custom code that manually invokes the launcher's launch more than once.","solutions":["Ensure the launcher only launches once: subsequent phases (test/validate/predict) should reuse existing processes or a new launcher instance is created by a fresh Trainer/strategy","Restructure the script so all distributed work happens within one launch closure","Check you are not manually calling strategy.launcher.launch in custom code"],"exampleFix":"# before\nlauncher.launch(fit_fn)\nlauncher.launch(test_fn)  # RuntimeError\n\n# after\nlauncher.launch(lambda: (fit_fn(), test_fn()))  # single launch for all distributed work","handlingStrategy":"validation","validationCode":"assert not getattr(launcher, \"procs\", None), \"launcher already spawned once\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Do all distributed work inside a single launch closure","Create a new strategy/Trainer (and thus launcher) for any second distributed phase"],"tags":["subprocess","launcher","distributed","pytorch-lightning"],"backgroundTag":"launcher-single-use-violation","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}