{"record":{"id":"1134ea5c27c3a147","repo":"can1357/oh-my-pi","slug":"gitcommanderror-git-worktree-prune-failed-incl","errorCode":null,"errorMessage":"GitCommandError: git worktree prune failed (incl. 124 timeout)","messagePattern":"GitCommandError: git worktree prune failed \\(incl\\. 124 timeout\\)","errorType":"exception","errorClass":"GitCommandError","httpStatus":null,"severity":"error","filePath":"python/robomp/src/sandbox.py","lineNumber":1218,"sourceCode":"                if repo_dir.exists():\n                    removed = _safe_run([\"git\", \"worktree\", \"remove\", \"--force\", str(repo_dir)], cwd=pool)\n                    if removed.returncode != 0:\n                        shutil.rmtree(repo_dir, ignore_errors=True)\n                        needs_prune = True\n                elif ws_root.exists():\n                    # Checkout gone but the workspace root remains -> a prior op\n                    # was killed mid-flight and may have left a dangling\n                    # registration. A fully-cleaned workspace has no ws_root, so\n                    # a plain repeat close prunes nothing.\n                    needs_prune = True\n                if needs_prune:\n                    pruned = _safe_run([\"git\", \"worktree\", \"prune\"], cwd=pool)\n                    if pruned.returncode != 0:\n                        # Prune is the step that clears the dangling registration.\n                        # If it fails (incl. a 124 timeout), report it so the\n                        # cleanup event retries instead of recording success with\n                        # stale metadata still blocking the next add.\n                        raise GitCommandError(\n                            [\"git\", \"worktree\", \"prune\"], pruned.returncode, pruned.stdout, pruned.stderr\n                        )\n            if ws_root.exists():\n                shutil.rmtree(ws_root, ignore_errors=True)\n\n    def reclaim_workspace_caches(self, *, repo: str, number: int | str) -> bool:\n        \"\"\"Strip re-creatable dependency caches from an idle workspace.\n\n        Every task run reinstalls ``node_modules`` (see\n        ``host_tools.ensure_workspace_dependencies``), so between runs the\n        checkout's ``node_modules`` and the workspace-private bun install\n        cache are dead weight — multiple GB per issue that would otherwise\n        persist until the issue closes, which is exactly how the host runs\n        out of disk. ``--continue`` resumes are unaffected: session\n        transcripts, context, artifacts and the worktree survive.\n\n        The rename pass runs under the per-repo lock (serialized against\n        ``ensure_workspace``); the slow deletes happen after the lock is","sourceCodeStart":1200,"sourceCodeEnd":1236,"githubUrl":"https://github.com/can1357/oh-my-pi/blob/969062200754ea02cfac922e5ebb8c608c079e15/python/robomp/src/sandbox.py#L1200-L1236","documentation":"When a worktree add fails or times out, the cleanup path runs `git worktree prune` in the pool to clear the dangling registration. If prune also fails (including the 120s timeout surfaced as returncode 124), a GitCommandError is raised — deliberately, so the cleanup event retries instead of recording success while stale metadata still blocks the next `git worktree add`.","triggerScenarios":"remove_workspace()/cleanup on a pool where a prior `git worktree add` was killed mid-operation and the follow-up prune fails — e.g. pool repo locked by another process, prune timing out after 120s (returncode 124), or a corrupt pool worktree metadata (.git/worktrees).","commonSituations":"Container killed (SIGKILL/OOM) during workspace provisioning leaving a partial worktree; two orchestrator instances sharing one /data volume; a hung git process on a stalled filesystem causing the 124 timeout.","solutions":["Retry the cleanup event — the error exists precisely so cleanup retries; transient locks often clear.","Check for a stale index.lock or hung git process in the pool dir and kill/remove it.","Ensure only one orchestrator runs against /data; stop duplicates.","As last resort, remove the pool clone directory and let the next event re-clone."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":"def is_prune_timeout(exc: BaseException) -> bool:\n    return isinstance(exc, GitCommandError) and exc.returncode == 124","tryCatchPattern":"try:\n    manager.remove_workspace(repo=repo, number=n)\nexcept GitCommandError as e:\n    if e.returncode == 124:\n        # inspect pool for hung git processes / locks, then re-enqueue cleanup\n        ...\n    else:\n        raise","preventionTips":["Ensure single-orchestrator ownership of /data","Gracefully stop containers instead of SIGKILL to avoid partial worktree adds","Alert on cleanup events that retry repeatedly — indicates a persistent pool lock"],"tags":["git","worktree","cleanup","timeout"],"backgroundTag":"git-command-failed","analyzedSha":"969062200754ea02cfac922e5ebb8c608c079e15","analyzedAt":"2026-08-31T10:29:35.737Z","schemaVersion":2},"datasetVersion":"2026-08-31T14:17:45.589Z"}