{"record":{"id":"b92956e61dd7e5f0","repo":"can1357/oh-my-pi","slug":"124","errorCode":"124","errorMessage":"git timed out after {timeout:.0f}s","messagePattern":"git timed out after (.+?)s","errorType":"exception","errorClass":"GitCommandError","httpStatus":null,"severity":"error","filePath":"python/robomp/src/sandbox.py","lineNumber":390,"sourceCode":"\ndef _run(\n    cmd: list[str],\n    *,\n    cwd: Path | None = None,\n    timeout: float | None = _DEFAULT_SANDBOX_SUBPROCESS_TIMEOUT,\n) -> subprocess.CompletedProcess[str]:\n    \"\"\"Legacy raising helper (still used by a sandbox test). Forwards to subprocess.run.\"\"\"\n    try:\n        proc = subprocess.run(\n            cmd,\n            cwd=str(cwd) if cwd else None,\n            check=False,\n            capture_output=True,\n            text=True,\n            timeout=timeout,\n        )\n    except subprocess.TimeoutExpired as exc:\n        raise GitCommandError(cmd, 124, \"\", f\"git timed out after {timeout:.0f}s\") from exc\n    if proc.returncode != 0:\n        raise GitCommandError(cmd, proc.returncode, proc.stdout, proc.stderr)\n    return proc\n\n\ndef _worktree_add(add_cmd: list[str], *, pool: Path, repo_dir: Path) -> None:\n    \"\"\"Run `git worktree add`, cleaning partial state on failure.\n\n    A worktree-add killed mid-operation (the 120s `_run` timeout surfaces as\n    GitCommandError 124, or any nonzero git failure) can leave a partial\n    checkout at `repo_dir` and/or a dangling pool worktree registration. Left\n    behind, the event retry hits stale metadata and fails again on the same\n    path. Best-effort remove the checkout and prune the pool, then re-raise so the\n    retry starts from a clean path. If the prune itself fails (incl. a 124\n    timeout), raise that instead — chained from the add error — since a\n    dangling registration left behind is exactly what poisons the retry.\n    \"\"\"\n    try:","sourceCodeStart":372,"sourceCodeEnd":408,"githubUrl":"https://github.com/can1357/oh-my-pi/blob/969062200754ea02cfac922e5ebb8c608c079e15/python/robomp/src/sandbox.py#L372-L408","documentation":"_run() is the sandbox's internal wrapper around subprocess git invocations with a hard timeout. When git exceeds the timeout, subprocess.TimeoutExpired is caught and re-raised as GitCommandError with the conventional exit code 124 and the message 'git timed out after Ns'. This converts hangs (network stalls, filesystem locks) into a bounded, retryable failure instead of a wedged dispatcher thread.","triggerScenarios":"Any sandbox git operation routed through _run — including `git worktree add` via _worktree_add, and `git branch -m` — that exceeds the configured timeout because git is waiting on a network fetch, an index.lock held by another process, or a hung filesystem (NFS, full disk).","commonSituations":"git worktree add triggering an on-demand fetch from a slow or unreachable remote; a crashed process leaving index.lock in the shared pool clone; disk I/O saturation on the host running many concurrent workspaces.","solutions":["Look for a stale lock: check `<pool>/.git/index.lock` and `<pool>/.git/worktrees/*/locked`; remove only if no git process is live, then retry.","Increase the timeout if the repo is legitimately large (check the timeout parameter / settings) — 124 with slow-but-progressing ops means the limit is too tight.","Retry: the dispatcher requeues failed events, and transient network stalls usually clear.","If worktree add specifically hangs, prune stale worktree metadata first: `git worktree prune` in the pool."],"exampleFix":"# before\nmanager.ensure_workspace(...)  # GitCommandError code=124 after 120s on a huge repo\n# after\n# raise the timeout via settings/env, or pre-fetch outside the timeout path\ntransport.fetch_pool(repo=repo, pool_dir=pool)\nmanager.ensure_workspace(...)  # local-only worktree add now completes quickly","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    manager.ensure_workspace(...)\nexcept GitCommandError as e:\n    if getattr(e, \"returncode\", None) == 124:\n        schedule_retry(event, backoff=60)  # transient hang; queue will re-run\n    else:\n        mark_failed(event, e)","preventionTips":["Tune the git timeout for large repos / slow storage.","Keep pool clones on fast local disk, not network mounts.","Monitor for stale index.lock files and clean them after crashes.","Bound concurrency so pool operations don't starve each other."],"tags":["git","timeout","subprocess"],"backgroundTag":"git-command-timeout","analyzedSha":"969062200754ea02cfac922e5ebb8c608c079e15","analyzedAt":"2026-08-31T10:29:35.737Z","schemaVersion":2},"datasetVersion":"2026-08-31T14:17:45.589Z"}