{"record":{"id":"1ff21180a0facfd4","repo":"ruvnet/ruflo","slug":"timed-out-acquiring-ai-budget-lock","errorCode":null,"errorMessage":"timed out acquiring ai-budget lock","messagePattern":"timed out acquiring ai-budget lock","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"v3/@claude-flow/cli/src/services/global-ai-budget.ts","lineNumber":396,"sourceCode":"      try {\n        const fd = fs.openSync(this.lockFile, fs.constants.O_CREAT | fs.constants.O_EXCL | fs.constants.O_WRONLY, 0o600);\n        fs.writeSync(fd, String(process.pid));\n        fs.closeSync(fd);\n        return () => {\n          try { fs.unlinkSync(this.lockFile); } catch { /* already gone */ }\n        };\n      } catch (e) {\n        if ((e as NodeJS.ErrnoException).code !== 'EEXIST') throw e;\n        // Stale lock from a crashed process — take over.\n        try {\n          const st = fs.lstatSync(this.lockFile);\n          if (Date.now() - st.mtimeMs > LOCK_STALE_MS) {\n            fs.unlinkSync(this.lockFile);\n            continue;\n          }\n        } catch { /* raced — retry */ }\n        if (Date.now() > deadline) {\n          throw new Error('timed out acquiring ai-budget lock');\n        }\n        await delay(25);\n      }\n    }\n  }\n\n  /** Read + prune the ledger. Caller must hold the lock for read-modify-write. */\n  private readLedger(now: number): Ledger {\n    assertNotSymlink(this.ledgerFile);\n    let ledger: Ledger = { version: 1, launches: [], active: [] };\n    if (fs.existsSync(this.ledgerFile)) {\n      try {\n        const raw = JSON.parse(fs.readFileSync(this.ledgerFile, 'utf-8'));\n        if (raw && typeof raw === 'object') {\n          ledger = {\n            version: 1,\n            launches: Array.isArray(raw.launches) ? raw.launches.filter((l: LaunchRecord) => typeof l?.at === 'number') : [],\n            active: Array.isArray(raw.active) ? raw.active.filter((a: ActiveRecord) => typeof a?.at === 'number') : [],","sourceCodeStart":378,"sourceCodeEnd":414,"githubUrl":"https://github.com/ruvnet/ruflo/blob/fa13ee4ad60ac2090b1480656eb233521790d640/v3/@claude-flow/cli/src/services/global-ai-budget.ts#L378-L414","documentation":"GlobalAiBudget serializes ledger mutations with an O_EXCL lock (`~/.claude-flow/ai-budget.lock`); a lock older than 10s (LOCK_STALE_MS) is considered abandoned and taken over. Because the budget is user-global, every daemon on the machine contends for this single lock — when a fresh lock persists past the acquire deadline (25ms poll loop), this timeout is thrown.","triggerScenarios":"Many ruflo daemons/worktrees requesting AI launches at nearly the same moment (the lock is user-wide, not per-workspace), or a process that crashed immediately after creating the lock, leaving it younger than the 10s staleness window.","commonSituations":"Several worktree daemons whose worker schedules align; a burst of parallel `claude --print` launches; a machine recovering from a crash with leftover locks from dead processes.","solutions":["Retry the launch — contention over a 10s-stale lock resolves itself within seconds","Stagger daemon worker schedules (cron jitter) so budget checks do not all fire in the same instant","If it persists, verify no process holds the lock (`lsof ~/.claude-flow/ai-budget.lock`), then delete it once the holder is gone","Reduce the number of concurrently scheduled AI workers per user"],"exampleFix":"// before: single attempt during a burst of daemon launches\nconst permit = await budget.acquire(request);\n\n// after: short backoff retry, since the lock is machine-global and short-lived\nfor (let i = 0; i < 5; i++) {\n  try { permit = await budget.acquire(request); break; }\n  catch (e) {\n    if (e?.message !== 'timed out acquiring ai-budget lock') throw e;\n    await new Promise((r) => setTimeout(r, 2000 * (i + 1)));\n  }\n}","handlingStrategy":"retry","validationCode":"import { existsSync, lstatSync } from 'node:fs';\nimport { join } from 'node:path';\nimport { homedir } from 'node:os';\n\nconst LOCK_STALE_MS = 10_000;\n\nfunction aiBudgetLockStatus(): 'free' | 'stale' | 'held' {\n  const lock = join(homedir(), '.claude-flow', 'ai-budget.lock');\n  if (!existsSync(lock)) return 'free';\n  return Date.now() - lstatSync(lock).mtimeMs > LOCK_STALE_MS ? 'stale' : 'held';\n}","typeGuard":null,"tryCatchPattern":"for (let attempt = 1; attempt <= 5; attempt++) {\n  try {\n    permit = await budget.acquire(request);\n    break;\n  } catch (e) {\n    if (e?.message !== 'timed out acquiring ai-budget lock') throw e;\n    // lock is machine-global but short-lived; back off and retry\n    await new Promise((r) => setTimeout(r, attempt * 2_000));\n  }\n}","preventionTips":["Stagger daemon worker schedules across worktrees so budget checks don't align","Remember the lock is user-global: N daemons contend for ONE file","Keep launches of `claude --print` bundled rather than fanned out simultaneously","A stale lock self-heals after 10s — prefer waiting over manual deletion"],"tags":["concurrency","file-lock","timeout","ai-budget"],"backgroundTag":"file-lock-timeout","analyzedSha":"fa13ee4ad60ac2090b1480656eb233521790d640","analyzedAt":"2026-08-18T21:34:22.708Z","contentChangedAt":"2026-08-18T21:34:22.708Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}