{"record":{"id":"1ff21180a0facfd4","repo":"ruvnet/ruflo","slug":"timed-out-acquiring-ai-budget-lock","errorCode":null,"errorMessage":"timed out acquiring ai-budget lock","messagePattern":"timed out acquiring ai-budget lock","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"v3/@claude-flow/cli/src/services/global-ai-budget.ts","lineNumber":396,"sourceCode":"      try {\n        const fd = fs.openSync(this.lockFile, fs.constants.O_CREAT | fs.constants.O_EXCL | fs.constants.O_WRONLY, 0o600);\n        fs.writeSync(fd, String(process.pid));\n        fs.closeSync(fd);\n        return () => {\n          try { fs.unlinkSync(this.lockFile); } catch { /* already gone */ }\n        };\n      } catch (e) {\n        if ((e as NodeJS.ErrnoException).code !== 'EEXIST') throw e;\n        // Stale lock from a crashed process — take over.\n        try {\n          const st = fs.lstatSync(this.lockFile);\n          if (Date.now() - st.mtimeMs > LOCK_STALE_MS) {\n            fs.unlinkSync(this.lockFile);\n            continue;\n          }\n        } catch { /* raced — retry */ }\n        if (Date.now() > deadline) {\n          throw new Error('timed out acquiring ai-budget lock');\n        }\n        await delay(25);\n      }\n    }\n  }\n\n  /** Read + prune the ledger. Caller must hold the lock for read-modify-write. */\n  private readLedger(now: number): Ledger {\n    assertNotSymlink(this.ledgerFile);\n    let ledger: Ledger = { version: 1, launches: [], active: [] };\n    if (fs.existsSync(this.ledgerFile)) {\n      try {\n        const raw = JSON.parse(fs.readFileSync(this.ledgerFile, 'utf-8'));\n        if (raw && typeof raw === 'object') {\n          ledger = {\n            version: 1,\n            launches: Array.isArray(raw.launches) ? raw.launches.filter((l: LaunchRecord) => typeof l?.at === 'number') : [],\n            active: Array.isArray(raw.active) ? raw.active.filter((a: ActiveRecord) => typeof a?.at === 'number') : [],","sourceCodeStart":378,"sourceCodeEnd":414,"githubUrl":"https://github.com/ruvnet/ruflo/blob/6b01dc5a687b26b3e218f796de45ec51f8fa9e8c/v3/@claude-flow/cli/src/services/global-ai-budget.ts#L378-L414","documentation":"Thrown by GlobalAiBudget.acquireLock() when the O_EXCL lock file (~/.claude-flow/ai-budget.lock) cannot be acquired within a 2-second deadline. The budget fuse is intentionally strict — failure to account means denial — so lock contention directly blocks launches. Stale locks older than LOCK_STALE_MS are taken over automatically.","triggerScenarios":"Multiple processes/ci jobs calling reserve() concurrently against the same budget dir; a process holding ai-budget.lock for >2s (slow I/O on a network home dir); heavy concurrent agent spawning across workspaces.","commonSituations":"Many parallel swarm agents launching from the same $HOME; a network-mounted home directory with high latency; a dead process whose lock hasn't aged past LOCK_STALE_MS yet; the ledger write/fsync being slow.","solutions":["Reduce concurrent launch bursts (respect maxConcurrentGlobal).","Identify the holder via the PID in ai-budget.lock and confirm it is alive.","Move the budget dir to fast local storage (set RUFLO_AI_BUDGET_DIR).","If the holder is a confirmed dead process and the lock is stale, remove it manually.","As an EMERGENCY escape hatch only, set RUFLO_AI_BUDGET_DISABLE=1 (this disables all accounting)."],"exampleFix":"# diagnose\nps -p \"$(cat ~/.claude-flow/ai-budget.lock)\"\n# if dead and stale, remove\nrm ~/.claude-flow/ai-budget.lock\n# or move budget dir to local storage\nexport RUFLO_AI_BUDGET_DIR=/var/tmp/ai-budget","handlingStrategy":"retry","validationCode":"function budgetLockStale(lockFile: string, staleMs: number): boolean {\n  try { return Date.now() - fs.lstatSync(lockFile).mtimeMs > staleMs; }\n  catch { return false; }\n}","typeGuard":null,"tryCatchPattern":"try {\n  await budget.reserve(req);\n} catch (e) {\n  if (e instanceof Error && /timed out acquiring ai-budget lock/.test(e.message)) {\n    // emergency only — otherwise surface a 'budget system busy' denial\n    throw new Error('AI budget lock contention — reduce concurrent launches or retry');\n  }\n  throw e;\n}","preventionTips":["Set RUFLO_AI_BUDGET_DIR to fast local storage (not a network home dir).","Respect maxConcurrentGlobal — do not launch more agents than the budget allows.","Monitor for zombie processes holding ai-budget.lock.","Keep RUFLO_AI_BUDGET_DISABLE=1 reserved for true emergencies only."],"tags":["concurrency","locking","budget","timeout","filesystem"],"backgroundTag":null,"analyzedSha":"6b01dc5a687b26b3e218f796de45ec51f8fa9e8c","analyzedAt":"2026-08-12T13:20:50.148Z","schemaVersion":2},"datasetVersion":"2026-08-12T18:17:37.767Z"}