ruvnet/ruflo · error · Error

timed out acquiring repo-supervisor lock

Error message

timed out acquiring repo-supervisor lock

What it means

Thrown by RepoSupervisorRegistry.withLock() when an O_CREAT|O_EXCL lockfile (<record>.json.lock) cannot be created within a hard 2-second deadline (25ms retry loop). A lock left by a crashed process is auto-reclaimed once older than LOCK_STALE_MS (10s), so this error means another daemon held a *fresh* lock for the entire 2s window — the election body (fn) runs while holding the lock, so a slow or contended critical section on a peer extends hold time.

Source

Thrown at v3/@claude-flow/cli/src/services/repo-supervisor.ts:121

      try {
        const fd = fs.openSync(lockFile, fs.constants.O_CREAT | fs.constants.O_EXCL | fs.constants.O_WRONLY, 0o600);
        fs.writeSync(fd, String(process.pid));
        fs.closeSync(fd);
        try {
          return fn();
        } finally {
          try { fs.unlinkSync(lockFile); } catch { /* already gone */ }
        }
      } catch (e) {
        if ((e as NodeJS.ErrnoException).code !== 'EEXIST') throw e;
        try {
          const st = fs.lstatSync(lockFile);
          if (Date.now() - st.mtimeMs > LOCK_STALE_MS) {
            fs.unlinkSync(lockFile);
            continue;
          }
        } catch { /* raced — retry */ }
        if (Date.now() > deadline) throw new Error('timed out acquiring repo-supervisor lock');
        await delay(25);
      }
    }
  }

  private readRecord(repositoryId: string): SupervisorRecord | null {
    const file = this.fileFor(repositoryId);
    assertNotSymlink(file);
    if (!fs.existsSync(file)) return null;
    try {
      const raw = JSON.parse(fs.readFileSync(file, 'utf-8'));
      if (raw && typeof raw.pid === 'number' && typeof raw.lastHeartbeat === 'number') {
        return raw as SupervisorRecord;
      }
    } catch { /* corrupt — treat as absent */ }
    return null;
  }

View on GitHub (pinned to fa13ee4ad6)

Solutions

1. Simplest: do nothing this tick — the daemon retries election on its next lifecycle tick; contention resolves itself once one daemon wins.
2. Stagger daemon startup (jittered delays) so N worktrees don't all contest the lock in the same 2s window.
3. Move RUFLO_AI_BUDGET_DIR (and thus the registry) to local disk instead of NFS/network home to make the critical section fast.
4. If it persists: lsof/stat the .lock file — a fresh mtime means a live peer is slow (reduce concurrent daemons); an old mtime means the 10s stale-reclaim will clear it within seconds, so just wait.

Example fix

// before: N worktrees start daemons at once
await Promise.all(worktrees.map(w => startDaemon(w))); // several throw 'timed out acquiring repo-supervisor lock'

// after: jittered startup lets one daemon win, others participate
await Promise.all(worktrees.map(async (w, i) => {
  await new Promise(r => setTimeout(r, i * 250));
  await startDaemon(w); // losers just retry next tick
}));

import * as fs from 'fs';

// best-effort pre-check: is the lock free or stale (reclaimable) right now?
function lockAcquirable(lockFile: string, staleMs = 10_000): boolean {
  try { return Date.now() - fs.lstatSync(lockFile).mtimeMs > staleMs; }
  catch (e) { return (e as NodeJS.ErrnoException).code === 'ENOENT'; }
}

try {
  return await registry.tryAcquireSupervision(worktreeRoot);
} catch (e) {
  if (e instanceof Error && e.message === 'timed out acquiring repo-supervisor lock') {
    // Contention, not corruption: another worktree's daemon holds a fresh lock.
    // Skip this tick — the next lifecycle tick (~60s) retries election.
    return { isSupervisor: false, record: null };
  }
  throw e;
}

When it happens

Trigger: Many worktrees of the same repository starting daemons simultaneously (each ticks the election); the peer inside withLock does slow fs work on a cold NFS/lazy network home; the Node event loop of the lock holder is blocked, delaying its unlink; a just-crashed process left a lock younger than 10s.

Common situations: Monorepos checked out as 10+ git worktrees with a daemon in each; RUFLO_AI_BUDGET_DIR on NFS where fs.writeSync/unlinkSync latency is high; CI machines fan-out starting daemons at the same instant; the 2s deadline being shorter than a peer's legitimate critical section under heavy load.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• timed out acquiring ai-budget lock
• timed out acquiring flywheel attempts lock
• timed out acquiring workspace-lease lock
• timed out acquiring flywheel transaction lock
• swarm state is busy; retry the outcome update

AI-assisted analysis of ruvnet/ruflo@fa13ee4ad6 (2026-08-18). Data as JSON: /api/errors/3b65bb00bdd493e9. Report an issue: GitHub.