abhigyanpatwari/GitNexus · critical · AggregateError

Guard and workload-lock cleanup failed

Error message

Guard and workload-lock cleanup failed: ${guardPath}. Acquisition refused; see RUNBOOK.md for quiesced recovery.

What it means

This AggregateError is thrown by releaseAcquisitionGuard when, after the guard-verification step already failed (guardError), rolling back this attempt's own workload-lock record also fails (cleanupError). The acquisition is refused (the pending handle is discarded) but the filesystem may be left with a token-exact analyze.lock or an unverifiable guard; recovery requires quiescing all writers and following RUNBOOK.md. It reports BOTH failures together so neither is masked.

Solutions

1. Quiesce all gitnexus writers on this repo (no analyze running), then follow RUNBOOK.md quiesced recovery: verify/remove the leftover analyze.lock and analyze.lock.guard and retry.
2. Check filesystem health: free disk space (ENOSPC), permissions on .gitnexus/ (EACCES/EPERM), and mount writability.
3. Exclude the .gitnexus/ directory from antivirus/backup/sync tools that race file creation and deletion.
4. Move the index off NFS/network shares to a local filesystem if errors persist.
5. If the directory is badly corrupted, re-create .gitnexus/ from scratch (clean and re-run analyze) once no writers are active.

Example fix

// before (guard/lock files left in a synced dir)
// index at ~/Dropbox/repo/.gitnexus — sync client races unlink

// after (local, excluded from sync)
$ gitnexus analyze  # with .gitnexus/ on a local disk, e.g. GITNEXUS_STORAGE_PATH=~/.local/share/gitnexus/repo

// Pre-flight: index dir writable, not read-only, has space
import { accessSync, constants, statfsSync } from 'node:fs';
try {
  accessSync('.gitnexus', constants.W_OK);
  const { bavail, bsize } = statfsSync('.gitnexus');
  if (bavail * bsize < 10 * 1024 * 1024) throw new Error('low disk space');
} catch (e) {
  console.error('index dir not writable or low on space:', e);
}

const isGuardCleanupAggregate = (e: unknown): e is AggregateError & { errors: [unknown, unknown] } =>
  e instanceof AggregateError &&
  typeof e.message === 'string' &&
  e.message.startsWith('Guard and workload-lock cleanup failed');

try {
  await acquireIndexLock(lockDir);
} catch (err) {
  if (isGuardCleanupAggregate(err)) {
    const [guardError, cleanupError] = err.errors;
    console.error('acquisition refused; quiesce writers and follow RUNBOOK.md', guardError, cleanupError);
    return; // never proceed without the lock
  }
  throw err;
}

When it happens

Trigger: During acquireViaFile's finally block: guard verification threw (foreign token, unreadable, or vanished guard), createdMain was true (this attempt had created analyze.lock), and the rollback unlinkSync/readRecord on the workload lock itself threw (I/O error, EACCES/EPERM on unlink, file replaced mid-read).

Common situations: Read-only or full index directory (ENOSPC/EACCES) breaking both guard writes and lock cleanup; concurrent processes or antivirus/backup tools interfering with files under .gitnexus/; NFS/network mounts where unlink semantics are unreliable; a corrupted .gitnexus directory from a prior crash.

Related errors

• Workload-lock cleanup failed
• Clone failed and partial checkout could not be quarantined
• Could not remove the shadowed branch sub-index; keeping its…
• Index lock acquisition timed out
• Index lock verification failed

AI-assisted analysis of abhigyanpatwari/GitNexus@ac9a4e9abd (2026-09-15). Data as JSON: /api/errors/09140f2296fcc70c. Report an issue: GitHub.

Appendix: source

Thrown at gitnexus/src/storage/index-lock.ts:380

      lstatSync(guardPath);
    } catch (err) {
      if ((err as NodeJS.ErrnoException).code === 'ENOENT') {
        throw unverifiedGuardError(guardPath);
      }
      throw err;
    }
    // Exists but unreadable: this process created the name via O_EXCL.
    // Drop it so a failed metadata write cannot leave a permanent orphan,
    // then refuse this attempt.
    unlinkSync(guardPath);
    throw unverifiedGuardError(guardPath);
  } catch (guardError) {
    // Roll back only the token-exact record this attempt created.
    if (createdMain) {
      try {
        if (readRecord(lockPath)?.token === me.token) unlinkSync(lockPath);
      } catch (cleanupError) {
        throw new AggregateError(
          [guardError, cleanupError],
          `Guard and workload-lock cleanup failed: ${guardPath}. Acquisition refused; see RUNBOOK.md for quiesced recovery.`,
        );
      }
    }
    throw guardError;
  }
};

/**
 * Filesystem-create errors eligible for a read-only, non-owning handle when
 * neither lock nor guard exists. Denied creation does NOT prove other writers
 * lack access (ACLs may differ). Callers that write must reject lockFree handles.
 */
export const LOCK_UNWRITABLE_CODES: ReadonlySet<string> = new Set(['EROFS', 'EACCES', 'EPERM']);
export const isLockUnwritableCode = (code: string | undefined): boolean =>
  code !== undefined && LOCK_UNWRITABLE_CODES.has(code);

View on GitHub (pinned to ac9a4e9abd)