{"record":{"id":"09140f2296fcc70c","repo":"abhigyanpatwari/GitNexus","slug":"guard-and-workload-lock-cleanup-failed-guardpath-acquisition","errorCode":null,"errorMessage":"Guard and workload-lock cleanup failed: ${guardPath}. Acquisition refused; see RUNBOOK.md for quiesced recovery.","messagePattern":"Guard and workload-lock cleanup failed: (.+?)\\. Acquisition refused; see RUNBOOK\\.md for quiesced recovery\\.","errorType":"exception","errorClass":"AggregateError","httpStatus":null,"severity":"critical","filePath":"gitnexus/src/storage/index-lock.ts","lineNumber":380,"sourceCode":"      lstatSync(guardPath);\n    } catch (err) {\n      if ((err as NodeJS.ErrnoException).code === 'ENOENT') {\n        throw unverifiedGuardError(guardPath);\n      }\n      throw err;\n    }\n    // Exists but unreadable: this process created the name via O_EXCL.\n    // Drop it so a failed metadata write cannot leave a permanent orphan,\n    // then refuse this attempt.\n    unlinkSync(guardPath);\n    throw unverifiedGuardError(guardPath);\n  } catch (guardError) {\n    // Roll back only the token-exact record this attempt created.\n    if (createdMain) {\n      try {\n        if (readRecord(lockPath)?.token === me.token) unlinkSync(lockPath);\n      } catch (cleanupError) {\n        throw new AggregateError(\n          [guardError, cleanupError],\n          `Guard and workload-lock cleanup failed: ${guardPath}. Acquisition refused; see RUNBOOK.md for quiesced recovery.`,\n        );\n      }\n    }\n    throw guardError;\n  }\n};\n\n/**\n * Filesystem-create errors eligible for a read-only, non-owning handle when\n * neither lock nor guard exists. Denied creation does NOT prove other writers\n * lack access (ACLs may differ). Callers that write must reject lockFree handles.\n */\nexport const LOCK_UNWRITABLE_CODES: ReadonlySet<string> = new Set(['EROFS', 'EACCES', 'EPERM']);\nexport const isLockUnwritableCode = (code: string | undefined): boolean =>\n  code !== undefined && LOCK_UNWRITABLE_CODES.has(code);\n","sourceCodeStart":362,"sourceCodeEnd":398,"githubUrl":"https://github.com/abhigyanpatwari/GitNexus/blob/ac9a4e9abd8fd3058c070b72c23402a4f887929a/gitnexus/src/storage/index-lock.ts#L362-L398","documentation":"This AggregateError is thrown by releaseAcquisitionGuard when, after the guard-verification step already failed (guardError), rolling back this attempt's own workload-lock record also fails (cleanupError). The acquisition is refused (the pending handle is discarded) but the filesystem may be left with a token-exact analyze.lock or an unverifiable guard; recovery requires quiescing all writers and following RUNBOOK.md. It reports BOTH failures together so neither is masked.","triggerScenarios":"During acquireViaFile's finally block: guard verification threw (foreign token, unreadable, or vanished guard), createdMain was true (this attempt had created analyze.lock), and the rollback unlinkSync/readRecord on the workload lock itself threw (I/O error, EACCES/EPERM on unlink, file replaced mid-read).","commonSituations":"Read-only or full index directory (ENOSPC/EACCES) breaking both guard writes and lock cleanup; concurrent processes or antivirus/backup tools interfering with files under .gitnexus/; NFS/network mounts where unlink semantics are unreliable; a corrupted .gitnexus directory from a prior crash.","solutions":["Quiesce all gitnexus writers on this repo (no analyze running), then follow RUNBOOK.md quiesced recovery: verify/remove the leftover analyze.lock and analyze.lock.guard and retry.","Check filesystem health: free disk space (ENOSPC), permissions on .gitnexus/ (EACCES/EPERM), and mount writability.","Exclude the .gitnexus/ directory from antivirus/backup/sync tools that race file creation and deletion.","Move the index off NFS/network shares to a local filesystem if errors persist.","If the directory is badly corrupted, re-create .gitnexus/ from scratch (clean and re-run analyze) once no writers are active."],"exampleFix":"// before (guard/lock files left in a synced dir)\n// index at ~/Dropbox/repo/.gitnexus — sync client races unlink\n\n// after (local, excluded from sync)\n$ gitnexus analyze  # with .gitnexus/ on a local disk, e.g. GITNEXUS_STORAGE_PATH=~/.local/share/gitnexus/repo","handlingStrategy":"try-catch","validationCode":"// Pre-flight: index dir writable, not read-only, has space\nimport { accessSync, constants, statfsSync } from 'node:fs';\ntry {\n  accessSync('.gitnexus', constants.W_OK);\n  const { bavail, bsize } = statfsSync('.gitnexus');\n  if (bavail * bsize < 10 * 1024 * 1024) throw new Error('low disk space');\n} catch (e) {\n  console.error('index dir not writable or low on space:', e);\n}","typeGuard":"const isGuardCleanupAggregate = (e: unknown): e is AggregateError & { errors: [unknown, unknown] } =>\n  e instanceof AggregateError &&\n  typeof e.message === 'string' &&\n  e.message.startsWith('Guard and workload-lock cleanup failed');","tryCatchPattern":"try {\n  await acquireIndexLock(lockDir);\n} catch (err) {\n  if (isGuardCleanupAggregate(err)) {\n    const [guardError, cleanupError] = err.errors;\n    console.error('acquisition refused; quiesce writers and follow RUNBOOK.md', guardError, cleanupError);\n    return; // never proceed without the lock\n  }\n  throw err;\n}","preventionTips":["Keep .gitnexus/ on a local, writable filesystem with free space.","Exclude .gitnexus/ from sync/backup/AV tools that race create/unlink.","Quiesce and inspect after any AggregateError — leftover guard/lock files need RUNBOOK recovery.","Monitor disk space and mount health on CI runners."],"tags":["filesystem","locking","cleanup","atomicity"],"backgroundTag":"lock-cleanup-failed","analyzedSha":"ac9a4e9abd8fd3058c070b72c23402a4f887929a","analyzedAt":"2026-09-15T23:29:44.066Z","contentChangedAt":"2026-09-15T23:29:44.066Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}